Teaching to the Test

In the teaching profession, there is an ongoing debate over the value of standardized testing – testing designed to measure the performance of both the student and the educator. One of the more powerful arguments against standardized testing – or at least standardized testing in its current form – does not relate to the testing itself but to one of its unintended consequences. That unintended consequence is “teaching to the test” instead of teaching a core curriculum and critical thinking skills.

“Teaching to the test” means that the test becomes an end in itself. It means teachers focus on the content that will likely be on the test and use the format of the test as a basis for teaching. Since this kind of teaching leads primarily to improved test-taking skills, increases in test scores do not necessarily mean improvement in real academic performance. “Teaching to the test” also narrows the curriculum, forcing teachers and students to concentrate on memorization of isolated facts instead of developing fundamental and higher order abilities. What difference does all this make? Answer this simple question: When you have a serious illness, would you rather have a doctor with an in-depth understanding of his or her medical specialty or one who was good at memorizing isolated facts and taking tests?

In the same way teachers are pressured to “teach to the test,” managers of businesses are pressured to “manage to the test” – the test being the upcoming period’s GAAP-based financial results. “Managing to the test” means that the company’s GAAP-based financial results become an end in itself. It means managers focus on the issues that will likely impact the upcoming test and use the format of the test as a basis for measuring performance. Since this kind of managing leads to the best possible short-term GAAP-based test results, improvement in test scores does not necessarily mean improvement in the long-term financial health of the organization. “Managing to the test” narrows manager’s focus, forcing them to manipulate near-term events instead of taking actions that will add maximum value to the organization over the long run.

Not only is managers’ performance evaluated against such test results, but their compensation is also often based on those results. As a consequence, how can you blame decision makers for making decisions that result in good near-term test scores, but lessen the organization’s ability to succeed – or even survive – in the future?

This “managing to the test” problem exists at all levels of an organization. Whether it’s a plant supervisor who continues to make parts of questionable quality because his “test” is based on direct-labor efficiency or a CEO who defers maintenance, cuts back on research and development and accepts low-margin jobs because her “test” is based on EBITDA, the result is the same, the mid- and long-term underperformance of the organization.

This problem is not new. One of its most common manifestations is “the hockey stick syndrome.” This problem occurs when managers “put on a kick” at the end of each reporting period – month, quarter, and/or year – in an attempt to meet the period’s performance targets.

I can recall instances of plant managers pulling workers off of their normal assignments to complete jobs that were not due to be shipped for weeks but that were nearing their standard cost “pay points” so that they could absorb more overhead and show greater efficiency during a period. I also recall many instances of shipping ahead of schedule to get sales into a period and then having the customer return the pre-shipped goods – at the shipper’s expense – during the next period, keeping the sales book open an extra day or two to increase sales, paying dunnage to hold purchased items outside the plant grounds so they wouldn’t show up in inventory, building unneeded stocks of finished goods to absorb overhead, and pulling many other “stunts” that damaged the company’s longer-term results in order to improve the upcoming period’s test score.

In all instances, the hockey stick syndrome puts the business’ operation out of balance for the beginning of the subsequent period making it necessary to refill the pipeline and balance the operation before moving forward in any kind of efficient manner. The net result is less than optimum performance over the long term in order to sustain the illusion of good performance during each individual period. The cumulative performance of the business will always suffer

Another “managing to the test” problem surfaced at a company that created an incentive plan for front-line supervisors that would pay them a bonus of up to 25% based on three performance measures: direct labor efficiency, spending against a flexible budget, and the plant’s scrap percentage. For these individuals, managing to the test not only had an impact on the boss’ view of their performance, it had a significant impact on their pocketbooks. It didn’t take long for these front-line supervisors to realize that it was next to impossible for them to earn the full 25% bonus every month – balancing these three performance measures was just too difficult. If, however, they generated a lot of scrap – which improved their efficiency and increased their flexible budget – they found they could get an 18% bonus almost every month. To solve their scrap problems, they would have to reduce their direct labor efficiency and the level of their flexible budget (which was based on labor hours earned), so they figured they would sacrifice the company’s overall performance to optimize their personal bonuses.

When this problem exists among the organization’s C-level executives it usually permeates throughout the entire organization. At many of today’s companies owned by private-equity firms, EBITDA is the primary measurement by which the company’s C-level executives are judged. This measure leads to a myriad of high-level decisions that threaten the organization’s long-term viability. It leads to deterioration in the company’s capital asset base, failure to develop new products, services, or processes, deferral of critical maintenance, acceptance of marginal orders, and adoption of questionable accounting practices. The lower-level managers working within the organization are then measured by how well they help damage the company’s long-term success and do everything they can to score high on the upcoming test.

Generally accepted accounting principles (GAAP) are designed to record the activities and events that took place between two balance sheet dates, not to reveal how the organization actually performed. Consider a company that requires an annual expenditure of $5 million on research and development if it is to develop the new products that will enable it to maintain its existing volume of business. Under pressure to pass its financial test one year it decides to cut research and development to $4 million. This move increases its earnings by $1 million and enables it to pass the test. But did GAAP actually reveal how well the company performed? Does it show that the company put its future in jeopardy? Does it indicate that it has borrowed $1 million from the future in order to pass this year’s test and will probably have to spend at least an additional $1 million plus to catch up in future test periods? Has management actually done something to make the company a more viable business in the future?

Consider another organization that accepts several new long-term programs based on an incremental pricing strategy in order to improve current year profits. The revenues from these programs will cover all of the variable costs required to produce the programs’ products and contribute toward – but not fully cover – the fixed cost of resources used in producing the products. This move will improve the company’s GAAP-based profitability in the year the programs begin. But it will also tie up capacity and preclude the company from using that capacity to produce much more profitable products during the remaining years of the programs. Did GAAP actually reveal how well the company performed? Does it show that the company has dedicated valuable capacity to produce marginally profitable – or maybe unprofitable products – far into the future? Does it highlight how valuable capacity was borrowed from the future in order to pass this year’s test? Has management actually done something to make the company a more viable business in the future?

There is a myriad of actions management can take in a given year to help the company’s current GAAP-based results pass the upcoming test. In most instances, those actions only make it more difficult for the company to perform well in the future – a future that will be continue to be driven by the desire to pass upcoming tests, not the desire to lead the company into a successful future. Managing to the test turns decision makers into “game players” instead of “stewards” that work for the long-term success of the organization.

The Profitability Analytics Framework provides decision makers with a comprehensive guide for developing the type of quality decision support information they need to guide their organizations toward a sustainable, financially successful future.