Project Management Institute

Operational measurements for product development organizations - Part 1

by Tony Rizzo


WE MEASURE MANY things in product development organizations. We even have a balanced scorecard of measurements. But, of all the things we measure in virtually all product development organizations today, none is an effective operational measurement.

Don't be led astray. Put the right measurements in their proper place.

What's an operational measurement, and how does it differ from our current measurements? Let's use an automobile analogy. Imagine that you are driving down a country road. You're moving at a fast clip in your Dodge Viper—you can dream, can't you? Suddenly you realize you're approaching a curve. The road sign alerts you that the recommended speed limit for the curve is 40 miles per hour. Your eyes jump to the speedometer for an instant. On the basis of that instantaneous measurement of speed, you choose your course of action for the next two seconds. That's an operational measurement.

How does this differ from what we might otherwise call a performance measurement? It's simple. The average speed for your entire trip is a performance measurement. So is the average fuel consumption rate. But, the instantaneous readings of your speedometer, coolant temperature gauge, oil pressure gauge, and tachometer are operational measurements. These, among others, become the basis for the countless on-the-spot decisions you make during your joy ride.

Tony Rizzo is an internal management consultant with Lucent Technologies, where he specializes in TOC-based consulting and training. He previously functioned as an internal finite element analysis consultant with Bell Labs.

The effective operation of every system requires the use of the right operational measurements. Our product development systems are no exception. Unfortunately, these systems often are run not with the use of operational measurements but with the use of performance measurements. By pretending these performance measurements can be used as operational measurements, we cause ourselves to make many decisions that actually damage our organization's performance. Let's take a closer look at some of the so-called operational measurements that drive many of our day-to-day decisions.

Business Measurements

Business measurements are among the most frequently used measurements. These include revenues, cost, profit, return on investment and return on net assets. Is there anything wrong with these? Absolutely not! They are most necessary for external reporting purposes. The Internal Revenue Service, Wall Street analysts, and stockholders are very interested in these measurements. But, do they help managers make day-to-day decisions?

Consider this: Business measurements are taken over extended periods, such as a month or a quarter. Consequently, they provide no information about the instantaneous state of the organizational system. Imagine trying to drive in the Indianapolis 500 with a tachometer that displays not current engine speed, but only the average engine speed for the previous lap. Could you negotiate a curve effectively with such information? Could you decide when to shift gears?

For all their worth, business measurements make exceedingly poor operational measurements because the interval over which they are taken is far too long. Like average lap speed and average fuel consumption, they provide good estimates of system performance over an extended period. But, just as the latter do nothing to support a driver's short-term decision-making process, business measurements do nothing to support a manager's short-term decision-making process. Business measurements are not operational measurements.

Averaged Local Performance Measurements

Another situation we encounter frequently is the use of averaged local performance measurements, which we apply to pieces of an organization, such as a drafting department or a software development group. Examples of these include the number of lines of code, number of widgets designed, drawings completed, and purchase orders processed. Often, these local performance measurements are averaged over an extended period. For example, we might measure the performance of a drafting department in terms of drawings completed per month.

Are these effective operational measurements? Imagine you're flying an aircraft. Your altimeter gives you a reading not of current altitude but of your average altitude over the last 100 miles of flight. Would you be able to fly your plane safely, or in cloudy weather?

Again, these performance measurements may be necessary and useful for the purpose of reporting to those who are external to the local organization, but they do nothing to support the day-to-day decision-making process of the managers in that organization. Consequently, they are not effective operational measurements.

Normalized Local Performance Measurements

One damaging variant of the local performance measurement is the normalized local performance measurement. Examples of such measurements include drawings per person per month, lines of code per person per month, and so forth. These often cause a severe problem: they create denominator managers.

Whenever a department manager is held accountable for such a normalized measurement, he or she feels significant pressure to prevent the measurement's degradation. Unfortunately, the manager never has much control over the numerator of the ratio that is the normalized measurement. For example, there is little the manager of a drafting department can do to generate more demand for her people's services. Similarly, there is virtually nothing the manager of a purchasing department can do to increase the number of purchase requisitions that others in the corporation generate. Hence, whenever the numerator of the measurement decreases, the manager can respond only with attempts to decrease the denominator. This is denominator management. This also is how considerable damage is done to an organization.

Reader Service Number 084

Typically, there are only two variables represented in the denominator. One is the length of the measurement interval, such as a month or a quarter; the manager can do nothing to alter this variable. The other variable in the denominator, usually, is the number of people in the department. If the manager is held accountable for the ratio, and if the manager is able to affect neither the numerator nor one of the two variables in the denominator, he or she feels pressure to maintain the ratio by decreasing the remaining variable in the denominator. Of course, people may not be the only operative denominator variable. Cost dollars might be another variable. So might be investment dollars or training dollars. Continual changes in any of these variables are indicative of denominator management.

Are such normalized measurements effective operational measurements? A trucking analogy may answer this question. Let's say you operate a truck, and you are held accountable for maintaining the monthly average ratio of payload weight to vehicle weight. (Benchmark data for your industry has been given to you by top management; you are expected to be bet-ter than average with this measurement.) Usually, your major client has you haul his truck engines, which are quite heavy; you have no difficulty maintaining your normalized measurement. But one month your major client goes away and is replaced by a manufacturer of mattresses; you are still held accountable for maintaining the ratio of payload weight to vehicle weight. Would this measurement help you to make better day-to-day decisions regarding the operation of your truck? Clearly, it would not. Would it make sense for you to remove the fenders from your truck to maintain the ratio? Clearly, it would not. Yet, many managers are continually held accountable for a variety of similar measurements; their organizations suffer the consequences.

The effect is not surprising. A department's capacity to do useful work is reduced periodically to prevent the measurement from degrading. Inevitably, the department's performance is ratcheted downward until it is no longer able to meet the needs of the rest of the organizational system. The outcome often is that millions of dollars of business are delayed by a function like drafting, or purchasing, or shipping and receiving. When this happens, a very small tail can end up wagging a huge corporate dog.

Reader Service Number 059

Cost-Focused Measurements

We measure, record, track, (try to) manage, and generally worship cost minimization everywhere. Cost is king among measurements. We have armies of cost accountants to measure this one thing. We've even invented an entire profession for the purpose. Industries have been created primarily to measure and use cost figures for decision-making. (Is your organization using activity-based costing or activity-based management yet?) So, it's only natural that some of us try to use cost as the primary operational measurement.

Does it make sense to use cost this way? Let's go back to that aircraft example. If you were flying a plane, the equivalent of this cost focus would have you measure, monitor, record, and report fuel consumption religiously. Your instruments would allocate fuel consumption to every subsystem in the aircraft. Your airspeed indicator would read not airspeed but a fuel-consumption equivalent of airspeed, averaged over the last 100 miles. Altitude would be reported in an equivalent fuel-consumption rate as well.

But your cost-focused aircraft management system wouldn't stop there. Your instruments’ readouts also would reflect accrual accounting rules. The airspeed indicator would read the fuel consumption rate equivalent of airspeed averaged over the last 100 miles, but this reading would be modified to account for the speed that you'd expect to achieve during the next 50 miles. And, don't forget to take depreciation into account as well.

The situation is laughable. No! It would be laughable were it not so close to home. I paraphrase a colleague: We measure things that are mere observable quantities, and we act as if we could control them directly. Then, we corrupt even these measurements with our cost accounting practices.

The only conclusion we can draw from all this is that we have no effective operational measurements. Managers are flying by the seat of their pants, and frequently they scrape the treetops. As one listener noted during a recent conference by D.H. Brown Associates, this is precisely the reason why we invented the acronym C.Y.A.

A Sure Sign

It's easy to determine if your organization's managers and performance suffer from the wrong measurements. Improper measurements cause widespread conflicts throughout an organization. Look for the conflicts.

Consider the manager of a drafting department measured by the internal charges that the department collects from internal clients—this is a rather widespread funding model. The manager constantly faces a horrible conflict between charging a high rate for her people's drafting services and not charging a high rate. To meet the measurement to which she is held—that is, to keep her department alive—the manager must set a high internal charge rate. At the same time, to avoid discouraging project teams from using her people's necessary services, the manager must set a low rate or no rate at all.

It's clear that such a manager wants nothing more than to do the best for her employer. The manager also understands that under ideal circumstances her people would provide whatever drafting services the company's project teams needed. Yet, the manager is forced to charge a high rate internally, occasionally preventing some project teams from utilizing her people's services effectively. It is this realization that brutally assaults the manager's sense of logic and creates her horrible conflict.

There are many instances when managers, engineers, and others resolve such conflicts heroically, in favor of the greater organization; by doing so they compromise their measurements. But, such heroic behavior is self-limiting, because heroes ultimately get killed. By compromising her measurement, the heroic manager exposes herself and her department to punitive action from higher levels. If she persists in her heroic decisions, she soon finds herself without a department and without a job. In the long term, the measurement prevails. The manager, or her replacement, resolves the conflict in favor of survival and charges the higher rate, no matter how many projects are delayed for lack of prompt drafting services.

Reader Service Number 085

It's clear that we cannot run our expensive, complex, product development organizations effectively with such erroneous, cost-focused measurements. We don't need cost-focused measurements to ensure that the vast majority of day-today decisions throughout a product development organization are consistent with that organization's defined goal. What we need is goal-focused, operational measurements!

It is this need that brings us to the next question: What should be the defined goal of a product development organization?

The Goal of a Product Development Organization

If your company lives or dies by its ability to bring new products to market faster than its competitors, the following statements are true for your company:

images Product quality, value, reliability, service quality, and so forth, are all necessary conditions. These must meet customer expectations; otherwise, your company begins to lose customers and money.

images Each new product is nothing short of a money pump for your company. New products create entirely new cash flows, pumping cash from the market into the company.

images Your stockholders want your company to create high-capacity money pumps. Therefore, your new products must bring innovative, powerful solutions to your customers.

images Once all the above necessary conditions are met, your stockholders want your company to turn on many new money pumps as soon as possible. Therefore …

images The goal of your product development organization, which is the source of new money pumps for your company, is SPEED!

Some may argue with this conclusion, claiming that a product development organization's goal is quality or innovation. To be sure, these are very important, necessary conditions. If your company's products are of poor quality or don't solve your customers’ more severe problems, then your company's customers spend their money with your competitors. However, given the requisite levels of quality, innovation, and so forth, the company that reaches the market first with a competitive product line gets the lion's share of profits. A company that is late to market with a similar product line may not even recover the product's development costs. Therefore, for every new-product development organization, speed is the goal.

But how can we translate this goal into a set of operational measurements that drive the correct day-to-day decisions throughout the product development organization? This is where the Theory of Constraints and the genius of Eliyahu M. Goldratt come in.

THE FIRST STEP toward translating the goal of speed into a set of operational measurements that drive the correct day-to-day decisions requires that we understand Goldratt's Critical Chain Method. The critical chain method is a strategy for planning and managing individual projects. Once we've covered the critical chain method, we'll be prepared to discuss Goldratt's enter-prisewide resource management solution for product development organizations, i.e., the TOC multiproject management method. As we'll see in Part 2, this is a system approach to managing an entire product development organization; it defines the optimum set of operational measurements for product development. images

Reader Service Number 086

This material has been reproduced with the permission of the copyright owner. Unauthorized reproduction of this material is strictly prohibited. For permission to reproduce this material, please contact PMI.

November 1999 PM Network



Related Content