Using unders to offset overs
by Arnold M. Ruskin, PMP
UNCERTAINTY ABOUNDS IN many development projects. One cannot estimate with confidence in advance the duration or cost of many critical steps. Indeed, one may be satisfied just to bound duration or cost, and identify the most likely values.
Nevertheless, commitments need to be made to corporate sponsors or customers—commitments upon which they will make further promises and to which you will be held accountable. Given this situation, how can you make plausible commitments that are neither impossible to meet nor based upon a sum of worst cases that will make your project noncompetitive or unacceptable? The key lies in using unders in some areas to offset overs in other areas.
I'll describe a process for arranging appropriate under-over offsets from the beginning, for making appropriate commitments, and for setting and managing meaningful contingency allowances. I will also examine a hazard in requiring component parts of the project to make firm commitments in an attempt to ensure that you can meet your own commitments.
The Common Problems
Here are 10 common problems in estimating effort (E), duration (D), and cost (C) and in making project commitments:
Arnold M. Ruskin, PMP, Ph.D., P.E., CMC, is a project management practitioner, coach, consultant and trainer, with 30+ years of industrial and consulting experience. A founding partner of Claremont Consulting Group, his clients range from small entrepreneurs to Fortune 500 companies and government agencies. He has written over 35 papers on project management, engineering management, and engineering, and three books. He is a member of the Project Management Journal editorial review board and the Engineering Management Journal editorial board.
Exhibit 1. Determining probability distribution functions is one of the theoretical underpinnings of using unders to offset overs to help eliminate uncertainty in development projects.
Exhibit 2. A triangular distribution is used to estimate the shape of distribution curves when figuring effort, duration, or cost for an individual task.
1. Estimates are requested with little or no information about how the numbers will be used.
2. “Single point” estimates are the norm.
3. Estimators often give “most likely” values, perhaps with symmetrical +/- tolerances.
4. 50/50 estimates—for example, numbers that will be exceeded half the time—are requested with little regard to the breadth of distributions about these numbers.
5. Whatever estimates are provided, corporate or program management tells the estimators to reduce their numbers to be more competitive or to fit within perceived customer limits. However, management gives scant information about what to omit or de-scope.
6. Whatever EDC estimates are provided, project management takes some percentage when setting task allocations, leading estimators and management to play games.
7. Only rarely are risks identified, characterized, and assessed before commitments are made to upper levels in the organization or to the ultimate customer.
8. Project personnel are held in low regard if they exceed their EDC allocations (see 6 above)—no matter the reason.
9. Project personnel are held in high regard if they perform within their EDC estimates—no matter how generous the estimates or how wasteful they might be.
10. Commitments are made with little or no understanding of the likelihood of meeting the commitments or the organization's risk exposure.
If any of these descriptions fit your organization, improvements are possible in the way estimates are prepared and used in the pursuit of making and meeting realistic project commitments.
Here's an approach that avoids the above-described situations by providing and harvesting unders in some areas to offset overs in other areas. Thus, it enables appropriate and meaningful commitments that do not necessitate heroics for their accomplishment.
Our approach rests on the following five theoretical underpinnings.
Probability Distributions of Effort, Duration, and Cost for Individual Tasks. First, let's consider a typical probability curve for the effort, duration, or cost for an individual task (see Exhibit 1). We can imagine such distributions, but generally we don't really know the exact shape of the curves; thus, we have to approximate them. This we do with a triangular distribution (see Exhibit 2).
Triangular distributions are particularly nice because their geometry is well known. The Pythagorean theorem—the formula for a triangle's area, and so forth—makes it easy to determine the height of the triangle if two conditions are met: (1) the area of the triangle is set equal to 1.0, corresponding to 100 percent of the cases in which we are interested; (2) the triangle's minimum, most likely (i.e., the mode), and maximum values along the X-axis are estimated. The first of these conditions is met by decree; the second condition is also easily met, as follows.
Exhibit 3. The Monte Carlo method is used when estimating aggregate EDC distributions from individual EDC distributions—a process involved in the second theoretical underpinning of using unders to offset overs.
Exhibit 4. The CDF is the third theoretical underpinning of estimating EDC to reduce uncertainty in projects, representing the integral of the X probability distribution curve (0) as a function of X.
The minimum value along the X-axis corresponds to the most favorable conditions one can imagine, whereas the maximum value corresponds to the most outrageous combination of adverse circumstances one can imagine. The most likely value, or mode, is the value that usually comes to mind when one is asked for a quick singlepoint estimate.
Three-point estimates, as they are called, are relatively easy to make. In fact, most people who consider uncertainties in their work find three-point estimates easier to make than single-point estimates.
So now we have useful surrogates for our real EDC probability distributions.
Probability Distributions of Aggregate Effort, Duration, and Cost for an Ensemble of Individual Tasks—The Monte Carlo Method. The second theoretical underpinning involves estimating aggregate EDC distributions from the individual distributions of effort, duration, or cost. Because each of the individual task EDCs is probabilistic, we can't just add them together to get aggregate EDC distribution curves. Instead, we have to sample all the individual curves at once and record the aggregate value, then sample all of them a second time and record the aggregate value, then sample all of them a third time and record the aggregate value, and so forth. The aggregate totals are then plotted as a histogram whose envelope is the probability distribution for the totals (see Exhibit 3). This approach is called the Monte Carlo method. Upward of 5,000 sets of samples are generally drawn in order to avoid biases from inadequate sampling.
Exhibit 5. The percentage consumption of each contingency allowance can be plotted against the properly measured percentage accomplishment to figure the contingency allowance, although using unders to offset overs can eliminate the need for such allowances.
The Monte Carlo method is typically applied by using a computer simulation. Commercial off-the-shelf software packages are available to perform the simulation, and some project scheduling software tools include this capability. The capability can also be provided as an add-on to scheduling tools.
A minor note before proceeding: When calculating aggregate duration by the Monte Carlo method, the individual task durations for a given sample set must be run through a critical path calculation to determine the aggregate overall duration for that sample set. In this regard, having Monte Carlo capability coupled with the scheduling software is a major plus.
When the duration is calculated for a given sample set, the individual tasks that lie on the critical path for that sample set are noted. Because the durations of individual parallel tasks are probabilistic, their relative magnitudes may shift from one sample set to another. Thus, when parallel tasks exist, particular individual tasks are not guaranteed to always be on the critical path. In the end, we can say only that a given task has some percentage likelihood of being on the critical path.
Cumulative Probability Distributions. A third theoretical underpinning is the cumulative probability distribution function, typically called cumulative distribution function (CDF). The CDF gives the probability that the value of a variable will be less than some particular value. Mathematically, the CDF is the integral of the probability distribution curve of X, beginning at X equals zero, as a function of X (see Exhibit 4).
CDFs enable us to say, “We have an X likelihood of being able to do the project for such-and-such a budget,” or, “We have an X likelihood of being able to complete the project within a duration of such-and-such a time.” Alternatively, CDFs enable us to say, “If you want a such-and-so likelihood of success, the budget needs to be X1 and the duration needs to be X2.”
We can also have CDFs for individual task EDCs, based on their individual probability distributions.
Reader Service Number 023
Expected Value. A fourth theoretical underpinning is the notion of expected value. Expected value is the value of a variable that has a 50 percent likelihood of being exceeded and a 50 percent likelihood of not being reached. That is, the expected value of a variable is the one that corresponds to a CDF of 50 percent.
Consider a project that is broken into many individual tasks of similar magnitude in effort, duration, or cost. If half of them will be under their expected values and half of them over, the unders should be able to offset the overs, provided that two conditions occur:
1. The unders must be captured when they occur; for example, they must not be squandered. Some unders can be saved, such as money not spent. Time not spent, however, will be lost if the following task is unable to start when the preceding task is done.
Recognizing that rearranging resource assignments to satisfy this condition is difficult, we must nevertheless be prepared to replan in order to take advantage of unders. This also means that we must monitor not only problems but also progress, particularly opportunities to capture and deploy unders.
2. The unders in one area must be either directly useful or capable of being transformed into something that can be used in another area. Money is relatively versatile and can be used in many areas. However, a specialized talent that finishes early on one task might not be useful in other areas.
Contingency Allowances. A contingency allowance is a provision for work that is likely to occur but cannot be localized and identified in advance. It is not arbitrary, nor is it “pad” or “cushion.” Rather, it is a specific, calculated provision that enables a project manager to promise a likelihood of success that is greater than 50 percent.
The rationale for promising greater than 50 percent likelihood of project success is straightforward: For any number of reasons, the customer may not be able to use unders in some areas to offset overs in others. One reason is that the customer may be required to use fixed-price contracts, which essentially guarantees that no unders will occur. Another reason is that the customer may have too few projects to rely on using probabilities. A third reason is that the customer is very sensitive to overruns on one particular project or another and cannot argue that an overrun on one project is offset by un-derruns elsewhere. This latter condition may exist for legislative or political reasons quite independently of its technical merits.
Exhibit 6. Sometimes providing an additional allowance is the best solution to project difficulties.
The Use of Task EDC Distributions to Set Project Commitments
With the underpinnings now in place, we can quickly proceed to set project commitments, as follows:
■ Project personnel make three-point estimates of EDC for their respective tasks in the project. The tasks should be of comparable magnitude, with the smallest and the largest being, say, within a factor of 2 in size.
■ Using appropriate software, CDFs are determined for the project as a whole.
■ The CDFs for effort, duration, and cost are plotted.
■ Together with corporate or program management or with the customer, the project manager selects combinations of total effort and likelihood of success, total duration and likelihood of success, and total cost and likelihood of success. These selected points become the project commitments. Note that these commitments do not represent 100 percent success, but rather represent as high a likelihood of success as the customer is prepared to pay.
Establishing and Managing Project Contingency Allowances
Project contingency allowances for effort, duration, and cost are the differences between the commitment values determined above and the corresponding expected values. These contingency allowances do not guarantee 100 percent success; in fact, they may fall short of what is needed to meet all project commitments in an actual case.
To determine how much contingency allowance is needed to meet project commitments once the project is under way, the percentage consumption of each contingency allowance can be plotted against the percentage accomplishment of work (properly measured!). This type of plot is shown in Exhibit 5.
Exhibit 7. Using fixed commitments may backfire by eliminating unders that could be used to offset overs.
If a contingency allowance is being consumed at a disproportionately fast rate, it will be exhausted before the work is completed. This does not bode well for completing the project without adjustments. Accordingly, to meet project requirements, (a) more allowance will be needed, (b) some nonrequired work will have to be reduced or omitted, (c) some required work will have to be de-scoped by amending the scope of work, or (d) some combination of these three will be needed.
If the solution to the difficulty is to provide additional allowance, extrapolation of the allowance used versus work accomplished to the point where all of the work is done will indicate the amount of additional allowance required (see Exhibit 6).
Who will absorb or provide an additional allowance is partly determined by contract type. If the contract is cost-reimbursable, the customer bears the burden. If the contract is fixed-price, the project organization bears the burden. If the contract is cost-reimbursable with a not-to-exceed (NTE) figure, the customer bears the burden up to the NTE amount and the project organization bears any further burden.
Incidentally, schedule lateness is always felt by the customer. However, a customer's pain might be salved by lateness penalties for the project organization. If the penalties are painful enough for the project organization, it may choose to spend its own money to avoid being late in the first place. Beware, however: Not all lateness problems can be solved with money.
The Hazard of Fixed Commitments Within Projects
Some project managers require their individual task managers to provide fixed commitments. While perhaps well intentioned, if in a competitive situation, they are misguided if they believe that these individual task commitments will help ensure that they meet their own commitments. The logic is shown in Exhibit 7.
Project managers who require their individual task managers to make fixed commitments practically ensure that there will never be unders. However, overs can occur despite commitments—either directly or indirectly. Examples of direct overs are late deliveries. Examples of indirect overs are defective or deficient products that require their project manager recipients to expend extra resources to remedy the situation. These extra resources must be provided by the project managers’ own contingency allowances, as shown in Exhibit 7. In other words, the existence of fixed commitments by individual task managers does not obviate the need for project managers to have their own contingency allowances.
When the latter contingency allowances are added to the sum of the fixed commitments, the project is likely to be noncompetitive. Or, if the project was bid competitively in the first place, the project will overrun to the extent that the total allowance required to meet project commitments exceeds that included in commitment-likelihood combinations.
The organizational culture must suit the approach described here or it will not produce the benefits of which it is capable. If planning is shortchanged so that unders cannot be harvested when they occur, the approach is severely handicapped. If fixed commitments are required of individual project tasks, the approach is crippled. And, most of all, if hoarding and squandering resources are favored over parsimonious use of resources, the approach will fail. However, if cooperation is encouraged by word and deed and if generating and harvesting unders are rewarded, this approach will produce the benefits it promises.
While creating such a culture may seem daunting in some organizations, it can be developed by capable leaders who are consistent and persistent in their actions. It can be done within a single project as long as the organization at large does not fight it. And it can be spread throughout an organization by effective top management.
WHEN UNCERTAINTY IS INHERENT in project tasks, the use of unders to offset overs begins with accommodating these uncertainties in estimating task effort, duration, and cost, and using triangular distributions based on three-point estimates as surrogates for actual but unknown EDC probability distributions. Cumulative distribution functions of EDC can then be determined via the Monte Carlo method. CDFs show commitment-likelihood combinations that can then be used to set commitments, recognizing that 100 percent likelihood can be outrageously expensive or take an outrageously long time.
Setting commitment-likelihood combinations also establishes the EDC contingency allowances, which are the amount of effort, duration, or cost needed above their corresponding expected values to have the desired commitment-likelihood combinations.
Unders can be used to offset overs to minimize the need to use project contingency allowances, which are used to handle net excesses of overs over unders. Extrapolating plots of the percentage of project contingency allowance used versus the percentage of work done indicates how much additional allowance, if any, is needed to meet project commitments.
If project managers require fixed commitments for individual project tasks, they are unlikely to realize any unders to offset overs. At the same time, overs are likely to occur despite the fixed commitments. This means that contingency allowances are still required to have commitment-likelihood combinations greater than 50 percent likelihood. These contingency allowances-plus the expected values (which correspond to 50 percent likelihood commitments) may cause the project to be non-competitive.
Lastly, the organizational culture must encourage and reward cooperation if this approach is to succeed. With a proper culture, the approach can be applied locally, and it can be applied globally. ■
Reader Service Number 103
February 2000 PM Network