Introduction
Origins of Earned Value
To succeed in the current accelerated environment, projects must be completed in record time to take advantage of marketing and other “windows” of opportunity. However, public reports of project results are not encouraging. Too frequently we read of projects that are dismal failures – they are overspent by some enormous amount of money, the time overrun is measured in months or even years, and the final product of the project, if it emerges at all, lacks some of the planned features or has unexpected performance failures.
Those companies that claim that their projects are never late may be referring to the latest baseline due date (which may be considerably later than the original date) or their project plans may have allowed such long task times that the project takes far longer than it should. But there are project successes that inspire us to continue the examination of our processes and the search for better methodologies. The Earned Value Management System (EVMS) is one of those methodologies.
Recognizing the problem of poor project performance, and having been shocked by many surprising overruns and underperformances, the U. S. government, over 40 years ago, commissioned the development of a project measurement system based on a standard cost model. This system became known as Cost/Schedule Control Systems Criteria (C/SCSC). Private industry found the requirements cumbersome and suggested a revision in the mid 1990‘s. The new criteria were formally named the EVMS, and the 32 revised criteria were adopted by the government in 1996. American National Standard Guidelines for EVMS (ANSI/EIA-748-1998) were issued in 1998. Private governmental contractors have long been required to use EVMS on large government projects. Operations performed by the U.S. government for itself, such as repair depots and other project-oriented work, are rapidly implementing earned value metrics that affect total funding.
Earned Value Defined
Of the 32 EVMS criteria, about 40 percent establish organizational structure and project planning and change control requirements and about 60 percent involve accounting for costs including budgeting, reporting, and forecasting. Two major reporting metrics are cost and schedule variances. The cost variance is the difference between the budgeted cost of work actually completed and the actual cost of that work. The schedule variance is the difference between the budgeted cost of work that was scheduled to have been completed by the status date and the budgeted cost of the work that was actually completed by the status date. A perfect analogy to “Earned Value” analysis is found in a standard cost system. The familiar “triangle” approach to variances in a standard cost system can be directly applied to earned value as shown in Exhibit 1.
Exhibit 1
These two basic metrics can be converted into schedule and cost ratios [the Schedule Performance Index (SPI) and the Cost Performance Index (CPI)] that enable comparison across projects. Further, these two indices can be formulated into a host of other metrics that provide estimates of total cost at completion and then into an index ratio that, once again, can be used to compare performance on many different projects.
The SPI shows schedule completion on the measurement date, the earned value, relative to planned completion by that date. The CPI shows the cost that should have been incurred, once again the earned value, relative to the costs actually incurred. Specifically, at any status date:
Schedule Performance Index = (Earned Value)/(Planned Work); Cost Performance Index = (Earned Value)/(Actual Cost of Work Performed). Estimate to Complete = (Budget at Completion – Earned Value)/(CPI) = (BAC – EV)/CPI or = (BAC – EV)/(CPI * SPI). Estimate at Completion (revised total final cost) = Actual Costs (to date) + Estimate to Complete; To Complete Performance Index = (Budget for Remaining Work)/(Estimated Cost of Remaining Work) = (BAC – EV)/(EAC – Actual Cost).
What's Good About Earned Value?
Objective Evidence
Having become familiar with EVMS, some companies in private industry are now adopting EVMS, with good reason, for non-governmental projects. EVMS requires a Work Breakdown Structure, task dependencies, organizational responsibilities, a time-phased budget, and other formal planning and reporting procedures.
Without some objective measure of performance stakeholders must rely on the subjective judgment of the project manager, who may be encouraged by optimism to report that everything is progressing as planned (on time, on budget) even if difficulties have been encountered that have required more time and money than planned. The manager may truly believe that a time and/or cost overrun can be made up later and does not want project work to be questioned or stopped. Therefore, it is not uncommon to report, after one month of a three-month project, that the project is “on time” and “on budget” if one-third of the total budget has been spent, regardless of how much project effort remains. In this way, the bad news may be delayed until it becomes obvious that the project has no possibility of successful completion.
The traditional accounting now used on many projects compares costs incurred to date with expected budgeted expenditures for the period since the project began (sometimes using cash flows only with no accrual accounting). With no connection to schedule completion, information content and predictive ability of the resulting “variance” is extremely low or nonexistent. That is, there are no answers to questions such as, “Are we ahead or behind schedule?” and “Will the project finish on time, within cost and to specification?” For example, assuming a $208,000 project for a one-year time period, the cost/funding plan might look something like Exhibit 2. The project should cost $70,000 by the end of the first quarter, $108,000 by the end of the second quarter, $148,000 by the end of the third quarter, and $208,000 by the end of the project.
Exhibit 2
The executive management committee fully expects the project manager to stay within the limits of their $208,000 commitment and to continuously monitor the performance during the life of the yearlong project. At the end of the first quarter, when the plan called for an expenditure of $70,000, the actual expenditure has been only $65,000. When asked if the project is behind schedule, the project manager quickly responds that the project is right on course (meaning that one quarter of the one-year project has elapsed) and under budget by $5,000.
Another approach sometimes used is to compare actual costs incurred with estimated total completion costs of the project and thereby estimate the percentage completed. This is the standard accounting percentage of completion approach. Of course, the estimated cost of the project may change from period to period, but anecdotal evidence suggests that estimated total project cost is “sticky” and not readily adjusted. Also, a bottoms-up approach to estimating the total cost of work yet to be completed is very time consuming. (Using this approach, and not adjusting the total cost, would result in the assumption that the project is over 30% complete at the end of the first quarter.)
To obtain more relevant information, costs incurred must be tied to schedule completion. Earned value management requires a time-phased budget baseline against which contract performance can be measured. Thus, earned value looks not only at the planned and actual spending, but also the scheduled work. To provide this detail, the project plan must include detailed tasks to be accomplished, estimated durations of the tasks, and a sequencing of the tasks. An activity-in-box network for the previous project example is shown in Exhibit 3. The red line represents the critical path, the longest series of sequential tasks, requiring a total of 52 weeks to complete the project.
Exhibit 3
By the end of the first quarter, Tasks 1 and 3 should be completed, Task 2 should be about 17 percent complete, and Task 4 should be about 59 ½ percent complete. The project manager reports, however, that while Tasks 1 and 3 have been completed, Task 2 has not been started, and only 20% of Task 4 has been completed. Assuming costs for each task after Tasks 1 and Task 3 are incurred uniformly over time, the project's earned value is $59,000 ($30,000 + $25,000 + .2*$20,000). Therefore, the project has “earned” $59,000, not the $65,000 actually spent. This situation is portrayed in Exhibit 4.
Exhibit 4
It is clear that this project is not “on course” and “under budget.” The schedule variance is $11,000 unfavorable ($59,000 - $70,000); the cost variance is $6,000 unfavorable ($59,000 - $65,000); SPI is .84, meaning this project is running about 16% behind schedule; CPI is .91, showing that the project is almost 10 percent over budget at the end of the first quarter. Further, based on progress to date, the estimate at completion is not $208,000 but somewhere between $228,571 and $272,109.
An earned value system provides objective measures of progress and enables some meaningful projections of future costs. Thus, project owners have access to performance data other than verbal assurances of managers intimately involved with the project. In addition, the planning, reporting and control requirements of EVMS reflect good project management techniques.
What's Bad About Earned Value?
Measurement Information
Like standard costs, from which earned value was developed, earned value metrics summarize past performance, compare actual results to those planned, and signal developing problems. However, earned value metrics do not unambiguously indicate appropriate actions that should be taken. EVMS variances, even more so than standard cost variances, do not report information that is sufficiently timely and transparent to all interested parties. For example, consider the two projects shown in Exhibit 5. The projects have identical baselines and each task (block) requires the same time to complete and has the same planned value. The darkened blocks are those tasks that have been completed.
Exhibit 5
Both projects have the same earned value, schedule performance index (SPI), and cost performance index (CPI), but, which project is likely to be completed first? Other things being equal, the project plans in Exhibit 5 indicate that Project B will complete before Project A, which has seven uncompleted tasks on one path. EVMS metrics would not signal this situation. Earned value metrics must be related to a project's time-phased baseline.
The fact that the SPI must be compared to the project schedule itself to obtain an accurate picture of the true schedule position of the project has been well known at least since an Arthur D. Little Company study was commissioned by the Department of Defense in the 1980s (Fleming & Koppelman, 2000, pp. 113-120). However, rarely do stakeholders receiving the earned value measures have access to the detailed project plans. Most commonly, comparison of EV measures to the project plan is not a formal part of earned value metrics. Even if stakeholders did have access to all project plans, they do not have the same information as the project manager and the proper corrective action is not obvious (to either the stakeholders or the project manager).
None of the EVPM metrics that purport to reflect progress and remaining work actually correlate directly with completion of the project plan itself. While SPI and CPI are designed to measure cost and schedule deviations from the planned baseline, both indices incorporate the “earned value” metric and both have components that are denominated in dollars. Therefore, SPI and CPI have the appearance of measuring progress on the project baseline. However, as pointed out by the Arthur D. Little Company analysis and illustrated in Exhibit 5, the measures are susceptible to manipulation.
Resource managers control assignment of resources to projects. Once a project has begun, the resource manager can expect that the project manager will demand their share of resources. A resource manager typically has to show weekly progress on all projects. Because EV metrics are not perfectly linked to the project plan of work, when managers use EV to manage their projects, priorities between tasks on different paths, as well as between projects, are not always clear to either the resource manager or the project manager. A project easily may suffer delays due to the assignment of limited resources to non-critical tasks.
Although EV can provide some objective measures of performance, the dollar aggregated project measurements can point neither to specific areas of good or poor performance nor to their causes. Without knowledge of the area of concern, remedial tactics cannot be undertaken or directed. If corrective action must be taken, other project management tools must be employed to identify problem areas and potential solutions.
What's Ugly About Earned Value?
Dysfunctional Behavior
Because of a basic disconnect between EVMS metrics and project operations/completion, and because project funding often is tied to EVMS performance, some project managers have developed certain earned value coping mechanisms. These coping behaviors are an example of the impact of dysfunctional measures that Austin defines as “consequences of organizational actions that interfere with attainment of the spirit of stated intentions of the organization” (Austin, 1996, p10). He maintains that an incentive system becomes dysfunctional when it encourages people to take actions that reduce value to the customer below what it would have been without the incentive system. Further, Austin maintains that the main problem for most incentive plans is not measurement error but bias intentionally introduced by the people being measured. However, this bias need not be designed to deceive and may result merely from the conscious effort of people to do what they are being asked to do.
Deming emphasized the importance of distinguishing between two kinds of variation: common cause variation (variation within the capability of a system to repeatedly produce results – that is, a system in control) and special cause variation (variation beyond the capability of a system to repeatedly produce results, usually variation with causes outside the system – or, an out-of-control situation). Attempting to “fix” common cause variation makes results worse by overcorrecting an in-control process to an out-of-control state. The same concept applies to projects. That is, responding to small variances by making project changes very likely will result in worse project performance.
Obviously, there are possible tradeoffs between the Cost Performance Index (CPI) and Schedule Performance Index (SPI). For example, additional costs might be incurred (overtime, acquisition of additional resources, etc.) to improve a poor schedule performance. Or, the schedule might be put on hold until prior resource commitments have been fulfilled, reducing the need for additional costs, but resulting in schedule slippage. The attempt to keep both measures on track (versus making appropriate decisions to complete the project) frequently results in conflicted behavior.
Inherent Conflicts in EV
Wanting to be successful, project managers simultaneously want to look good on both cost and schedule metrics. Textbooks and earned value implementation documents usually warn that a cost variance and a schedule variance should never be “netted” together. One author specifically cautions that, “A poor cost variance combined with a good schedule variance does not mean that everything is all right” (Humphreys, 2002, p675). Since meeting the schedule may be critical to the customer, meaning we must meet SPI targets, project teams are encouraged to spend as much time as necessary to meet SPI targets.
At the same time, meeting cost targets may be critical to the company responsible for completing the project. Therefore, project teams are pressured to spend as few hours as necessary to meet CPI targets. Project managers try to expend the necessary resources to meet SPI targets while at the same time feeling strong pressure to use as few resources as necessary to meet CPI targets. Thus there is a conflict between the two metrics which typically leads to alternating compromises between them as shown in the cause-effect diagram of an earned value system in Exhibit 6 where the numbers indicate the order in which the diagram should be read.
Exhibit 6
Attempting to “do everything right,” as reflected in the SPI, and “stay within budget,” as reflected in the CPI, along with the use of EV to manage the project, means that project managers will try to make both metrics look better by constantly adjusting behavior to garner better performance on each metric. Because of the trade-off effect, focus on an unfavorable SPI (or CPI) naturally makes the CPI (or SPI) metric worse. The conflict between spending as few hours as possible and spending as many resources as necessary means that attempting to improve both SPI and CPI simultaneously (this means taking actions that will favorably impact these measures) insures that neither metric will improve. Neither metric improves because the actions intended to make each metric immediately look better (by the next status date) means that shortcuts and temporary patches can quickly cause other problems making remaining project tasks even more difficult. This situation ultimately and inevitably results in failing to meet targets.
Discussion and Conclusion
We must have measurements to communicate progress but the ideal indicator of performance may be especially difficult, or even impossible, to measure. Arrow (1971) showed that all measures of performance are not equally easy to measure. Thus, performance metrics adopted by organizations are constrained by practicality and cost. It should not be surprising that all critical dimensions of project performance do not get measured and reported.
Austin (1996) partitions measurement into two categories: motivational measurements that are explicitly intended to affect behavior and informational measurements that are used primarily for the information they convey. He maintains that informational measurements should be used to improve decisions and should not be used to effect behavioral changes. Earned value metrics are information to project owners but they may inadvertently affect the behavior of project team members attempting to perform to the metrics. In fact, it is as true today as when described by Flamholtz (1979) that accounting measurements are simultaneously intended to facilitate multiple functions including accountability, performance evaluation, and motivation as well as provide information for decision making. This “one size fits all” objective of internal managerial accounting information must be questioned just as many analysts recently have questioned the appropriateness of one financial report for external users.
Managers who make decisions determined by their impact on the next quarterly results or annual report rather than the long-term health of the organization for all stakeholders have been severely criticized and punished by the market. Analogously, making project decisions based on an imperfect measurement system that does not measure all critical dimensions of project performance is detrimental to ongoing project process improvement and possibly to project completion.
In any event, there are some implications for any organization that has an EVPM system. First, the earned value metrics that are a source of information to stakeholders must not be used to make project decisions. That means that the EV metrics must not be used to evaluate the project manager and project team members. Project decisions must be based on sound project management practices. Project team members must be motivated to complete a project as quickly as they can and at the lowest cost. Decisions must be made using real-time data, not on either historical data or the impact of the decision on historical data to be reported in the future.
A 1990 study by Beach illustrated that as early as 15 percent into a project, EV metrics can predict the ultimate completion date and project cost (Fleming & Koppelman, 2000, p39). However, EV measures may not be tracking closely with project performance early in the project. Therefore, management may have to be careful not to be too hasty to demand corrective action if initial reports show unfavorable earned value variances.
Finally, organizations must be very aware of the behavioral responses to measures and be extremely careful in designing performance evaluation systems. If Austin (1996) is correct, motivation and information metrics must be separated in order to minimize dysfunctional worker behavior. Establishing appropriate organization measurements is hard work and may require that management not be allowed access to all the “information” measures that workers need in order to do their best work. Management not having access to all information possessed by workers also might mean that management must place more reliance on the subjective judgment of supervisors and fellow employees.
Due to the size of projects that typically use EVMS, EV metrics can only be tracked and reported using a software program. Fortunately, such software is readily available. CS Solutions’ wInsight and Welcom's Cobra are two of the programs most well-known for drill down capability, forecasting, schedule integration, top-down planning and customization, but a number of companies provide such software. The following web site has a very comprehensive list of software vendors: http://www.acq.osd.mil/pm/tools/tools.htm.