Introduction
We are currently seeing some clear industry trends. Firms that were once forced to implement EVMS on large government contracts are now widely using these techniques to manage commercial portions of their businesses. What has caused this change? Streamlined procedures, industry taking ownership of the process, and huge advancements in information technology for organizing, planning, tracking and analyzing projects. With more stakeholder involvement, the need for better, more effective management systems, together with seamless, integrated tool sets for managers are more important now than ever before.
This presentation provides an overview of how poor project visibility can be overcome and identifies specific methods to accomplish this goal. Organized management systems and the performance measurement approach provide management visibility into technical, schedule and cost performance throughout the life of the project. This information can be used for high-level management-by-exception reporting and for developing more confident forecasts of project completion dates and costs. As responsible managers, we cannot sit and wait to see what happens as the project finishes. In the performance-oriented approach to management, given the proper kind of information, responsible managers can in fact affect the outcome of a project. This presentation discusses the Earned Value Management approach to integrated management systems and how to effectively use the performance data to manage projects.
Project Organization
The Work Breakdown Structure (WBS) provides a methodical, systematic way to define and understand the work to be done on a project by progressively subdividing the total scope of the effort into its component elements. The WBS contributes to project management by helping to define and understand the work to be done; by providing a framework for integrating cost, budget, schedule and work scope with each other, and by establishing a structure for reporting project status for internal management and the customer. Exhibit 1 displays a summary level WBS.
Exhibit 1
Exhibit 2
Exhibit 3
Exhibit 4
The organization breakdown structure (OBS) interrelates with the WBS to define key management control points and responsibilities for doing the work. Exhibit 2 displays a summary level OBS. The intersection where the OBS and the WBS come together is a managerially significant element of work normally designated as the control account. Exhibit 3 displays the interrelationship between the WBS, the OBS, and control accounts.
A WBS dictionary is developed that ties the scope of the program together by describing the work required to produce the elements of the Work Breakdown Structure.
Scheduling, Estimating and Performance Budgeting
There is no substitute for planning. Understanding the magnitude of the task at the outset—understanding the parts that describe the whole—is crucial to the success of any project, administrative or substantive. The objective of the planning and scheduling baselines is to identify all the known requirements; schedule all the project work; and to use a disciplined, integrated approach. The schedule baseline serves as the basis for the development of the budget, and thus the performance measurement baseline. Estimating and scheduling proceed in parallel since the two are interrelated. Performance budgeting provides a time-phased resource plan against which accomplishment and resource expenditure can be measured. The intent is to have a baseline plan, which is consistent, for both cost and schedule information. This time-phased baseline is commonly referred to as Budgeted Cost for Work Scheduled (BCWS). The performance measurement baseline is the product of a planning process that involves identifying and defining the work, designating and assigning organizational responsibility for doing the work, scheduling the work tasks in accordance with established targets, and allocating budgets to the scheduled activities as shown in Exhibit 4.
Performance Measurement
Basic Data Elements
Performance measurement consists of evaluating work status in order to determine earned value or Budgeted Cost for Work Performed (BCWP). Comparison of planned value (budget) vs. earned value (accomplishment) is made to obtain schedule variances; comparison of actual costs (Actual Cost of Work Performed (ACWP)) to earned value is made to obtain cost variances. As shown in Exhibit 5, the project is both behind schedule and over cost. Without earned value we have no insight as to the work actually accomplished and can only determine that the project is behind in its spend plan, or under spending. Performance measurement data provides a basis for management decisions by both the corporation and customer when summarized for higher levels of the project. Performance measurement provides work progress status, relationship of planned cost and schedule to actual accomplishment, valid and timely data, and a basis for developing or evaluating the estimated cost at completion (EAC). One purpose of this presentation is to give an overview of a commonsense approach to the EAC. Later we will discuss EAC development and outline the basic use of performance indices and other analysis techniques in assessing when to incorporate a new EAC.
Exhibit 5
Variance Analysis
Significant variances require routine analysis and periodic explanation to management and the customer. Cost or schedule variances are identified separately and the reasons for the variances are clearly identified, with emphasis on quantitative and specific causes. The impact of the variances on cost, schedule, technical and other organizations must be considered along with a description of specific corrective actions planned or taken to alleviate the impact of the problem. The exact metrics used to manage a project and trigger variance analysis can vary widely. They can include information from cost data, schedule data, technical data and other items that may help track progress of the project. In the area of cost performance data, people normally view cost and schedule variances in dollars/hours or percentage/index-based metrics. Dollar values are great to understand the possible funding impact, but percentages/index-based metrics normalize the data in accordance with the size of the account. For example, is a -500K cost variance significant? It depends. If -$500k represents only a -2% variance then probably not; however, if it represents a -15% variance it most likely will be considered significant. Listed in Exhibit 6 are some of the more common formulae used in cost performance metrics. All can be calculated with dollars, hours or equivalent heads, and items 1 thru 6 can be calculated incrementally or on a cumulative to-date basis.
Schedule metrics calculation from earned value data such as the SV and/or SPI are useful, but should be supplemented with metrics from the actual project schedule. Earned value data provides insight as to a variance from plan (e.g., ahead or behind schedule), but gives no insight as to whether the variance impacts the critical path. A negative SV or an SPI lower than 1 would indicate a behind schedule condition; however, if the element has float in the schedule there may have been a prudent business decision to delay that work to focus on other items on the critical path that may be experiencing problems. Sample scheduling metrics that we tend to look at are float (is it on the critical path or how close is it to the critical path), slip from baseline finish (how far the activity slipped from its original plan), consecutive slips (how many times has the projected finish date been slipped in the last several months), and how many milestones were planned versus completed in a the period or cumulative to date basis. Of course, there are many more potential metrics, but correlating the earned value variances to the project schedule is an essential step in the variance analysis process.
Exhibit 6
Estimates at Completion
Developing estimates at completion is often called a “black art” and, too often, estimates are structured to fit some predetermined idea of a number that is politically salable. When ranges of estimates are prepared, opponents of the program tend to focus on the high end while proponents want to look at the low end. In the final analysis, most cost estimates are “products of management judgment.” They are affected by political considerations, budgetary considerations, important upcoming events, such as technical reviews, and other factors. There is also the feeling that more/better information will be available next month or next quarter, so perhaps the estimate update should wait until then.
Advocates of Earned Value Management have been accused of concentrating too much on historical information while the program manager, company management and the customer are more interested in what lies ahead and what the final cost will be. However, to figure out what the cost will be at completion requires an evaluation of the work remaining, and to determine how much work is remaining requires a solid understanding of where the program is now. Earned Value Management provides that essential information because earned value represents the amount of work accomplished in terms of the overall program target. Subtracting earned value from the program target cost reveals the budget value of the work remaining.
Exhibit 7
All estimating is based on assumptions. If the assumption is that this program is just like the last program, historical data provides the estimate. If it is similar, but more complex, a “complexity factor” can be applied to make the estimate more realistic.
The Earned Value Management baseline represents the plan for doing the program and considerable effort is expended to make it a meaningful, workable plan. It is updated as the program moves along to incorporate changes in scope or direction, and revised to compensate for problems experienced along the way. Assuming that the baseline validity is maintained, a reasonable estimate of cost at completion can be derived by simply adding the baseline budget for work remaining to the actual costs to date or by adding the cumulative cost variance to the program target cost. Exhibit 7 illustrates this approach.
Target cost (80) minus earned value (35) equals the budget for work remaining (45). Actual costs (45) plus the budget for work remaining (45) equals the estimate at completion (90). Since the current cost variance is -10, if the plan is achieved, the cost variance will not change and when added to the target cost, also provides an estimate at completion of 90; target cost (80) plus the cost variance (10).
If, however, the program has been overrunning, using the baseline plan from now to completion represents improved performance, and an explanation should be provided as to how this improvement will be achieved. Studies of hundreds of programs have revealed that program performance does not tend to improve. In fact, once a program passes the 15% completion point, performance almost never surpasses the average performance to date, and often gets worse. Average performance to date can be easily calculated by dividing the cumulative earned value by the cumulative actual costs. The resultant value is called the Cost Performance Index (CPI) and a number less than 1 reflects unfavorable cost performance. For example, a CPI of 0.85 means that for every dollar being spent, only 85 cents worth of work is getting done. Dividing the program target cost by the CPI of 0.85 will provide a cost estimate that reflects a continuation of performance to date to the end of the program. Program managers, in reports to senior management, should be asked to explain any estimate lower than that derived using the CPI approach. Exhibit 8 illustrates the CPI technique where the earned value (35) is divided by the actual costs (45) to produce a CPI (0.78). Dividing the target cost (80) by the CPI (0.78) results in an estimate for the program of 103. This approach is simply a linear extrapolation and does not take schedule projections into consideration. In essence, it assumes that if the program is 10% overrun today,it will be 10% overrun at completion. If desired, a CPI can be calculated for a recent period of performance (e.g., last six months) if there is reason to believe it would produce a higher quality estimate.
If dividing earned value by actual costs provides a cost performance index, then dividing earned value (35) by the cumulative budget (40) provides a schedule performance index (SPI) (0.88). If the program is not only overrunning cost, but is also behind schedule, additional cost impact can be expected either due to potential schedule slippage or from acceleration of the effort to finish on time; both alternatives cost money. Multiplying the CPI (0.78) by the SPI (0.88) provides a composite index (0.69) that can be applied to the budget for work remaining (45 divided by 0.69 equals 65) and added to the actual costs (45) to produce yet another estimate (110). Exhibit 9 illustrates this concept.
Exhibit 8
Thus a range of estimates (90, 103, 110) can be derived quickly and easily from the Earned Value Management data elements of budget, earned value and actual costs. These estimates should not be used as the final cost estimates, but as a sanity check for cost estimates derived through other methods, such as engineering estimates, grassroots estimates, parametric estimates, other statistical estimates, and estimates based on management experience.
At least annually, a comprehensive estimate of cost at completion should be performed using a variety of estimating techniques. This estimate is needed in support of business base projections, the annual financial plan, and baseline revisions and cash flow management. The statistical techniques described above are used to update the estimate as time passes and as a crosscheck against unrealistic projections.
Change Management/Revisions
After a baseline has been established, processes must be put in place to control and implement changes to that baseline properly and in a timely manner. When the customer authorizes the contractor to make a change to the technical, schedule, and/or budget baseline, the contractor must modify internal authorizations to change the appropriate baseline. Internally initiated changes use the same methods of implementation as those that are customer initiated. An effective change control program provides for early recognition, thorough evaluation, and proper processing.
Building a Performance-Based Organization
Having made a case that earned value is a valuable tool to help manage projects, why has the resultant data been so under utilized in the past? Vast amounts of project management data, including earned value data has been generated by most large government programs for over thirty years; however, it's only in the last five years that the data has actually been used to manage a project rather than just report on it. The driving force behind this change has been advancements in information technology. We've all become smarter in regards to managing projects, but the reason that earned value data was not used in the past is simply because it was not timely. When stakeholders realized that the data generated was not used to manage the project, the effort expended to initially develop and then status plans was minimal and half-hearted at best. Let's look back at a typical defense program of the 1980's. Suppliers where required to submit cost performance data on a monthly basis and given about 25 days after the end of the accounting month to submit the report. This meant that in the best case scenario we had information to react to a problem that occurred 25 days ago, and in the worst case the problem occurred 55 days ago. Contrast that with what is happening today where projects are able to close out performance on a weekly basis and have earned value information online to stakeholders in a matter of days. We are not talking about small projects within a single organization. Weekly earned value is being implemented on some of the largest projects even with the participation of international partners. With this type of turn around, the data is significantly more meaningful since managers have an opportunity to take action based upon the information. Further, information systems have evolved to the point where management-by-exception tools can drill down to the exact elements that are causing cost and schedule performance problems, cutting the time to analyze data to minutes instead of days. These tools allow managers to cut through the bureaucracy and zero in on accounts that need immediate attention. Finally, the underlying cost and schedule tools have become much more user friendly, which allows the stakeholders the opportunity to participate in building the initial plan for their work area. We are starting to see more and more technical managers building their own plans in scheduling tools with resource-loaded schedules. This provides a true sense of “ownership” of the plan, which is critical to engaging technical managers in the resultant use of the data. Once a technical manager fully understands and agrees with the plan, it is much easier for them to assess performance.
Exhibit 9
Critical to the successful transition to a performance based organization is senior level management “commitment.” In every corporation where a metrics based “management-by-exception” system is implemented there is an organizational “cultural change” that must take place. It is often much harder to implement the cultural change than it is to implement the information technology tools. With virtual certainty, the first project will include a technical manager that explains their variances in the following manner, “Those variance aren't real. I didn't plan the work the way I am actually executing it, and that's causing a lot of false variances. Do you want me to spend my time doing plans and answer questions, or do you want me to do the work?” First and foremost, senior management must stop this type of “variance explanation” dead in its tracks. Management must stress that the company needs both a quality plan and the work to be executed. Replanning of the work should be accomplished in the next rolling-wave planning process to reflect the manner in which the work will be accomplished. Management must make it clear that the organization depends on good plans for manpower projections, material procurement, cash flow projections, EACs, and to maintain creditably with the customer. Many organizations are moving to enterprise resource planning (ERP) tools that base every aspect of running the business on the “plan.” If management does not see the value in developing good plans and tracking performance against the plan, no automation tool can overcome the “garbage-in/garbage-out" syndrome.
Once the EVMS is functioning and generating data you need to capture the technical manager's attention. Engaging them in performance measurement can be a tricky business. You need to make their jobs easier, not harder. You will likely only get one shot, so do it right the first time. Each manager should have customized views that display only the accounts for which they have responsibility, with the corresponding key metrics for each. Make sure that parameters are set to isolate significant work-in-process issues, not past problems. If managers have to spend time digging through the data to find their information they will simply not do it. Use color codes to identify element status and trend indicators to display performance since last period. This allows the manager to quickly assess the status and determine if they are getting better or worse. Typically red is used for unsatisfactory performance, yellow for marginal, and green for good. The metrics window should not only provide the necessary information to locate problems, but also must link to related trend charts, reports, and schedule activities of the selected element. Finally, it should provide a mechanism for the manager to communicate the qualitative assessment of the situation back to other teammates. Exhibit 10 displays a sample screen with earned value metrics.
Exhibit 10
During the execution of a project the manager often times already realizes there is a problem before the earned value data has arrived. The earned value data simply quantifies it. Don't assume that it is intuitively obvious to the manager what to do next with the data. They really need to be lead to the next step and understand why it is important. The computer has done the number crunching and presented the information, but only the manager knows how to use it to potentially improve performance and generate an accurate EAC. Reiterate the items that are important in the qualitative phase. First, they must find out what caused the problem including the cause of the problem. Why? Because if they don't know the cause, how can they determine how to fix it or whether it can even be fixed. For example, assume that a cost variance is related to the first batch of materials on the project because an expediting fee was paid to meet the project schedule. Let's also assume that the expediting fee will not persist in the future since we have proper lead-time for the remaining units. In this case the overrun would not be forecasted to continue for future units. The variance to-date would not be recoverable and would show as a portion of the VAC. However, contrast that scenario with one where the amount of material required to manufacture each unit was underestimated. The overrun would be expected to reoccur for each future unit, and is essential information to support EAC development. In this case the VAC should reflect the cost variance to date and the projected overrun for each additional unit planned. Of course there can be many other events that have yet to occur that can affect an EAC. For example, the renegotiation of a union labor contract may affect future labor rates in an unfavorable manner. The earned value data to-date does not reflect this “known” condition, but you must teach managers to think about the future even in accounts that seem to be doing fine.
As stated earlier, implementing a performance-based organization is a significant cultural change. It will require changes in processes, training, and most likely new automation tools. Again, don't expect to accomplish it without solid senior management support. One way to demonstrate management commitment is to use the management by exception tools when conducting program reviews. Experience indicates that when using such tools the program reviews can be much more focused, thus significantly reducing the length of the meeting. One of our customers was able to reduce the comprehensive monthly review from two to one day by using the tools to focus the meeting. Not only are rewards significant in implementing a performance-based organization, but also one could argue that to be competitive today you must move to this model.
Where Are We Headed?
Certainly trends indicate that earned value will be used on more government and commercial projects, both in the United States and internationally. In terms of the information technology, we will see better integrated tools that link cost, schedule, technical, financial, configuration management, document management, labor collection, and other project related items into a comprehensive project management system. These tools as well as tools to analyze performance will be web enabled to support the geographically disperse teams that typically execute large projects. Further, these tools will do a better job of integrating vendors and suppliers with business-to-business Internet commerce to improve the turnaround time to organize, plan, track, bill, pay, and analyze projects.
Proceedings of the Project Management Institute Annual Seminars & Symposium
September 7–16, 2000 • Houston, Texas, USA