Integrating risk and earned value management
Both risk management and earned value management are mature disciplines in the project management profession, but they need to be integrated to understand their combined affect on program cost and schedule estimates. An National Defense Industrial Association NDIA working group in 2002 explored the integration of risk management with earned value management and concluded that although programs would benefit from this integration, 43% of respondents assessed the state of integration as “poor” or “very poor.” The primary barriers included a lack of risk management and earned value process maturity and a lack of knowledge and skills. This paper defines several key concepts about control accounts, risk management, and earned value management and describes how they can be combined to develop an integrated estimate of cost and schedule at complete.
Why the Interest in Integrating Risk and Earned Value Management?
For major acquisition projects in the United States Department of Defense (US DoD), the answer is simple: it’s required. Recent US DoD guidance from the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD AT&L, 2007) reemphasized the importance of earned value management (EVM). The Defense Contract Management Agency Earned Value Implementation Guide requires schedule risk assessment (DCMA, 2006, paragraph 188.8.131.52.5), and the DoD Risk Management Guide (OUSD [AT&L], 2006, Para 2.4) emphasizes the importance of the integrated baseline review, schedule risk assessment, and establishing a realistic schedule and funding baseline for the program.
The US DoD has been a driving force in many project management innovations and disciplines in use today. The 32 current EVM system criteria in ANSI/EIA Standard 748 derive from the 35 Cost/Schedule Control System Criteria (C/SCSC) originally introduced to industry by DoD in 1967 (Fleming & Koppelman, 2000, p. 157). For over 40 years, earned value has demonstrated its worth as a sound project management discipline.
Additionally, an NDIA working group in 2002 explored the integration of risk management and EVM and concluded that although programs would benefit from this integration, 43% of respondents assessed the state of integration as “poor” or “very poor” (NDIA, 2002, p. 5).
What are the Challenges?
The primary barriers identified in the NDIA Study included a lack of risk management and earned value process maturity and a lack of knowledge and skills (NDIA, 2002, p. 6).
Both earned value and risk management are relatively simple concepts. The need to implement them was learned in 4th grade math lessons. However, the processes have to be executed, and that requires organizational discipline---namely, the discipline to:
- Collect the data on which to develop cost-estimating relationships
- Create a sound program management baseline
- Execute the plan
- Manage the uncertainties
Major DoD acquisition programs undergo a great deal of scrutiny, and their cost and schedule estimates are reviewed by several independent cost analysis organizations such as the Office of the Secretary of Defense Cost Analysis Improvement Group (OSD CAIG). Stakeholders in system engineering, logistics, testing, and training review the program to make certain that no scope is overlooked. Program dependencies, synchronization, and support requirements are examined to make sure that related programs are on track and the necessary supporting data will be available when needed. All this is part of the review process that major programs go through on the way to a milestone decision. However, not every program has the benefit of this kind of scrutiny and institutional infrastructure. Absent the processes enforced by a well-documented procurement system, it falls to the program manager to ensure proper planning and execution.
What are the Benefits and Implications?
Integrating risk and EVM provides early visibility into the range of estimates for cost and schedule at complete, with an understanding of the probability associated with individual point estimates for each. By making a distinction between uncertainty associated with cost and schedule estimates, and uncertainties associated with discrete events or probabilistic branching, management can focus their attention on the right things.
The most essential concept is that of the control account, because this is where earned value and risk management come together under the responsibility of the control account manager (CAM). Control accounts include the following:
◆ Scope of work
◆ Resource Requirements (Cost)
◆ Duration (Schedule)
◆ Responsible organization (organizational breakdown structure, or OBS)
The control account is often referred to as the intersection of the work breakdown structure (WBS) and OBS, but as the four items listed above also show, each control account is essentially a mini-contract. The CAM (OBS) agrees to deliver something (scope) in an agreed amount of time (schedule), using only the resources that can be obtained within a certain budget (cost).
The sum of all control accounts, along with undistributed budget and summary planning packages, when laid out over time based on the logical dependencies, comprise the program management baseline (PMB) against which earned value performance is measured. Therefore, a realistic PMB is essential to successful earned value performance. The US Department of Defense recently reemphasized this point in a memo to the acquisition executives (USD AT&L, 2007):
Correctly imposing the EVM requirements on contract and establishing the baseline are critical prerequisites to the successful implementation of EVM. […] In addition, the Components should establish and maintain realistic, executable performance measurement baselines against which to measure contract performance. (p. 2)
No CAM wants to “overrun” budget or schedule, so their estimates will reflect cost and schedule uncertainty as well as any discrete risks that are known. Whether individual CAMs are allowed to manage the budget allocated for discrete risks or whether this management reserve is held at higher levels is a matter of policy. However, discrete risks---things that typically appear in a risk register---are of obvious concern to the CAM, since they may cause the control account to overrun cost or schedule or fail to satisfy his requirements.
Work Packages and Planning Packages.
Control accounts are divided into individual work packages where costs will be accumulated and progress measured. Much like a contract, each work package has its own period of performance; it will be “opened,” a charge number will be activated, and costs will be accrued. When the work has been completed, the work package will be “closed” and no further charges can be made to it. This requires a period-based accounting system that, in addition to the financial accounting, provides a way to “earn value” for work performed during the same accounting period in which the costs for that work were accrued. Mature organizations with accounting systems in place that support this type of financial and performance accounting have a distinct advantage. Simple solutions for implementing EVM just don’t scale well for larger programs.
Mature organizations also have another advantage. As data are collected over time, cost-estimating relationships can be developed that cost estimators and CAMS can use to reduce uncertainty in their estimates. DoD was very strict about having programs adhere to standard WBS structures at the top three levels so that cost-estimating relationships could be developed. Occasionally, a cost estimate for a program may be prepared that is structured differently than the WBS because the cost-estimating relationships on which the cost estimate is based are different from the structure of the work being planned. Where this happens, there needs to be a mapping of elements of the cost-estimating structure (CES) to the WBS to ensure that all elements of the work are accounted for.
Flow Down and Allocation
The WBS forms the basis for the flow down and allocation of requirements for the system. Both System Engineering Fundamentals (Defense Acquisition University) and MIL-HDBK 881A (Department of Defense) provide excellent explanations of this process and suggestions for how to construct a good WBS. Since the US Department of Defense has been collecting and analyzing data on programs for over 40 years, these are both useful and practical documents, and even though they are DoD-oriented, they contain a wealth of practical guidance that can be readily adapted for use in other areas.
The Program Management Baseline
The goal is for a project to perform “on cost, on schedule, and achieve all performance requirements.” If a program deviates from the baseline, it is for one of three primary reasons: a poor baseline, poor execution, or the occurrence of an unforeseen risk event. Since two of the three reasons involve the PMB directly, it is essential that it be realistic and accurate.
Risk management deals with uncertainty that can affect cost, schedule, or performance in any combination. This paper is focused on the cost and schedule impacts, and groups these uncertainties into two general categories: “Common” or “Special” causes of uncertainty.
Common Causes of Uncertainty
These are the things that drive variation in cost or schedule estimates. Tasks that have been performed many times by mature, data-driven organizations may have a small distribution around the point estimate; those that are being performed for the first time or by less mature organizations would be expected to have a wider distribution. These uncertainties are typically not found in a risk register, and they account for the variations in cost and duration that typically occur. This is similar in concept to Deming’s “common causes” of variation in a manufacturing process. For example, it would consider normal variability in worker productivity, but not the chance of the union going on strike. The result is a distribution, narrow or wide, around a point estimate.
Special Causes of Uncertainty
These are the things that cause quantum changes in cost or schedule by causing effort to be added that was not planned, either to mitigate a risk or to respond to the fact that a risk event has occurred. They should appear in a risk register, and although they may have an uncertainty distribution around them, the main impact is whether or not the task exists at all. This is called existence uncertainty or probabilistic branching and is similar in concept to Deming’s “special causes” of variation. For example, it would assess the chance of the union going on strike and the resulting impact to the program, but not the normal variability in worker productivity. The result is a task that may or may not exist, with some distribution around that point estimate, and includes probabilistic branches, if more than one outcome of a risk event is possible.
Risks can affect performance, and attempts to correct the problem may have cost and schedule impact. However, it is also possible for performance impacts to occur that do not impact cost or schedule. The approach taken here is to ignore performance impacts and assess cost and schedule impacts separately. The program clearly needs to manage performance, but unless it impacts on cost or schedule, performance can be ignored for purposes of this analysis.
Earned Value Management
Earned value management has proven its value as a way to integrate cost, schedule, and scope to create an objective view of contract performance. It uses performance relative to a PMB to predict future cost and schedule efficiency. This relies on two important assumptions:
- The PMB is as accurate (or flawed) in the future as it was in the past
- Task efficiency will be the same in the future as it was in the past
Earned value management does not “predict” where risks will occur; it only applies past performance to future estimates of cost and schedule. A control account that was poorly planned or poorly estimated, or is being poorly executed will probably exhibit poor EVM performance, but this is more of an issue than it is a risk. On the other hand, a control account that was well planned and estimated and is being well executed may encounter unforeseen challenges and begin showing poor EVM performance. This is where EVM does provide an indication of risk, since additional time and money spent trying to correct deficiencies will show up as variances. However, it is important to realize that this is a trailing indicator. If this is how your program finds out about risks, you’re being reactive, not proactive. A well-functioning risk management program will identify risks well in advance of any actual program impact and provide management with the opportunity to mitigate them.
A Layman’s Explanation of Earned Value
Earned value is a relatively simple concept; it is quite literally, the “value earned.” We assign value to things every day by buying them for the cost shown on the price tag. We pay people for the services they render and pay the contractors whom we have hired to perform more complex tasks. We create a mental budget of the acceptable cost of things that we need, and if the price tag is less than our budgeted cost, we are likely to buy them. This “budgeted cost” represents the “value” we have assigned, and regardless of whether this budgeted cost is compared with items listed in the Sears catalogue, GSA catalogue, or a Data Accession List, the concept is the same.
The “budgeted cost” can be earned when the work has been performed. The terms “earned value” and “budgeted cost of work performed” represent the same concept: how much the work that was actually performed is worth. This “earned value” can be earned in pieces over time. For instance, if your son agrees to cut the front and back lawn for $20.00, you might agree to pay him $10 once he’s finished the front yard so he can go to a noon matinee with his friends, provided that he agrees to return to cut the back lawn and earn the remaining $10 when the movie is over. On the other hand, if you’re getting a haircut for $20.00, you’re not likely to pay anything until the job is completed to your satisfaction.
EVM compares this “value earned” with two things: the value of the work that was planned over time (planned value or budgeted cost of work scheduled), and to the actual costs of performing the work (actual value or actual cost of work performed). The differences (variances) and ratios (performance indices) provide the basic information needed to implement an EVM System.
A More Rigorous Explanation
The Earned Value Implementation Guide, issued by the Defense Contract Management Agency (DCMA) as the DoD’s Executive Agent for EVM Systems, provides a complete explanation of EVM and the procedures used to implement it. It includes pre- and postcontract award activities, procedures to be used when conducting Integrated Baseline Reviews (IBRs) and other guidance that has been coordinated with the Air Force, Army, Navy, Missile Defense Agency, and other organizations with extensive experience in program management. Extensive guidance is readily available, including documents available for free from the US DoD.
When formally implementing an EVM System in accordance with the ANSI standard, mature organizations have well-defined processes in place for developing the WBS and allocating requirements, resources, cost, and schedule to the control accounts. This provides a mechanism for the program to develop a realistic baseline against which to measure performance.
The fundamental EVM performance indexes are:
|CV = BCWP – ACWP||SV = BCWP – BCWS|
|CPI = BCWP/ACWP (Earned Value/Actual Value)||SPI = BCWP/BCWS (Earned Value/Planned Value)|
Cost and schedule are closely related, but it is worth noting that the comparison of budgeted cost to actual cost, and work performed to work scheduled both use the common factor of budgeted cost of work performed (earned value), to provides separate indicators of cost and schedule performance. A schedule delay does not always result in a cost increase, and costs can exceed budget while schedule is preserved; each can occur independent of the other. Therefore, it makes sense to evaluate EVM performance indices independently when considering the relevance of past performance to future performance.
- On Cost, Over Schedule. A task can be on cost but can take longer to complete than planned. For example, staff ramp-up may be going slower than planned or critical staff may still be assigned to other tasks and available to work part-time only. A higher-priority project may delay access to facilities or equipment, or cause personnel to be reassigned.
- On Schedule, Over Cost. A task can be on schedule but can cost more than planned. For example, additional staff or more expensive staff than originally planned may be required to preserve the schedule. Equipment or facility costs can be higher than anticipated but still not affect the schedule.
The cost variance and cost performance index represent the financial state and vector of the program. As of the status date, it indicates how much the actual cost differs from the budgeted cost of the work actually completed, and how efficiently that work is being done. Adding the budgeted cost of work remaining divided by the Consumer Price Index (CPI) to the actual cost as of the status date yields the estimate at complete (EAC), as shown in Exhibit 1. These processes are fairly well defined and various formulas exist for calculating EAC based on the cumulative cost performance index, trailing 3 or 6 months cost performance index, or combinations of CPI and schedule performance index.
The schedule variance and schedule performance index are “dollar-ized” representations of how much the schedule is behind as of the status date. Another way to look at this information is to see how far back in time it was that the planned value equaled the current earned value, or “When did the current budgeted cost of work performed equal the budgeted cost of work scheduled?” The answer to that question is how far the schedule is behind---so far. If the work required to catch up is performed as inefficiently as the schedule performance index would indicate, the schedule is further behind by that additional amount. Move remaining durations forward of the status date and apply the schedule performance index to all the remaining work to get an estimate of the schedule at complete. The effect of this on schedule is shown in Exhibit 1.
Putting it All Together
Linking earned value performance and risk requires that cost performance index and schedule performance index be applied to cost and duration estimates for tasks that remain after the status date, and that probabilistic tasks and branches be included in the schedule, including a distribution around point estimates for cost and duration for specific tasks. A Monte Carlo simulation can then be used to conduct a schedule risk analysis, yielding a distribution of cost and schedule estimates at complete. Exhibit 2 shows a conceptual representation of the results of this approach. The distribution of cost and schedule in the graph shows a symmetrical distribution resulting from uncertainty in the cost and duration estimates, while the box to the right shows what the distribution looks like when discrete risk events are included in the analysis.
In order for a Monte Carlo simulation to be run, the schedule has to be “dynamic.” That is, tasks need to be free to move according to schedule logic for each run of the simulation. This is also necessary for a realistic critical path to be determined. This is harder than it seems, and personnel need proper training to develop effective schedules. Fortunately, some tools (such as Pertmaster Risk™) provide automated schedule-checking features that highlight most problems, but even that cannot correct poor schedule logic.
General uncertainty around cost and duration estimates will result in a symmetric distribution, while including probabilistic events from the risk register will cause one or more secondary peaks to occur, as shown in the box to the right of Exhibit 2. The program manager can select a desired level of confidence and identify the appropriate date and cost at complete to report to stakeholders. However, nothing substitutes for disciplined execution. Telling the team that the program has only a 63% chance of meeting the deterministic schedule can become a self-fulfilling prophecy. Use this information to manage the expectations of external stakeholders, but internally, the program manager should establish a program culture of rigorous execution according to plan.
Control accounts are “mini-contracts” that make up the program’s performance measurement baseline. Uncertainty in the cost and schedule estimates can result from unclear requirements and lack of experience or relevant data on which to base the estimate, and increases the tendency to inflate them. Forcing an unrealistically low estimate dooms the program to poor EVM metrics. Control account managers will identify risks at their level, which may create additional tasks or branches in the schedule. These additional tasks will themselves have uncertainties associated with their cost and duration. Creating a dynamic program schedule allows the program to conduct a schedule risk analysis using Monte Carlo simulation that results in a cumulative probability for cost and schedule, based on task uncertainty and probabilistic risk events and branching. By applying cost performance index and schedule performance index to remaining costs and durations before running the Monte Carlo simulation, an integrated view of the impact of risk and EVM on program cost and schedule can be obtained.
Defense Contract Management Agency. (October 2006). Earned Value Implementation Guide, United States Department of Defense. Retrieved on June 27, 2008, from http://guidebook.dcma.mil/79/EVMIG.doc
Defense Acquisition University. (2001). System engineering fundamentals. Retrieved June 8, 2008, from http://www.dau.mil/pubs/gdbks/sys_eng_fund.asp
Department of Defense. (July 30, 2005). MIL-HDBK-881A Department of Defense Handbook, Work Breakdown Structures for Defense Materiel Items. Washington DC: United States Department of Defense.
Fleming, Q., & Koppelman, J. (2000). Earned value project management. Newtown Square, PA: Project Management Institute.
National Defense Industrial Association Program Management Systems Committee Risk Management Working Group. (2002). Integrating risk management with earned value management. Retrieved June 21 2008, from https://acc.dau.mil/CommunityBrowser.aspx?id=17784&lang=en-US
OUSD (AT&L) Systems and Software Engineering, Enterprise Development. (August 2006). Risk management guide for DoD acquisition (6th ed.)(Version 1.0). Washington DC: United States Department of Defense.
Under Secretary of Defense for Acquisition, Technology, and Logistics Memorandum. (July 3, 2007). Use of Earned Value Management (EVM) in the Department of Defense. Washington DC: United States Department of Defense.
© 2008, Bill Shepherd
Originally published as a part of 2008 PMI Global Congress Proceedings – Denver, Colorado, USA