Earned value management and risk management

Introduction

In today's uncertain business environment there is understandable pressure to improve the quality of decision-making at all levels in the organization. A number of techniques have been developed to address this concern, in an attempt to introduce some rational framework to the decision-making process. Two of the leading approaches are Earned Value Management (EVM) and Risk Management (RM). These stand out from other decision support techniques because both EVM and RM can and should be applied in an integrated way across the organization. Starting at the project level, both EVM and RM offer powerful insights into factors affecting project performance. While this information is invaluable in assisting the project management task, it can also be rolled up to portfolio, programmed, departmental or corporate levels, through the use of consistent assessment and reporting frameworks.

Another key similarity between the two techniques lies in the word “management”. It is possible to conduct “Earned Value Analysis” and “Risk Analysis” to expose underlying drivers of performance. But both techniques emphasize the need to move from analysis to management, using the information to support proactive decision-making. Consequently, both EVM and RM encourage those using the techniques to take appropriate management action based on the results, and not to stop at mere analysis.

Since both EVM and RM address the same problem space (performance of projects, programmers, portfolios and businesses), and both provide management information to provide a basis for decisions and action, there has been considerable interest in the possibility of developing a combined approach to create synergistic benefits. Currently EVM and RM operate as parallel coexisting processes without systematic integration (although good project managers may intuitively link the two in practice). Much of the discussion to date on the relationship between EVM and RM has been rather theoretical, addressing the key principles underlying the two techniques. This paper however outlines practical steps that can be implemented to combine EVM and RM in order to gain maximum benefit for projects and the organization. It is assumed that the reader is familiar with the principles and processes of EVM and RM, and these are therefore not described here.

Weakness in Each Technique

The strengths of EVM and RM have been well described elsewhere, as their proponents seek to encourage wider uptake and use. Each technique however has at least one key weakness which presents a significant danger to those relying on the output to support strategic or tactical decision-making.

For EVM, one of the main perceived weaknesses (at least by non-experts) is its reliance on a key assumption, that future performance can be predicted based on past performance. Calculated performance measures (CPI, SPI, CV, SV etc) are used to predict forwards and estimate cost at completion or overall duration. Unfortunately there is no guarantee that the basic EVM assumption will be true, and it is likely that the future will deviate from that predicted by simply extrapolating from past performance. Einstein defined “insanity” as “doing the same thing over and over again and expecting different results.” Indeed the management element of EVM is designed to ensure that the future predicted by calculated performance measures does not materialize, since it encourages the taking of corrective action in response to the analysis results. In addition to being affected by the actions deliberately taken by management, the remaining elements of the project are also subject to risk, both positive opportunity and negative threat, introducing variation and ambiguity into future performance.

The strength of EVM lies in its rigorous examination of what has already occurred on the project, using quantitative metrics to evaluate project past performance. It goes on however to predict future performance by extrapolating from the past. But it is not possible to drive a car by only looking in the rear-view mirror. A forward view is also required, and this is what RM offers.

While project planning looks at the next steps which lie immediately ahead, RM has a horizon further into the future. It acts as a forward-looking radar, scanning the uncertain and unclear future to identify potential dangers to be avoided, as well as seeking possible additional benefits to be captured. However this undoubted strength of being resolutely and exclusively future-focused is also one of the key weaknesses in RM. Anything which occurred in the past is of little or no interest to the risk process, since there is no uncertainty associated with past events. RM starts with today's status quo and looks ahead. How the project reached its current position is not relevant to the risk process, unless one is seeking to learn lessons to assist RM on future projects. As a result RM as commonly implemented often lacks a meaningful context within which to interpret identified risks, since it has no means of capturing past performance and feeding this into the decision-making process.

If EVM is weakened by assuming that future performance can be predicted from past performance, and if RM is weakened by looking only forwards with no real awareness of the past, a useful synergy might be obtained if a combined EVM-RM approach were able to address these weaknesses. Combining a rear-view mirror with a forward-looking radar would use the strengths of complementary approaches to compensate for the weaknesses inherent in using each alone. Consequently it is possible to produce significant benefits by using RM to provide the forward view required by EVM, and by using EVM to provide the context required for RM.

Synergies From a Combined Approach

Given the common aims of EVM and RM to examine and expose drivers of project performance in order to focus management attention on achievement of objectives, and given their differing perspectives towards the past and the future, a number of areas of possible synergy exist between the two techniques. These are outlined in the following sections. The steps required to implement these synergies are summarized in Exhibit 1.

Exhibit 1

Creating the baseline spend plan

The foundation for EVM is the baseline plan of expected spend over time, creating the profile of “Budgeted Cost of Work Scheduled” (BCWS) or “Planned Value” (PV) against which project performance is measured. This baseline is derived from a costed and resourced project plan, including fixed and variable costs arising from financial and human resources. The BCWS profile is typically presented as a cumulative curve, or S-curve.

Good practice EVM recommends that allowance should be made in the baseline BCWS to account for uncertainty and risk, and the integrated baseline review (IBR) process should verify the robustness and suitability of this allowance. However in reality many baselines are created without proper consideration of risk, and simply include a “contingency element” for unplanned work. Indeed contingency is often hidden in the baseline to avoid its removal by management prior to project start. And this contingency is usually unrelated to defined risks, but reflects an intuitive assessment of what might be required “just in case”.

The baseline BCWS exists as the benchmark against which project performance will be measured. However one of the first things a project manager learns is that reality will never precisely match the project plan. As soon as work starts, there are variations in productivity, resource and information availability, delivery dates, material costs, scope etc. This is why a rigorous change control process is vital to successful project management. Although not all changes can be foreseen before the project starts, it is possible to assess the degree of uncertainty in a project plan, in both time and cost dimensions. This is the domain of RM. One of the first contributions that RM can make to EVM is to make explicit the consideration of uncertainty and risk when constructing the baseline BCWS.

By undertaking a full risk assessment of the project plan before the project starts, addressing uncertainties in both time and cost, it is possible to evaluate the degree of risk in the baseline project plan. Quantitative risk analysis techniques are particularly useful for this, especially the use of Monte Carlo simulation on integrated models which include both time and cost uncertainty. These risk models take account of variability in planned values, also called “estimating uncertainty” (for example by replacing planned single-point estimates of duration or cost with three-point estimates or other distribution types), and they should also model the effect of discrete risks to reflect their assessed probability of occurrence and the subsequent impact on project time and/or cost (using stochastic branching constructs, both probabilistic and conditional). Both threats and opportunities should be addressed in the risk model, representing the possibility of exceeding or failing to meet the project plan. The risk model should also take account of planned responses to risks, developed during the risk process. These must also be reflected in the expected spend profile for the project.

The results of the risk analysis allow the best case project outcome to be determined, representing the cheapest and quickest way to reach project completion. Similarly a worst case profile can be produced, with highest cost and longest duration. All other possible outcomes are also calculated, allowing the “expected outcome” within this range to be identified. These can be shown as a set of three related S-curves, as in Exhibit 2, which take account of both estimating uncertainty (variability in planned events) and discrete risks (both positive opportunities and negative threats).

Exhibit 2

The ellipse at the end of the curves represents all possible calculated project outcomes (90% confidence limit), with the top-right value showing worst-case (highest cost, longest schedule), the bottom-left giving best-case (cheapest and quickest), and the centre of gravity of the ellipse being at the expected outcome of project cost and duration. This ellipse is known by risk practitioners as the “eyeball plot” (or the “football plot” in US). It may be thought to correspond with the “Box of uncertainty” described by some EVM practitioners referring to the area bounded by extrapolation from actual cost (ACWP) and earned value (BCWP). However the risk ellipse is derived from calculations based on defined risks, rather than merely extrapolating from past performance, so it is likely to be a more accurate representation of the range of possible future project outcomes.

The existence of this set of possible project outcomes raises the question of where the baseline spend profile for EVM should be set. The recommendation from a combined approach to EVM and RM is to use the expected value cumulative profile from a quantitative time-cost risk analysis as the baseline for BCWS. In other words, the central S-curve in Exhibit 2 would be used as the baseline instead of the original S-curve. This ensures that the EVM baseline fully reflects the risk associated with the project plan (including an appropriate amount for contingency which is automatically incorporated in the risk analysis results), rather than measuring performance against the raw “all-goes-to-plan” plan.

Clearly the risk analysis must be conducted using the same units as those required for EVM, i.e. measuring cost in monetary value (£, $, € etc), or as resource cost (man hours, days, weeks, months etc). It is also necessary to use an integrated risk analysis model which can simultaneously vary time and cost, including the “cost of time” (noting that some popular risk analysis tools do not support integrated time-cost risk modeling). Finally the issue of dependency and correlation in the risk model must be carefully considered to ensure that results are realistic and feasible.

Predicting future outcomes

Both EVM and RM attempt to predict the future outcome of the project, based on information currently known about the project. For EVM this is achieved using calculated performance indices, with a range of formulae in use for calculating Estimate At Completion (EAC). Most of these formulae start with the Actual Cost of Work Performed to date (ACWP, or Actual Cost AC), and add the remaining budget adjusted to take account of performance to date (usually using the Cost Performance Index CPI, or using a combined Performance Efficiency Factor based on both CPI and SPI). These calculations of the Estimate To Complete (ETC) are used to extrapolate the ACWP plot for the remainder of the project to estimate where the project might finally end (EAC). However calculating EAC in this way does not take explicit account of the effect of future risks on project outcome. One simple way to do this is by adding an amount into the EAC calculation to account for risk-weighted contingency or management reserve.

RM predicts a range of possible futures by analyzing the combined effect of known risks and unknown uncertainty on the remainder of the project. When an integrated time-cost risk model is used, the result is a set of S-curves similar to Exhibit 2, but covering the uncompleted portion of the project. In the same way that the initial spend baseline should be determined using both risk and earned value data, the remaining element of the project should also be estimated using both sets of information.

It is also possible to use risk analysis results to show the effect of specific risks (threats or opportunities) on project performance as measured by earned value. Since the risk analysis includes both estimating uncertainty and discrete risks, the model can be used to perform “what-if” scenario analysis showing the effect of addressing particular risks. For example, if a key threat is modeled using a probabilistic branch, a “what-if” analysis can set the probability of the threat occurring to zero, simulating the result if that risk is removed. Similarly the effect of capturing key opportunities can also be shown. The result is a series of cumulative probability distribution curves (S-curves), sometimes called an “onion-ring diagram”, showing the cumulative effect of addressing key risks. This allows identification of the most significant risks which need to be addressed as a priority. If the same technique is applied to the planned spend profile (BCWS or PV), the risk analysis will reveal which risks have the greatest influence over earned value and project performance, allowing management attention to be focused appropriately.

Similarly, the risk model will show the effect of outstanding risks and planned responses on the remainder of the project, reflected in the expected result from the quantitative risk analysis, and this information must also be taken into account when determining the expected spend profile (BCWS or PV) from time-now to project completion. The standard approach to EVM allows for draw-down of contingency (or management reserves) into the baseline if significant changes arise as a result of scope change or risk occurrence. The results from the quantitative risk analysis indicate how much contingency should be reallocated into the baseline to cover the expected level of risk in the remaining portion of the project.

Evaluating risk process effectiveness

A risk can be defined as “any uncertainty that, if it occurs, would have a positive or negative effect on achievement of one or more project objectives”. RM aims to address this uncertainty proactively in order to ensure that project objectives are achieved, including completing on time and within budget. As a result, if RM is fully effective, actual project performance should closely match the plan.

Since EVM performance indices (CPI, SPI) measure deviation from plan, they can be used to indicate whether the risk process is being effective in addressing uncertainty and controlling its effects on project performance.

• If CPI and/or SPI are below 1.0 indicating that project performance is falling short of the plan, then one of the most likely underlying causes is that the risk process is failing to keep the project on course. An ineffective risk process would fail to avoid adverse risks (threats) proactively, and when threats materialize into problems the project incurs delay and/or additional cost. Either the risk process is not identifying the threats, or it is not preventing them from occurring. In this situation, management attention should be directed to the risk process, to review its effectiveness and consider whether additional resources are required, or whether different techniques should be used.
• Conversely, if CPI and/or SPI are above 1.0 indicating that project performance is ahead of plan, the risk process should be focused on exploiting the opportunities created by this situation. Best-practice RM addresses both threats and opportunities, seeking to minimize threats and maximize opportunities. When EVM indicates that opportunities exist, the risk process should explore options to capture them and create additional benefits for the project.
• It should also be noted that if CPI and/or SPI far exceed 1.0, this may indicate other problems in the project and may not simply be due to the existence of opportunities. Typically, if actual performance is much greater than expected or planned, this could indicate poor planning or incorrect scoping when setting up the initial baseline plan. If this highly anomalous behavior continues, a baseline re-planning effort should be considered, which of course will involve the need for further risk management.
• Similarly if CPI and/or SPI are well below 1.0, this may not simply be due to the impact of unmanaged threats, but may indicate problems with the baseline plan or scope.

Exhibit 3 illustrates the relationship between the values of EVM indices (CPI and/or SPI) and RM process effectiveness.

Exhibit 3

The key to using EVM indices as indicators of RM effectiveness is to determine appropriate thresholds where action is required to refocus the risk process. Clearly some variation of EVM indices is to be expected as the project unfolds, and it would not be wise to modify the risk process in response to every small change in CPI and/or SPI. However if a trend develops and crosses the thresholds of “common variance”, action should be considered. Exhibit 4 illustrates this, with the thresholds of “common variance” for CPI and/or SPI set at ≥ 0.9 and ≤ 1.25. A further “warning threshold” is set at 0.75, suggesting that an adverse trend is developing and preparatory steps should be taken.

The thresholds of 0.75, 0.9 and 1.25 used in Exhibit 4 are illustrative only, and organizations may be able to determine more appropriate threshold values by reviewing historical trend data for CPI and SPI, and identifying the limits of “common variance” for their projects.

Plotting the trend of CPI and SPI over time against such thresholds also gives useful information on the type of risk exposure faced by the project at any given point. For example Exhibit 4 indicates that the project schedule is under pressure (SPI trend is consistently below 1.0), suggesting that the risk process should focus on addressing sources of time risk. The exhibit also suggests that cost savings are possible which might create opportunities that can be exploited, and the risk process might be able to maximize these. These recommended action types are illustrated in Exhibit 5, corresponding to the following four situations :

Exhibit 4

1. Both CPI and SPI high (top-right quadrant), creating opportunities to be captured.
2. Both CPI and SPI low (bottom-left quadrant), requiring aggressive action to address threats.
3. High SPI but low CPI (top-left quadrant), requiring focused attention to cost risk, with the possibility of spending additional time to address this.
4. High CPI but low SPI (bottom-right quadrant), where attention should be paid to addressing schedule risk, and cost trade-offs can be considered.

Exhibit 5 also suggests that if either CPI or SPI (or both) remain abnormally high or low, the baseline plan should be re-examined to determine whether the initial scope was correct or whether underlying planning assumptions were unfounded.

It is important to note that these action types should be viewed only as first options, since other considerations may lead to different actions. For example in projects with high schedule-constraints (e.g. product launch, event management etc), the trade-off between time and cost may be prioritized differently than in cost-constrained projects.

Exhibit 5

Conclusion

Both Earned Value Management (EVM) and Risk Management (RM) seek to improve decision-making by providing a rational framework based on project performance. EVM examines past performance against clearly-defined quantitative metrics, and uses these to predict the future outcome for the project. RM looks ahead to identify and assess uncertainties with the potential to affect project performance either positively or negatively, and develops responses to address each risk proactively.

Both techniques share a focus on project performance, and have the same purpose of developing effective actions to correct unwelcome trends in order to maximize the likelihood of achieving project objectives. One (EVM) does this by looking back at past performance as an indicator of likely future performance. The other (RM) looks ahead at possible influences on future project outcomes.

These two approaches are not in conflict or mutually exclusive. Indeed their commonalities imply a powerful synergy, which is available through combining the complementary strengths of each technique and using insights from one to inform the application of the other (as summarized in Exhibit 1). The practical suggestions outlined in this paper indicate that if they are used together, EVM and RM provide a potent framework for managing change on a project, based on a realistic assessment of both past performance and future uncertainty.