When planning your project, suppose you had used a stochastic (probabilistic) method to forecast Completion Time. What if, though extraordinary coincidence, each activity is completed in exactly time of the best original time estimate? Would this ensure that the project is completed exactly as forecast? Generally, no. This article explains stochastic variance, that portion of the deviation, actual minus estimate, attributable to the stochastic calculations.
Many people first become interested in risk and decision analysis as a way to characterize the uncertainty in estimates. A even greater benefit is more-accurate best estimates for decision making. Sometimes, the difference between forecast and actual can be substantial through no fault in project execution. When reconciling a forecast with the project's actual outcome, the variance analysis should separately recognize the component attributable to stochastic variance.
Introduction
In this series, we've used the terms: deterministic for traditional project models using single-point inputs; and stochastic, or probabilistic, for project models where uncertain inputs are represented as probability distributions.
The author makes a base case projection with a deterministic model using the expected value (EV or mean) for each input variable. This is but one scenario, especially useful as a reference. Although of interest, the base case scenario should not be used for making decisions. The problem is that the base case calculation often produces an incorrect forecast.
People on a project team should incorporate their best knowledge and information when making a forecast. Their judgments about uncertainty are captured as probability distributions. When probabilities are propagated through the project model calculations, the forecasts (e.g., Completion Time) come out as probability distributions. The best single-point forecast for Completion Time is the EV of the Completion Time probability distribution. EV Completion Time is often very different from the base case value. The difference is the stochastic variance. This name is in reference to variance analysis, the process of reconciling a forecast with actual.
Deterministic and stochastic calculations provide different best estimates. This surprises many people who thought decision and risk analysis were embellishments useful mainly to characterize uncertainties. This calculation difference is bothersome: In order to get the EV projection one needs to do calculations with probabilities. Anything less rigorous will likely introduce biases in the forecast.
In project management, one source of stochastic variance is the merge bias. This was introduced earlier in the series, and readers are encouraged to review the previous example.1 It showed a simple case where one activity's start was dependent upon three other activities being complete. The deterministic solution had the successor activity starting in 11.0 days, while the EV start under uncertainty was 13.1 days. The 2.1 days difference, merge bias, is the effect of uncertainty in the network project model. This is one of the stochastic variances that can arise.
Table 1. Spreadsheet Model
| Distribution | Correlation | |||
| Variable | Statistics | Type | Coefficients | Assumptions |
| Volume | 10 2 | lognormal | independent | 10.000 |
| Price | 30 5 | normal | -0.5 | 30.000 |
| Cost | 140 180 280 | triangle | -0.5 | 200.000 |
| NCF | ||||
| 100.0 |
Figure 1. Distribution for NCF
Variance Analysis
Variance analysis identifies the component causes for an actual outcome deviating from its forecast or budget. [Note: Do not confuse this variance analysis with the variance statistic.]
This article describes how variance components can be recognized when decision analysis techniques are used to make projections. The difference between forecast and actual:
Total Variance = (Actual Outcome) – (Forecast Outcome)
will be split into two components:
Total Variance = (Deterministic Variance) + (Stochastic Variance)
Decision analysis provides a more accurate forecast for many problems. It also helps the decision maker better understand the business or proposed transaction. A variance analysis format explains what happened from the context of the forecast calculation approach.
DECISION ANALYSIS IN PROJECTS
This is the eleventh in a series of twelve tutorials about the probabilistic methods of decision analysis. This installment presents a format for recognizing the correction component arising when stochastic methods are used for project forecasts. Readers may submit written questions and comments on this series to the author via PMI Communications.
1. Expected Value – The Cornerstone. Representing a probability distribution as an unbiased, single value.
2. Optimal Decision Policy. Appraising value or cost: a consistent approach suited to all decision types.
3. Decision Trees. Graphical decision model and EV calculation technique.
4. Value of Information. Evaluating an alternative to acquire additional information.
5. Monte Carlo Simulation. An alternative, popular technique for calculating expected values and outcome probability distributions.
6. Other Probabilistic Techniques. Other established and new probability techniques suited to simple situations.
7. Modeling Techniques – Part I. Project and cash flow projections: approaches, tools and techniques.
8. Modeling Techniques – Part II. Sensitivity analysis; correlation; dynamic models.
9. Judgments and Biases. Encoding expert judgments about risks and uncertainties.
10. Utility and Multi-Criteria Decisions (this article). Decisions involving objectives other than maximizing monetary value.
11. Stochastic Variance (this article). Recognizing that decision analysis provides improved accuracy in probabilistic calculations.
12. Summary and Recap. Analysis recommendations and management issues.
Forecasts Are Usually Wrong, But Can Be Improved. The stochastic approach is more accurate because it correctly recognizes how the uncertain variables interact. The stochastic variance can arise from common situations. Asymmetry and non-linearity are perhaps the most common causes.
Sources of stochastic variance include:
- The merge bias; schedule is affected by an activity only when it either enters or leaves the critical path.
- Contract bonus or penalty.
- Contingency plans and other ways subsequent decision points are represented in the project model.
- Correlation, or association, between variables; the example in this article illustrates this effect.
The stochastic modeling approach better represents reality. Further, the EV projection is an objective forecast: it is unbiased so long as the input values are unbiased. Biases can also enter an analysis through judgments about values and uncertainty. What we are considering here is bias related to the structure of the model: its formula relationships and constraints.
We started this article with the question: What if for every input variable:
Actual outcome = EV (prior assessment of parameter)?
One finds that the actual output value is equal to the value obtained with the deterministic analysis. This is often different than the EV of the forecast variable's distribution. The difference is the stochastic variance. Because this situation arises, it is important that a post-analysis recognize that a portion of the total variance can be attributed to the projection method.
Example Simulation Analysis and Variance Report
A simple model is used to illustrate a Monte Carlo simulation projection and a variance analysis:
Net Cash Flow = NCF
= Volume .Price – Cost
Let's assume your company's experts provided the following assessments for the three random variables:
- Volume: Lognormal distribution with EV = mean = 10,000 units and standard deviation of 2,000 units. Independent variable.
- Price: Normal distribution, with mean = $30/unit and standard deviation of $5/unit. Correlated with Volume with a correlation coefficient2 r= –.5
- Cost: Triangle distribution, with low = $140k, most likely = 180k, high = 280k. This has a mean of $200k and standard deviation of $29.4k. Correlated with Volume with a correlation coefficient r= –.5
Table 2. NCF Distribution Statistics
| Trials | 10,000 |
Mean,  | 95.09 |
| Standard Deviation, s | 69.79 |
Mean Std. Error,  | 0.70 |
Table 3. Example Variance Analysis
Figure 2. Cumulative Frequency Curve
The deterministic solution is simple.
Using the EV‘s for each parameter,
This would be the forecast using the conventional, single-point calculation approach.
In a decision analysis, we incorporate judgments about uncertainties as probability distributions. A simple Monte Carlo simulation model forecasts NCF as shown in Table 1. This example was developed using one of the popular spreadsheet add-in products.3 The input variables are shown at their expected values. The deterministic solution, the base case, shows NCF = $100,000.
Table 2 and Figure 1 show the principal simulation results. The EV NCF forecast is $95,090. Although the distribution in Figure 1 appears centered upon $100,000, the bias is significant. The stochastic variance is $100,000 – 95,090 = $4,910, or about a 5 percent difference. Often the stochastic variance is substantially greater or less than this.
A sample variance report for the NCF projection is shown as Table 3. The stochastic and deterministic variances should be recognized separately in a post-analysis. This table illustrates one possible format.
The joint variance in Table 3 results from simultaneous changes across multiple variables in the calculation. Cost accountants sometimes define item variances to include joint components. However, the author prefers to keep the joint contributions separate.
Figure 2 shows the cumulative distribution for the NCF projection. An example interpretation: There is about a 57 percent chance that NCF will be less than $100,000.
Notice that the graph conveys much more information to the decision maker than a single-value estimate would provide. If you were the decision maker, which would you prefer: a single value or a probability distribution graph?
Summary
Expected values (EVs) provide the best single-point assessments for each parameter in an analysis. And EV of the outcome distribution provides the best, single-point project forecast.
Even if the EV of each input parameter happens to be realized, there is usually a deviation from the EV forecast. This difference is the stochastic variance. The post-analysis should separately recognize this correction effect which arises with the more accurate stochastic calculations.
The unbiased EV forecast is a key benefit of the decision analysis approach. It is important to recognize the sources of deviation from forecast.
The variance analysis format described in this article supports a distinction between good decisions and good outcomes. With a solid decision analysis process, people are not evaluated on chance outcomes. Instead, they should be evaluated on their assessment performance and on the completeness and integrity of their analyses. 
Notes
1. Merge bias was introduced in the “Dynamic Modeling: Merge Bias” section in the “Other Probabilistic Techniques” installment of this series, PM Network, April 1994, pp. 20–24.
2. Technically, these correlation coefficients are used in the simulation model as rank correlation coefficients.
3. “Crystal Ball” and “@RISK” provide Monte Carlo simulation capabilities to Lotus 1-2-3 and Microsoft Excel (all names are trademarks).
John R. Schuyler, PE, CMA, is a principal of Decision Precision®, an Aurora, Colorado, firm specializing in risk and economic decision analysis. He teaches Petroleum Risks and Decision Analysis in association with Oil & Gas Consultants International.
