Decision analysis in projects
utlity and multi-criteria decisions
| || |
John R. Schuyler
The decision analysis approach to evaluation analysis is based upon appraisal. Available alternatives are valued, and the best one implemented. In evaluating a particular alternative, a value is calculated for each possible outcome. These outcome values are then weighted with their probabilities of occurrence. The probability-weighted average value is called expected value (EV).
Most business decisions are based upon monetary value. The cost/benefit/risk analysis focuses on the calculation of expected monetary value (EMV). Recall that EMV is EV present value (PV).1 Maximizing EMV is a suitable decision policy for most business situations.
Some circumstances are not appropriate for EMV maximizing. There are two situations, in particular, where this arises:
- The possible outcome values are considered large by a conservative decision maker.
- Money is not the appropriate measure, or, at least, not the sole measure of value. Non-money considerations cannot be converted easily into monetary-equivalents.
This article shows how utility theory provides a logical, consistent way to deal with these situations. Utility is a measure of value reflecting the preferences of the decision maker based upon beliefs and values. The EV concept applies for decision-making under uncertainty. However, outcome value will now be measured in utility units instead of PV. The decision rule is to choose the alternative having the highest expected utility (EU).
Conservative Risk Attitude
In decision analysis, probabilities are used to represent judgments about uncertainty. All of the analysis inputs should be judged as objectively as possible (i.e., without bias). Properly done, the analysis results in an unbiased distribution of project PVs. The distribution shows the range of possible outcomes and their relative likelihoods of occurrence. The risk-neutral decision maker wants to know the EMV for each alternative in order to choose the best one.
Using EMV presumes that the decision maker is risk-neutral. What does it mean to be “risk-neutral” or to be “conservative”? The following example illustrates the difference.
Suppose your company is having an expensive component manufactured. You are negotiating a cost-plus fabrication contract with the manufacturer. Discussions have been open and all data, e.g., cost details, have been shared.
There is a major uncertainty affecting cost. With a cost-plus contract, your company bears the risk. Assume that other factors, such as performance and schedule, are fixed.
The possible cost outcomes, to your company, and their probabilities under the cost-plus contract are:
Best Outcome = $1 million cost, p = .90
Worst Outcome = $4 million cost, p = .10
The EV cost is then:
[Feel free to factor these amounts, if desired, to make the outcomes more closely represent the range of costs that you encounter in your business. Also, although the discussion uses dollars, any currency will do.]
You are about to close the deal when the vendor offers to change the contract to a fixed-cost contract. They ask, “What fixed price would you be willing to pay, instead?”
Your answer has three possibilities:
- You are risk-neutral if you would be indifferent between paying a fixed $1.3 million or accepting the cost-plus contract outcomes and risks.
- If you would be willing to pay somewhat more than the $1.3 million EV cost, then you are conservative or risk-averse. For example, you might pay $1.5 million if the price was guaranteed. This is a common decision behavior.
- If you prefer the risky, cost-plus contract compared to a fixed price price at EV cost, then you are risk-seeking in this situation. This is unusual behavior except when there is entertainment value in gambling or when striving to reach an unlikely goal. For example, people pay more than the EMV payoff when playing casino games or when buying lottery tickets.
Conservative behavior is widespread. It is important to decide whether this should be part of decision policy. For small decisions, decision makers can afford to be risk-neutral. However, when the outcomes become large compared to the reference net worth, then adjustments for risk attitude can become substantial.
For large, publicly held corporations and governments, risks are shared by many individuals. For these entities, the author believes the EMV decision rule is appropriate.
For individuals and small, closely held companies, a conservative risk policy is suitable. The next section describes how to modify a decision analysis for a conservative risk attitude.
Utility Function. In this section, we'll assume that maximizing monetary value is the objective. The objective value measure, then, would be PV. In the probabilistic sense, the objective measure is EMV.
Risk aversion is exhibited in what economists call the “law of diminishing marginal utility.” For example, for most readers, winning a $1 million prize would be a tremendously exciting event. However, if one already has $10 million, the added $1 million would hardly be noticed. Incremental amounts decrease in incremental value as value is accumulated.
For conservative persons, value is not a linear function of PV. That is, twice the positive PV does not represent twice as much value. What we need is an equation or graph to convert PV dollars into measures of value. Decision theorists use the word utility as synonymous with value. Utility units are sometimes called utils.
The curve shown in Figure 1 is an example utility function. It translates objective value, PV (x-axis), into perceived value, utility (y-axis). The scaling of the y-axis is arbitrary, including choice of origin. The straight line at 45° in the figure represents the utility function of a risk-neutral decision maker. Utility curves that are concave downward represent conservative risk attitudes.
Many decision analysts scale the y-axis so that the worst possible outcome has utility = 0, and the best possible outcome has utility = 1. The shape of the curve, regardless of shape or scale, is usually deduced from the decision maker's answers to a series of hypothetical decisions.
Figure 1. Example Utility Function
The author prefers an exponential utility equation that provides utility measured in more tangible units, what he calls risk-neutral (RN) dollars. This makes the utility measure more meaningful:
RN$-1 million cost is 1 million times worse than a $1 cost.
RN$1 million benefit is 1 million times better than a $1 benefit.
Note that for positive PVs (benefits), incremental utility value decreases as PVs gets larger. However, costs or losses are amplified for negative PVs.
Note that for small decisions, near zero PV, the straight line and the curve are nearly coincident. The effect is that a value determined by EMV is nearly identical with the value obtained when additionally recognizing risk-aversion.
Solving for Certain Equivalent. Let's see how the utility function can be used to make decisions. An EMV decision tree for the contract example is shown in Figure 2. The objective cost forecast and assessment is the EV of the possible outcomes. The cost-plus contract alternative and the fixed-price contract alternatives have equal value to a risk-neutral decision maker when x = $1.3 million. Recognizing a conservative risk policy requires a slight adjustment to the method.
Figure 2. Decision Model for EMV Policy
Assume the company's risk policy is the utility function we saw in Figure 1. Some theorists suggest that this curve is appropriate to a company with a net worth of about $60 million.2 The degree of risk aversion is usually proportional to the size of the company.
To determine the fixed-price contract equivalent to the cost-plus contract uncertainty, we first calculate the expected utility of the Cost-Plus Contract. From the utility function, we read:3
U($1 million) = RN$-1.0517 million
U($4 million) = RN$-4.9182 million
Calculating expected utility (EU, actually EV utility),
EU = .9(-1.0517) + .1(-4.9182)
= RN$-1.4384 million.
[The additional decimal places are shown to demonstrate the calculation method rather than to imply that tremendous precision is forthcoming.]
Utility is suited for comparing alternatives that all have uncertainties. Here, however, we are comparing an uncertain alternative to a fixed outcome alternative. EU and dollars cannot be compared because of different units.
What we are trying to find is what equivalent cost, $X, shown on the fixed-price alternative is equivalent to the cost-plus alternative. $X represents what is called the certain equivalent (CE).
We can use the utility function to translate EU into CE. In Figure 1, we see4 that EU = RN$-1.4384 corresponds to a PV of $1.3439 million, the CE. Thus, for this company, a fixed $1.3439 million contract is equivalent to the cost-plus contract.
The approximately $44,000 (1.3439 - 1.3 million) difference between CE and EMV is called the risk premium. This will also be found in most other business situations. The risk premium is the additional amount the company is willing to pay to avoid the risk. Note that CE and EMV are nearly the same in this case. Figure 3 shows the completed analysis based on an expected utility decision policy.
Risk Policy. In decision analysis, alternatives are evaluated with either their EUs or CEs. The EMV decision policy is a special case; CE = EMV when the utility curve is a straight line. Either EU or CE criterion results in the same decisions. However, when one alternative is a fixed dollar amount, conversion to CEs is necessary. For an EU decision policy, the author evaluates every node's EMV, EU, and CE when back-solving decision trees. Interestingly, we sometimes find that the highest EMV alternative is not the same as the highest EU (or CE) alternative.
DECISION ANALYSIS IN PROJECTS
This is the tenth in a series of twelve tutorials about the probabilistic methods of decision analysis. This installment discusses decision policies for where an objective, monetary assessment of value is inappropriate.
Readers may submit written questions and comments on this series to the author via PMI Communications.
1. Expected Value - The Cornerstone. Representing a probability distribution as an unbiased, single value.
2. Optimal Decision Policy. Appraising value or cost: a consistent approach suited to all decision types.
3. Decision Trees. Graphical decision model and EV calculation technique.
4. Value of Information. Evaluating an alternative to acquire additional information.
5. Monte Carlo Simulation. An alternative, popular technique for calculating expected values and outcome probability distributions.
6. Other Probabilistic Techniques. Other established and new probability techniques suited to simple situations.
7. Modeling Techniques – Part I. Project and cash flow projections: approaches, tools and techniques.
8. Modeling Techniques – Part II. Sensitivity analysis; correlation; dynamic models.
9. Judgments and Biases. Encoding expert judgments about risks and uncertainties.
10. Utility and Multi-Criteria Decisions (this article). Decisions involving objectives other than maximizing monetary value.
11. Stochastic Variance. Decision analysis provides improved accuracy in probabilistic calculations.
12. Summary and Recap. Analysis recommendations and management issues.
In Monte Carlo simulation, the utility outcome is one of the simulation run output variables. Every trial scenario's PV is converted to utility units. EU is calculated (an approximation) as the arithmetic average of the outcomes measured in utility units.
If you are using Monte Carlo simulation, the author recommends providing the decision maker with risk versus value profile curves. This is the natural presentation format for simulation. Most decision tree software provides similar, although stairstep-looking, graphs. Example simulation-produced curves are shown as Figure 4. These are cumulative probability distributions for PV outcomes.
When the curves cross and when the best-EMV alternative is riskier (wider), then risk policy is needed. As shown, it is an unfortunate reality that alternatives with better values are usually associated with greater risk. Which alternative has the best risk versus value profile? The curves may be visually compared by a decision maker using intuition to make the decision. A more consistent (recommended) way is to convert each curve into its EU (or CE) and to choose the best alternative based upon EU (or CE).
The utility function is a succinct, complete risk policy for an organization or individual decision maker. This concept is very useful for a conservative company that wants to delegate decision making downward. This enables decision makers at all levels in the organization to make consistent risk versus value tradeoffs.
Note that the decision analysis keeps separate: risk attitude, time value of money, and judgments about uncertainty. This decomposition permits more logical and consistent evaluations.
Multi-criteria decision making (MCDM) is an approach for problems where value is multi-dimensioned. For example, a city government faces tradeoffs between schools, roads, and police protection. Utility theory provides a logical, consistent way to solve such problems.
This section provides three ways to deal with MCDM problems. The first is to simply recast the problem to focus solely on economic value. The second approach converts non-money considerations into monetary-equivalents. The third approach employs a multi-component value function.
Figure 3. Decision Model for EU Decision Policy
Figure 4. Comparing Two Alternatives
Monetary Value Objective Approach. In project management, goals commonly relate to cost, time, and performance. How are tradeoffs to be made between these three dimensions? Understanding and quantifying tradeoffs are difficult, unless one has a value model.
Consider a simple, two-facet problem. Suppose your project plan is in trouble. The main alternatives are (1) spend an additional $1 million to complete on time or (2) finish three months late but within budget. Which alternative is better?
Figure 5 maps project value according to two dimensions, cost and schedule. Project value is measured in dollars. In a situation where the outcome is uncertain, this means EMV. The wavy lines are iso-value contours and are obtained from the project feasibility model. The contour undulations reflect the seasonality of the business. Interpolating between the curves:
Alternative 1. Cost overrun.
EMV = $2.4 million
Alternative 2. Late.
EMV = $1.5 million
Both alternatives have unattractive outcomes compared to the original feasibility study. However, if maximizing EMV is our corporate decision policy, then Alternative 1 is the clear and logical choice.
Monetary-Equivalents Approach.Monetary value is a convenient measure for comparing decision alternatives. Most people already recognize corporate value in terms of dollars. Increasingly, decisions in government and not-for-profit organizations are also based primarily upon monetary value.
Sometimes, developing a comprehensive value model is impractical or inappropriate. Much of the time, costs or monetary value represents 80 percent or more of the decision basis. Minor additional considerations can often be included in the analysis by factoring in monetary-equivalent values for the side influences.
Consider the situation where a national oil company was locating a new refinery. One site was near an urban area that already had much of the needed infrastructure and had good availability of labor. Another alternative was to locate the refinery in an impoverished, rural area where jobs were greatly needed. Most of the decision was being driven by the refinery's value in the nation's economy. Since the company was government-owned, it was also important to consider and balance other sub-objectives, such as improving employment opportunities and technology transfer.
All that is needed for the substitution approach are ways to translate each non-monetary criterion into dollar equivalents. For example, it may cost $5,000 to create a new job in the local economy. The value and cost of job creation can be assessed by economists and added into the cash flow projections. Then the decision could be based upon EMV in the usual way.
Figure 5. Project Value Contours and Two Alternatives
Figure 6. Simple Objectives Hierarchy
Figure 7. Three Utility Scales
Multi-Criteria Value Functions. For MCDM problems, the principles of decision analysis still apply. The key and challenge is devising a way to measure value. Often multiple objectives are involved, usually the result of having multiple stakeholder groups. Different stakeholders, of course, have different preferences.
EMV-maximizing businesses also use MCDM on occasion. It is sometimes impractical to develop a model to characterize different alternatives’ impact on corporate value. Examples are hiring decisions and employee evaluations. While possible, it is usually cumbersome to attribute incremental cash flow to an employee; the cash flow benefits are too hard to quantify.
In project management, decisions are often made considering impact on cost, schedule, and performance. These can be thought of as forming a hierarchy as shown in Figure 6. This example structure has only two levels, but sometimes objectives hierarchies have many levels.
A popular way of constructing the hierarchy is the analytic hierarchy process.5 This technique uses subjective, pair-wise comparisons to determine the numeric weights of decision criteria and criteria scores for candidate alternatives. Regardless of whether a hierarchy structure is used, MCDM uses a value function which combines various attributes of the problem.
Consider a software project manager thinking in terms of cost, schedule and performance. Because she wants to be systematic in her thinking, she develops a utility scale for each dimension, as shown in Figure 7. The y-axis of each scale represents “goodness” or value in the context of the respective attribute. The x-axes are scaled according to the possibilities in this project.
The utility axes have arbitrary scales. Each function is unique to the sub-objective and can incorporate the decision makers's attitude toward risk.
Once the manager has the relevant quantitative attribute value measures, she can weight each attribute. The combination is a single objective function, often in the form:
Utility = wt1 x Utility1 + wt2
x Utility2 + wt3 x Utility3
where weights (wti) are chosen so that the value behaves according to her intuition about the problem. A linear form is most often chosen for the value equation. Other forms, such as having multiplicative terms, have been found useful.
This valuation approach is obviously highly subjective. The strength of the process is in providing a structure that helps keep the process logical and complete.
With a way to measure value, the project manager can do decision analysis problems as they arise during the project. To recognize uncertainty, outcome values are weighted with probabilities in EV calculations, and the choice is always to pick the alternative having the greatest EU.
Figure 8. Decision Criteria Distributions
Figure 9. Distributions of Outcome Value
Consider the decision about whether to staff the software project for normal or fast-track development. The difference between alternatives is the number of programmer/analysts (five or ten) assigned to the project.
A project model captures the project team's understanding of the development process and the workteam dynamics. Example input variables include application complexity, coding productivity, changing requirements, design performance, and amount of rework that will be needed.
The simulation model is run for each staffing alternative. On each trial, the completion time, system performance, and development cost are determined. These are converted to value components using the respective utility functions in Figure 7. The component utilities are combined with the weights assigned to each criterion. Component utilities use 0 to 1 scales, and the weights are fractions totaling 1. Therefore the utility value function also ranges from 0 to 1.
Running the Monte Carlo simulation model produces the following analysis results:
Normal Staffing: EU = 0.723
Fast-Track Staffing: EU = 0.693
Based on EU, the project manager logically chooses normal staffing. Figure 8 shows the three criteria distributions, and Figure 9 shows the composite value distribution. The graphs provide supplemental information to check and to enhance her intuition about the project.
This article has shown three ways to structure decision policy when there are several objectives or sub-objectives. The starting point is always: Be clear about the objectives. The author recommends that decision makers focus on a single objective, which may have several components. Multiple objectives conflict and lead to confusion as to what is trying to be achieved and optimized. Consider these example single objectives:
- For corporations: Maximize shareholder value.
- For governments: Maximize the quality of life for its citizens.
- For professional associations: Maximize the organization's value to members.
- For individuals: Maximize personal happiness over a lifetime.
In multi-criteria problems, management should be clear about whose interests are being represented and what is to be accomplished. Also, multiple criteria are sometimes used because it is inconvenient to fully develop the project model to assess value.
All approaches discussed in this article employ a means to arrive at a single value measure for possible outcomes. The decision analysis approach explicitly recognizes risks using probabilities, which are used to weight the possible outcomes. This is a better technique than simply recognizing “risk” as another criterion. Decision analysis provides the logical, consistent way to make decisions under uncertainty. The next article in this series shows how the improved accuracy of decision analysis calculations should be recognized when assessing project results.
1. The first two articles in this series discussed expected value and decision policy based upon expected monetary value decision policy. “Decision Analysis in Projects: Expected Value – The Cornerstone,” PM Network, January 1993, pp. 27–31 and “Optimal Decision Policy,” PM Network, April 1993, pp. 31–34.
2. The risk aversion coefficient (see note 3) is typically one-fifth or one-sixth of the net worth of the organization. The author contends this is far too low for widely held, public companies.
3. Actually, these values were calculated from the utility function's equation,
U(x) = 4(1 - e-x/r) with r, the risk
= $10 million.
4. Again, an equation was used, because using it has more precision than reading the graph. The inverse transform is CE = (-r)loge(1- EU/r).
5. The popular AHP technique is discussed by its inventor in: Saaty, Thomas L., 1994, “How to Make a Decision: The Analytic Hierarchy Process,” Interfaces, v. 24, n. 6, Nov-Dec, pp. 19–43, and somewhat less enthusiastically in Zehedi, Fatemeh, 1989, “The Analytic Hierarchy Process – A Journey of the Method and its Applications,” Interfaces, v. 16, n. 4, Jul-Aug, pp. 96–108.
Highly recommended: Interfaces is the leading journal communicating to managers about operations research (management science) applications. Most articles can be understood by non-mathematicians. Interfaces is available in most university libraries or from INFORMS—the Institute For Operations Research and the Management Sciences, 290 Westminster Street, Providence, RI 02903.
John R. Schuyler, PE, CMA, is principal of Decision Precision®, an Aurora, Colorado, firm specializing in risk and economic decision analysis. He teaches Petroleum Risks and Decision Analysis in association with Oil & Gas Consultants International.
PM Network ● April 1995