Expect the unexpected: That's a basic job requirement for organizations and their team of project managers. But as they work to reduce uncertainty around projects, organizations must continually grapple with a balancing act between risk and reward.
CHOOSING THE RIGHT TECHNIQUE
Risk-analysis approaches should match the particular needs of a project in order to provide a good fit for users' level of understanding, notes Patricia Galloway, PhD, PMP, CEO of Pegasus-Global Holdings Inc., an energy and infrastructure consultancy in Cle Elum, Washington, USA. Problems can emerge if a risk technique is too complex, for instance. While a highly detailed technique may be useful for the team performing the risk analysis, other users of that information may not have the same level of training—and could deem a technique unnecessarily detailed for their purposes.
“In this situation, the value of the initial risk analysis is lost, potentially undermining the work to be done in the risk-monitoring phase,” Dr. Galloway says.
What tool is right for your organization? Here are the pros and cons that come with some of the most popular quantitative risk-analysis approaches:
MONTE CARLO: A simulation technique that computes the project cost or schedule many times, using input values selected at random from probability distributions of possible costs or durations. The end result: a distribution of possible total project cost or completion dates.
PRO: Real-time adjustment. Project professionals can more easily update an initiative's risk-management focus as execution moves forward. “Over time, some risk element events disappear while others move into range,” says Jack Dignum, senior vice president and COO of Pegasus-Global Holdings Inc. in Cle Elum, Washington, USA. “For example, risks linked to permitting fade as permits are obtained, while risks linked to detailed design may become more prevalent once those permits have been received.”
CON: The lure of high-tech razzle-dazzle. Because it's a computer model, some project managers are lulled into believing it consistently spits out the right answer, Mr. Dignum warns. Gut instincts are often ignored. “Project managers won't challenge the model because they believe, somehow, that the computer is ‘smarter’ than they are,” he says.
OVER TIME, SOME RISK ELEMENT EVENTS DISAPPEAR WHILE OTHERS MOVE INTO RANGE.
–JACK DIGNUM, PEGASUS-GLOBAL HOLDINGS INC., CLE ELUM, WASHINGTON, USA
SENSITIVITY ANALYSIS: A modeling technique used to help determine which risks have the most potential impact on the project.
PRO: A fairly good overview. By looking at how variables deviate from expectations, project professionals can get a grasp on how risks associated with those variables will affect an initiative.
CON: Limitation. The approach is best suited to analyzing a single condition, but reality indicates that several variables change at the same time, says Rafael Alfredo Díaz Real, PhD, PMI-RMP, PMP, senior consultant at Alpha Consultoría, a project management training and consulting firm in Mexico
[CASE STUDY]
IN ACTION: EVALUATING A MEGAPROJECT'S RISK ANALYSIS
The largest civil engineering endeavor in Europe, the Crossrail project will create major railway connections, including 21 kilometers (13 miles) of tunnels, under the London, England area by 2018. HM Treasury, the United Kingdom's economics and finance ministry, hired Pegasus-Global Holdings Inc. in Cle Elum, Washington, USA to evaluate the approximately US$25 billion megaproject's initial risk analysis.
This was a “full stakeholder involvement” risk-management program, says Patricia Galloway, PhD, PMP. Everyone involved had an opportunity to participate in the development and modeling of the full program risk profile. For example, local governmental and quasi-governmental units were invited to participate—even though at that point they had no direct financial stake in the execution of the megaproject.
That included labor unions. While they do not have authority to pass laws, they do regulate the conditions under which their respective memberships will work, notes Jack Dignum of Pegasus-Global. “Given the high level of labor needed to execute the project and the fact that there would be significant competition for that labor pool throughout the duration of the project, the unions were invited to participate in the development of the risk profile for the project,” he says.
The total risk profile was divided into three types (delivery, program and sponsor risks) and included seven steps:
1. Risk Identification, Probability and Impact Quantification. All risk elements received from stakeholders were categorized in a complex matrix, which considered such things as the location of individual tasks. The result was the development of a detailed risk register that reported risk elements and impact quantification.
2. Risk Level Review. A committee was established to manage and control the risk-management program following development of the risk registers. The review committee was composed of commercial, planning and legal departments of the project sponsor, the U.K. Department for Transport.
Stakeholders created their own risk registers, with the Crossrail risk-management team adding each list to the total register and then culling the combined profile of duplications.
When the risk-management team asked the participants to review the combined register, there was significant pushback by participants, who complained that their specific risks had been dropped, Mr. Dignum says. “It was described as a series of meetings where no one was happy, everyone thought the other participants' risks were less important than their risks, and where no one would agree to give up anything,” he says.
To address the problem, the Department for Transport had participants from its commercial, planning and legal departments form a “risk review committee.”
The first task was to educate those committee members about the purposes for a risk model and how one worked. They were then given the full risk register to analyze and “rationalize” it to other stakeholders.
“Once that process was developed with the input of the committee, it was the committee that presented it and defended it to the risk-participant universe,” Mr. Dignum explains.
Construction of the Crossrail Tunnel portal west of Royal Oak Station, London, England
PHOTO COURTESY OF WIKIMEDIA
3. Risk Leveling. “The extremely high number of individual risks identified—several thousand—resulted in modeling being impractical,” Dr. Galloway says. It would take too much time to run models, and the results would be too unreliable over multiple runs. To address that concern, the review committee coalesced the highest risks in terms of probability and consequence into a document called the “Level 1 Risk Register.” That reduced the total number of risks, as well as the what-if combinations among various risk elements, she explains. The result: a practical model and a higher level of reliability in the model runs.
4. Risk Modeling. Risk analysis software with Monte Carlo simulation was used to run probabilistic models of the Level 1 Risk Register. The primary purpose of the model was to confirm or question the contingency cost amounts contained in the full megaproject cost estimate in preparation at that time.
5. Risk Model Reviews. The committee reviewed the results of the model runs, adjusting the baseline assumptions to refine the risk register and model assumptions. This review was conducted at the individual risk element level. Each change was fully documented and reported back to the stakeholders who had participated in the assembly of the original risk profile.
6. Model Adjustment. Based on committee reviews and considering stakeholder comments, the model was adjusted and rerun. Steps 4, 5 and 6 were repeated several times as the project definition was refined.
7. Final acceptance. “Our audit found the risk-management program to be prudent and opined that it met—and in a number of instances, exceeded—the standards in the transportation industry,” Dr. Galloway says.
City, Mexico. “Focusing on one change in one single variable is quite unrealistic in a realworld situation.”
2 in 3 business executives worldwide say their company's focus on risk management has increased since the 2008 financial crisis, but just 1 in 10 say their executive management is highly effective in creating a strong risk-management culture.
Source: Harvard Business Review Analytic Services and Zurich Financial Services Ltd. survey of 1,419 executives worldwide
DECISION TREE: This diagramming technique (using decision and chance nodes) shows possible outcomes of a decision, weighing cost and other factors.
PRO: Provides options. The tool explicitly reveals the various options available on a project—all in an easy-to-glean model. “If an aspect of a project is going very poorly, it can be stopped,” explains Charles Linville, PhD, president and founder of Ploughman Analytics, a data management business analytics consultancy in Champaign, Illinois, USA. “If an aspect of a project is going well, additional resources can be invested.”
CON: Lack of buy-in. Because it is often calculated with “educated guesses” by the project manager or team, upper management often doubts its accuracy, Dr. Díaz Real warns.
WHAT GEOGRAPHY HAS TO DO WITH IT
Risk management isn't as ingrained in some parts of the world. In the Latin American region, for example, risk management is still “in its toddler years,” says Rafael Alfredo Díaz Real, PhD, PMIRMP, PMP, Alpha Consultoría, Mexico City, Mexico.
Most large organizations carry out a risk-management process during project feasibility stages, which is generally repeated (but not reanalyzed) for the planning stage, he says.
If only a couple months have passed since a feasibility study, the risk-management process may be valid. But because it often takes at least six months to go from feasibility to planning, risks should be reviewed, Dr. Díaz Real advises.
Organizations tend to emphasize risk identification but neglect risk management as an integral process, he says.
“In my experience, although there may be a policy on this regard, it is rarely followed, as project teams are busier executing the project and solving risks turned into issues once they happen,” Dr. Díaz Real says. “There is not enough prevention culture.”
This oversight is reflected in staffing choices: In his experience, risk managers are rarely assigned, and the task falls on the project manager's shoulders, regardless of the size of the initiative. As a result, the project manager becomes overburdened, and risk management falls by the wayside.
Parts of the Middle East also suffer shortcomings when it comes to assessing risk, according to Imran Malik, PMP, director of the customer program management office for Du, the Emirates Integrated Telecommunication Company, in Dubai, United Arab Emirates.
“Organizations very often ignore risk management in our part of the world,” he says, citing two primary reasons:
1. There is a lack of organizational project management maturity in Middle Eastern markets.
2. The limited amount of qualified and skilled project professionals in the region keeps them too busy managing constraints to see the big picture.
“That blurs the program vision and hinders the project manager's ability to identify future risk,” he explains.
RISKS WORTH TAKING
Risk management profiling shouldn't be the only basis for decisions, notes Patricia Galloway, PhD, PMP, Pegasus-Global Holdings Inc., Cle Elum, Washington, USA. Overreliance on risk management could cause organizations to shy away from innovation.
“While a first-of-a-kind technology project may hold risks due to unknowns, those risks may be assumed in order to gain advantage of the benefit of the project results when completed,” Dr. Galloway says. And depending on the nature of the project, there may also be factors (such as tax incentives) that would offset some of the assumed financial investment risks, she says.
SENSITIVITY ANALYSIS IS BEST SUITED TO ANALYZING A SINGLE CONDITION, BUT REALITY INDICATES THAT SEVERAL VARIABLES CHANGEATTHE SAME TIME.
—Rafael Alfredo Díaz Real, PhD, PMI-RMP, PMP, Alpha Consultoría, Mexico City, Mexico
IT'S ALL IN THE RELATIONSHIPS
For an effective assessment of uncertainty, organizations need to recognize relationships among projects, Dr. Linville says. Determine:
- Dependence on common resources, such as personnel, equipment or budgets
- Dependence on the same uncertain quantities
- Interdependence (for example, Project A won't work unless Project B is successful)
“If a project management team is overseeing something that it has done many times before, following carefully documented procedures that pretty much always yield the same results, it may have had a chance to become very well-calibrated,” Dr. Linville says.
But that's not always the case. Many projects involve a significant amount of improvisation—and that's where an iterative approach helps to mitigate risk, he says. “We iterate because we often don't know what's going to be hard and what's going to be easy, what's going to be expensive and what's going to be affordable, what's going to be important and what is going to be immaterial,” he says. “We iterate because it's efficient to iterate.”
Organizations must also be willing to consider subjective assessments of uncertainty, Dr. Linville says. And many managers are loath to do so.
“Numbers people in corporations—accounting, finance, and in some cases, those who practice predictive analytics—often like empirical data, and if empirical data aren't available, they throw up their hands and say, ‘We don't know.’ Well, it doesn't matter if they know or not. The firm is going to be making a decision,” he says.
Even if it makes subject-matter experts uncomfortable, there are times when project professionals must elicit subjective assessments of uncertainty from them. But, they should do so with the understanding that people known as “experts” tend toward overconfidence, Dr. Linville warns.
“I'm not talking about overconfidence about the success of a project, although that may be present as well,” he says. “I'm talking about insufficiently recognizing the existence of uncertainty.”
SHARED RESPONSIBILITY
A careful analysis of risk can help organizations pinpoint worthwhile investments (and avoid bad ones). But there is no absolute answer in risk management, and it shouldn't fall on a single person's shoulders, Dr. Díaz Real notes.
“Tools may provide data—and in the best case, educated information. But the final decision and the responsibility falls on the project manager and the sponsor,” he says. “Both are part of the team, and decisions as important as these cannot be made alone.” PM