Twenty-first century risk management
Mike Griffiths, Leading Answers
Dennis Stevens, LeadingAgile
Today's complex projects need proactive risk management to be successful. Identifying risks and evaluating risk exposure on these projects is too multi-faceted to be left to intuition or checklists. A disciplined process of systemic risk assessment and mathematical analysis is necessary to successfully understand the risk exposure. Yet risk management is worthless if threats cannot be effectively avoided, transferred, or reduced and opportunities cannot be exploited to offset the impact of threats. Risk response is largely a human-focused activity with success criteria linked to social influence. This paper discusses how to combine mathematical analysis with social influence to create an effective risk management process.
Risk Management Defined
As defined in the Software Extension to the PMBOK® Guide (PMI, 2013), “risk is an uncertain event or condition that, if it occurs, has a positive or negative effect on a project's objectives.” The Software Extension also defines the risk management process as “a continuous process for systematically identifying, analyzing, treating, and monitoring risk throughout the life cycle of a product or service.” This definition is useful in a broader context as Software Development is a well-researched part of the broader area of Knowledge Work. The aim of Risk Management is to improve the probability of achieving the project goals by avoiding or minimizing threats and by exploiting opportunities.
The Importance of Risk Management
We spend a lot of our time estimating effort on projects. Then we spend a lot of energy focusing on managing utilization of the people to deliver the project. However, many of today's projects involve high uncertainty and dynamic, interconnected systems. In these projects it is risk events, not the work effort, that have the predominant role in deciding where delivery actually occurs (Magennis, 2011).
There are multiple approaches for Risk Management available, including ones from the PMBOK® Guide, PRINCE2, the Software Engineering Institute, and Agile approaches. But these models have challenges when dealing with the complex projects involving dynamic, interconnected systems. We will draw from these multiple approaches and incorporate social influence techniques from collaborative teams and “gamification” to define “A Risk Management Process for the 21st Century.”
Models of Risk Management
The two most popular project management approaches are the PMBOK® Guide and PRINCE2. These methods outline a project-manager driven process for risk identification and management.
PMBOK® Guide v5
10.1 Plan Risk Management—The process of defining how to conduct risk management activities for a project.
10.2 Identify Risks—The process of determining which risks may affect the project and documenting their characteristics.
10.3 Perform Qualitative Risk Analysis—The process of prioritizing risks for further analysis or action by assessing and combining their probability of occurrence and impact.
10.4 Perform Quantitative Risk Analysis—The process of numerically analyzing the effect of identified risks on overall project objectives.
10.5. Plan Risk Responses—The process of developing options and actions to enhance opportunities and to reduce threats to project objectives.
10.6 Control Risks—The process of implementing risk response plans, tracking identified risks, monitoring residual risks, identifying new risks, and evaluating risk process
Step 1 – Identify – Tailor a risk management strategy for the project and identify the risks including both threats and opportunities.
Step 2 – Assess - For each risk identified estimate the probability, impact and proximity. Evaluate the overall risk exposure of the project.
Step 3 – Plan - Plan the risk responses. Electing to Avoid, Share, Reduce, Accept, Fallback (contingent action) or Transfer risks(threats) and Share, Enhance, Exploit, or Reject Opportunities
Step 4 – Implement – Manage, control and report on the risks and their risk management progress.
Step 5 – Communicate – Continuously communicate the risks and their status to project stakeholders.
Common to both of these methods is a “scientific,” linear progression of risk management steps with the project manager as the main driver for the process. Both of these models incorporate a mathematical approach to prioritization and understanding of risk exposure on the project.
Some approaches have been developed specifically around software development. Mosaic, an approach developed under the Carnegie Mellon Software Engineering Institute's Mission Success for Complex Environments, is a systemic approach for assessing and managing success in distributed environments (Alberts & Dorofee, 2009). Systemic approaches assume a holistic view of risk to project outcomes by examining the aggregate effects of multiple conditions and potential events on a program's key objectives. Mosaic's systemic approach provides a good framework for identifying risk in large projects.
Agile has some built-in risk management approaches. One is Incremental Delivery. DeMarco and Lister (2003) demonstrated that increasing the size of the project has diseconomies of scale in software – and that smaller efforts therefore have the promise of more than proportional savings. Therefore, partitioning and learning are better approaches to address risk than striving to create a perfect plan. Agile wraps a cadence of planning and review ceremonies around incremental and iterative development. So the incremental and iterative delivery inherent in Agile provides a good basis for risk management.
Additionally, Agile incorporates a whole -team driven approach. The first value of the Agile Manifesto is “Individual and Interactions over Processes and Tools.” This is not to discount processes and tools. It is recognition that processes and tools are part of the environment and help people deliver successfully – they are not an end in itself. Agile recommends sufficient process and relies on a high level of cross-functional engagement and high levels of feedback throughout the project to identify and respond to threats and opportunities in real time.
In summary, PMBOK® Guide and PRINCE2 bring an intentional process to risk management and a mathematical approach to identifying and prioritizing risk exposure. SEI's Mosaic brings a systemic – mission outcome based view to Risk Management. Agile incorporates a cadence of incremental and iterative delivery and cross-functional engagement in responding to threats and opportunities. Each of these approaches contributes important aspects of effective Risk Management, but individually each is insufficient in practice.
Challenges with the Models
Risk is a factor of the level of uncertainty in a project. So by definition, higher-uncertainty projects have higher levels of risk. Many of the projects today fall onto the high-uncertainty part of this continuum. Additionally, the problems we are facing today deal with dynamic interconnected systems. Current Risk Management approaches are not sufficient to deal with these projects that operate with high uncertainty and dynamic, interconnected systems (Stevens, 2012).
Correct but not sufficient
When PMBOK® or PRINCE2 processes are implemented by the project manager or project management team they may result in poor engagement. Risk planning and identification may be done once upfront and not revisited. The processes are done independent of the overall organization and other planning processes and not integrated into the routine of the people delivering the work (Griffiths, 2012a).
We do not let the project manager estimate the project in isolation, so why would we let them manage the risks in isolation? Not only is relying on a single person to identify and analyze threats and opportunities inadequate, it also represents an unacceptable risk of its own. Confirmation bias can come into play when the Project Manager develops the risk plan and identifies the risks. Also, often there is a mismatch in personalities between the people best able to analyze risks and those best able to influence them. An intentional process is necessary – but, unless it is continuous and integrates cross-functional members that can influence responses, it is insufficient.
Mathematics isn't sufficient
Often, when risks are not estimated as independent events, we get the problem of compounding probabilities. For example, if six people each have an 80% chance of making it to dinner at 6:00 pm there is about a 25% chance that everyone will be there by 6:00 pm. Not an 80% chance as might be intuited. Additionally, when the risk exposure is padded into the effort estimates and not explicitly represented in our estimates, we aren't reflecting the risk impact in a meaningful way. We do not understand where we actually are on the project as the work is being performed. Analytical techniques are good for risk identification and prioritization but are not sufficient for risk resolution.
Tactical bottom-up approaches aren't sufficient
Many risk identification processes start with identifying risks from the project plan or from an established checklist. This assumes that the interrelationships and dependencies among conditions and risk events are well established. This approach to tactical risk analysis views a risk as a simple cause-and-effect pair. The number of risks in a complex project may become overwhelming. There may be multiple hundreds of risk statements and it can be time-consuming to aggregate the individual risk groups. At the end of the day local optimization may actually aggravate the risk impact on the project. Tactical bottom-up risk management may be useful in some contexts, but it is not sufficient to identify or respond to the risks.
Culture is an obstacle
When individuals are not engaged in the process, when risk impact is not explicitly reflected, and given the overwhelming and time-consuming efforts associated with managing a large risk list, it is not surprising that risk management is not viewed as helping teams maximize the chances of project success. This lack of perceived value contributes to an insidious challenge. DeMarco and Lister (2003) identify culture as a primary obstacle to risk management. Not only is the project team not engaged in the risk management process – project management often creates even greater organizational cultural obstacles. Common themes that project management imposes on project teams include: Don't be a negative thinker; don't bring a problem without a solution; don't say something is a problem unless you can prove it; don't be the spoiler – it is safer to power through, have someone else fail, and have plausible deniability; or, don't raise a problem unless you want to own it.
What is needed?
A Systemic risk perspective
A systemic approach for managing risk starts with the identification of a project's key objectives. Once the key objectives have been articulated, a set of drivers that influence the outcome are identified. A group of subject matter experts identify transition indicators. Mosaic provides a standard set of 20 systemic drivers to use to seed the discussion of risk conditions and transition indicators (Alberts & Dorofee, 2009). This framework of transition indicators is reviewed frequently during the project to assess whether conditions that are threats or opportunities are imminent.
Using probability and Monte Carlo Analysis, we can show the impact of risk on the project cost and schedule. Single point estimates do not communicate meaningful information. The risk exposure must be explicitly articulated in terms of potential schedule or cost impact. Understanding the amount of risk exposure on the project is more valuable than single point in time estimates with padding. With a spreadsheet, we can also do sensitivity analysis to identify risks drivers most likely to impact the project. Modeling is not expensive today and should be applied on projects of any significance or size (Stevens, 2012). Exhibit 6 shows two views of a project's percentage of completion: The first shows progress on identified and estimated work and the project appears on track, and the other is based on a risk-adjusted Monte Carlo analysis and shows that given current conditions the project is actually not likely to hit the due date. The difference between the two reflects the current risk exposure.
A risk management cadence
Boyd's Law of Iteration states that “In analyzing complexity, fast iteration almost always produces better results than in-depth analysis” (Richard, 2004). Agile has a cadence that supports progressive elaboration from Road-mapping, to release planning, to iteration planning, to iteration reviews, to release reviews. This cadence is implemented with a set of ceremonies. However, Agile doesn't prescribe a risk management approach that will scale effectively to support large complex projects. Incremental and iterative delivery, wrapped in a scaled-Agile cadence and supported by a systemic risk assessment model that has been mathematically assessed creates the right risk management opportunities with the right context.
Cross-functional engagement leveraging social influence
While project managers are key to project coordination and success, they are rarely the domain experts on modern projects and instead bring subject matter experts (SMEs) together to collaborate on novel solutions. These knowledge-worker projects require a whole-team approach to not only risk finding, but also to risk resolving. Having this cross-functional team systematically assessing risks on the planning cadence, reflecting on the results using mathematical analysis, and empowered to respond to the threats and opportunities creates conditions to maximize the success of the project.
We still need to gain effective engagement from these team members in risk planning, assessment, and response. This is achieved through incorporating visual collaborative games and visual reporting of project status and risk exposure into the risk process. Visual representation helps engage the left and right hemispheres of the brain. They allow us to tap into our spatial awareness and memory to avoid forgetting about risks (Griffiths, 2012a).
A 21st Century Risk Management Framework
We will show a four-step continuous process that is suitable for the complex, interdependent projects that we are undertaking today. It is built on: an ongoing systemic view of risk, risk drivers, and transition indicators; mathematical analysis of risk probability; cross-functional collaborative games; and social influence. The cross-functional collaborative games are adapted from those introduced in Mike Griffiths Collaborative Games for Risk Management (Griffiths, 2012).
In the risk assessment process, the subject matter experts define the appropriate risk drivers and transition indicators for this project. This framework will serve as the overall framework for managing risk over the course of the project.
The Risk Assessment involves two games – Plan Your Trip and Find Friends and Foes.
In Plan Your Trip subject matter experts tailor how risk management activities will be performed for this project. They are to ensure that the degree, type, and visibility of risk management are commensurate with both the risks and the importance of the project to the organization. If this team is new to risk management, then discuss the trade-off between risk and value, how the systemic approach provides context for ongoing risk management, what the cadence is for delivering the project, and how the cross-functional team will work together throughout the project across all four aspects of risk management.
The team then discusses various risk drivers, either drawing on the 20 from Mosaic or some other list. We have found that the following list (summarized from various sources) may create a sufficient context for most projects:
- Business: Do we have agreement on the business outcomes and the scope of the project?
- Technical: Do we have the technical skills to deliver this? Do we have legacy obstacles to overcome?
- Organizational: Do we have the staff to support this?
- Dependency: What dependencies exist on the project?
- Risk Response Approach: Does this team have the authority to make the tradeoffs necessary to respond to threats and leverage opportunities that arise?
The team members work independently to identify the success end state and the failure end state for each of these drivers. The goal is to gain clarity across the team around the project objectives and success modes and failure modes.
In Find Friends and Foes (risk and opportunity identification) the team evaluates the risk drivers to find indicators that the project is heading toward either the success state or the failure state. A useful collaborative game for identifying risk Indicators is the Dooms Day Clock. The risk drivers are placed around a circle, like a clock, and the team silently brainstorms on risk indicators associated with delivering this project. They may use historical information, check-lists, expert judgment or any other source for risk indicators. They then combine their findings during a collaborative discussion to ensure they have fully evaluated the project. You can also do the same with Karma Day to identify indicators that the project is moving toward the successful outcome.
The goal of this step is to gain agreement on how risk management will be conducted, build a census of systemic transition indicators that the team will pay attention to over the course of the project, and to gain a shared understanding on the cross-functional team on how to maximize success on the project.
In the Risk Analysis process, subject matter experts perform qualitative and quantitative analysis that is appropriate for this project. We will use a game called Post Your Ad for qualitative analysis and we recommend using a simple Monte Carlo simulation in Today's Forecast for the quantitative analysis.
In Post Your Ad, the team of subject matter experts takes the risk indicators identified and plays them on a board layout that incorporates impact on the x-axis and probability on the y-axis. Impact increases from 0-minimal impact at the center and increases to 5-dramatic impact at the right and the left. Benefits go to the left and costs to the right. Impact is based on the relative cost- and schedule- impact that would occur if the risk manifested. Probability goes from 0 at the bottom to 90%+ at the top. Place your Ad comes from the idea of trying to get “Investors” for the opportunities and “Help Wanted” for the threats. At this point, the goal is find the relative rating of the indicators – but also to gain a deep understanding around the risks.
Today's Forecast involves using quantitative analysis to determine a risk adjusted view of the current state of the project and a projection of when the project will be delivered. Exhibits 1 and 6 show examples of the output of Today's Forecast demonstrating the risk exposure on the project. Using a Monte Carlo analysis is the best way to determine this. Another benefit of using Monte Carlo analysis with the individual risk drivers estimated is that it provides a systemic sensitivity analysis on where to best focus risk-handling efforts.
The Risk Handling process is the team actually responding to threats and opportunities. The first thing to understand is which threats and opportunities have the biggest payback if mitigated. One common challenge is that when risk responses are locally optimized, that can actually have a negative impact on the chances for project success.
The Risk Report Card helps teams focus on systemic risk. Evaluate each of the risk drivers by balancing threats against opportunities using the results of the quantitative analysis. Then the team can identify where there are appropriate risk responses or opportunities to exploit. This can be presented visually as in Exhibit 7. The net of risk indicators shows value in addressing the Business, Technical and Risk Response driver categories. Technical appears to have the largest drag on the project in this case.
Inoculator involves reviewing the risk backlog as a whole team and identifying opportunities to make trade-offs or inject risk responses into the backlog to maximize chances for project success. The goal of Inoculator is to take specific actions to improve the forecast for the project. When we add responses to the backlog the estimated work will increase but the expected impact of the risks addressed should go down.
Another collaborative game is offering Risk Bounties and Opportunity Profit Shares for identifying actions that result in improved cost or schedule performance. For example, responding with a small investment to a threat that was showing a 10-day delay in the schedule would be rewarded with a sizeable Risk Bounty. Such recognition should be made public either by providing some benefit total or by awarding some sort of badge to the teams that identify the risk response. The objective is to get teams informed of risks, interested in understanding the impact of risk, and involved in responses that improve the likelihood that the project can be delivered.
The final ongoing process is Risk Monitoring. Risk Monitoring is the public presentation of the project status. There are a few common methods for performing Risk Monitoring. One is to plot features (or value) delivered in burn-up charts while plotting residual risk in a risk burn-down. Risk can be plotted against an ideal line to show status. In Exhibit 10, project B is in better shape than project A because the residual risk is below the trend line. In fact, project A looks to be on time but risk is getting pushed back into the project – so A actually has significant additional risks.
Another approach is to combine the two charts to a single burn-up showing Risk Adjusted Value (RAV). In Risk Adjusted Value, risk exposure has been converted to schedule or cost using some form of analysis (i.e., Expected Monetary Value, Monte Carlo) and the combined score of (value delivered less risk exposure) is plotted over time. Exhibit 11 shows a risk adjusted value chart where the product was finished with residual risk for the customer (such a high technical debit, fragile interfaces, or existence in a perilous marketplace).
A final common way to show project risk is Red-Yellow-Green status tracking. This traditionally tends to be a subjective assessment based on the project manager's belief that the project can be delivered heroically. Most organizations frown on projects that become Yellow or Red and so they find out late that the projects are in jeopardy. Instead of a subjective assessment, use Risk Adjust Value or a risk-adjusted Monte Carlo analysis of the values. If the analysis of work plus the risk exposure reveals that the project has a 90% chance of being delivered by the due date it is green. If it is 75-89% it is yellow. Below 75% is should be red. This model has the added benefit of getting teams to focus on burning risk down earlier in the project.
In complex projects with dynamic interdependent systems, risk management is at least as critical to the successful delivery of projects as work estimating. Existing models like the PMBOK® Guide, PRINCE2, Mosaic, and Agile have valuable components but are individually insufficient.
A 21st Century Approach to Risk Management builds on and integrates aspects from these existing risk approaches. It leverages an intentional process that is tailored to the project by the project team. Rather than a tactical bottom-up approach, it takes a systemic view of identifying and handing risks. Risk exposure is not blended into the schedule or padded into estimates. Risk is explicitly represented and project managers take a probabilistic and economic approach to understanding and communicating risk exposure. The process is continuously performed, tying in to the other planning and review efforts in the project. Finally, it uses a cross-functional team approach and social influence to engage the whole project team in helping maximize the chances for project success.
Alberts, C. J., & Dorofee A. J. (2009). A framework for categorizing key drivers of risk. Pittsburgh, PA: Carnegie Mellon University.
DeMarco, T., & Lister, T. (2003). Waltzing with bears, managing risk on software projects. New York, NY: Dorset House Publishing.
Griffiths, M. (2012). Collaborative games for risk management. Calgary, Canada: Leading Answers.
Magennis, T. (2011). Forecasting and simulating software development projects, effective modeling of Kanban and scrum projects using monte-carlo simulation. Seattle, WA: FocusedObjective.com.
Richard, C. (2004). Certain to win: The strategy of John Boyd, applied to business. Atlanta, GA: Xlibris, Corp.
Stevens, D. (2012). Agile and the nature of decision making. Retrieved from http://www.slideshare.net/dennisstevens/agile-and-the-nature-of-decision-making.
Project Management Institute. (2004). Software extension to the PMBOK® guide fifth edition. Newtown Square, PA: Project Management Institute, Inc.
© 2013, Mike Griffiths, Dennis Stevens
Originally published as part of 2013 PMI North American Congress Proceedings – Louisiana, USA