Collaborative games for agile risk management
This paper describes a set of collaborative team games used to engage the whole team in opportunity and risk management activities. It outlines an approach that overcomes many of the “correct-but-not-sufficient” aspects of risk management seen too often on projects:
Poor engagement — dry, boring, academic, done by project manager, does not drive enough change
Done once — typically near the start, when we know the least about the project
Not revisited enough — often “parked” off to one side and not reviewed again
Not integrated into the project life cycle — poor tools for task integration
Not engaging, poor visibility – few stakeholders regularly review the project risks
The paper starts with an explanation of the risk management strategies implicit in the agile life cycle, then describes a series of team games to explicitly tackle risk and opportunity management.
The Agile Advantage toward Risk Management
Agile methods incorporate many mechanisms for dealing with late breaking changes (an easily reprioritized backlog, short iterations, frequent inspection, and re-planning, etc) that also lend themselves to proactively responding to risks. We can insert risk avoidance and risk reduction actions into the backlog to proactively attack the risks before they have an impact on the project.
First of all, why play collaborative team games? Just as techniques like Planning Poker and Iteration Planning effectively make estimation and scheduling a team activity and gain the technical insights of engaging people closer to the work. So, too, do collaborative games for risk management; after all, why leave risk management to the person who is furthest from the technical work — the project manager?
Most software projects resemble problem-solving exercises more than plan execution exercises. It is very difficult to separate out the experimentation and risk mitigation from the pure execution. Team members are actively engaged in risk management every day. We can benefit from their input in the risk management process and if they are more aware of the project risks (by being engaged in determining them), how they approach their work can be more risk aware and successful.
The Benefits of a Collaborative Approach
The benefits of collaboration are widely acknowledged, and a study by Steven Yaffee from the University of Michigan cites the following benefits:
- Generates wiser decisions through the understanding of complex, cross-boundary problems via shared information
- Promotes problem solving rather than procedural decision making
- Fosters action by mobilizing shared resources to get work done
- Builds social capital by building relationships and understanding
- Fosters ownership of collective problems by valuing participation and shifting power downward
There are some powerful concepts here that are worth a second look. It is pretty obvious that engaging a larger group of stakeholders will produce a better list of possible risks and then yield more creative ways of avoiding or reducing those risks. However, the real benefits of engaging the team come from the changes that happen in the team.
By engaging the team not only do we get better input data and ideas, but we also encourage problem solving, foster action, build social capital and foster collective ownership of ideas. No longer do we have a single project manager worried about the risks, we now have a motivated, energized and empowered team proactively managing the risks.
Far too often, projects do a great job at identifying possible risks and a lousy job doing anything about them. The result is projects that get derailed when a known risk becomes an issue. When the team is fully plugged into the project risks, small changes in their behavior eliminate many risks at their source, long before they get large enough to threaten the project.
Finally, the last point in Yaffee's benefits of collaboration list is noteworthy too. Valuing participation and shifting power downward fits extremely well with the empowered teams and servant leadership model promoted by agile methods. We already encourage these ideas in reporting progress via daily stand-ups, estimating via planning poker, and decision making via fist of five techniques, so why not in risk management?
Of course, collaboration is no silver bullet. Brainstorming, for example, can actually stifle innovation and lead to groupthink. The New Yorker Magazine describes studies that show how brainstorming groups think of fewer, lower quality, ideas than the same number of people who work alone and later pool their ideas. So, let's be clear, collaboration, does not only mean brainstorming, it also includes pooling individual ideas and group validation.
A Risk Management Framework
A Guide to the Project Management Body of Knowledge (PMBOK® Guide) — Fifth Edition Exposure Draft describes a six-step process for risk management:
- Plan Risk Management
- Identify Risks
- Perform Qualitative Risk Analysis
- Perform Quantitative Risk Analysis
- Plan Risk Responses
- Control Risks
Through collaborative games, each of these six risk management steps can be recreated as highly visual, team-based activities that then create risk avoidance and risk mitigation stories for the product backlog.
We want visual collaborative games because visual representation helps engage the left and right hemispheres of the brain. They allow us to tap into our spatial awareness and memory to avoid forgetting about risks. This is the reason why today's military still uses visual tokens to represented enemy forces, despite having access to the world's most sophisticated tools. The impacts of forgetting about them can be fatal. The same goes for project risks.
The collaborative games that cover these steps are:
- 1) Plan Your Trip – (Plan Risk Management)
- 4Cs — Consider the Costs, Consequences, Context, and Choices
- Are we buying Coffee, a Couch, a Car, or a Condo? How much rigor is appropriate?
- Deposits and Bank Fees — understanding features and risks
- 2) Find Friends and Foes — (Risk and Opportunity Identification)
- Doomsday clock
- Other risk identification forms (risk profiles, project risk lists, retrospectives, user story analysis)
- 3) Post Your Ad — (Qualitative Risk Analysis)
- Investors and Help Wanted — classification and visualization of opportunities and risks
- Tug of War — project categorization
- 4) Today's Forecast — (Quantitative Risk Analysis)
- Dragons' Den — next best dollar spent
- Battle Bots — simulations
- 5) Backlog Injector — (Plan Risk Responses)
- Junction Function — choose the risk response path
- Dollar Balance — Risk/Opportunity EVM to ROI comparison
- Report Card — Customer/Product owner engagement
- Inoculator — inject risk avoidance/mitigation and opportunity stories into backlog
- 6) Risk Radar – (Monitoring and Control Risks)
- Risk Burn Down Graphs — Tracking and monitoring
- Risk Retrospectives — Evaluating the effectiveness of the risk management plan
- Rinse and Repeat — Updating risk management artifacts, revisiting process
1. Plan Your Trip — (Plan Risk Management)
This phase is about deciding and defining how to conduct risk management activities for the project. We want to tailor the process to ensure that the degree, type, and visibility of risk management are commensurate with both the risks and the importance of the project to the organization. The other goal we have for this phase is to teach some risk basics to the team since they may not be familiar with the concepts or terminology.
The name of the first exercise “Plan your trip” speaks to the goal of determining the appropriate level of rigor. Most people can associate with planning for a walk or hike and this is the context we use for the activity called the 4Cs. Early in any collaborative workshop, I like to get people working. If you let them spectate for too long, some will retreat into “observer” rather than “participator” mode.
Working individually (again to encourage active engagement and avoid groupthink) ask the team to consider what they would pack for a two-mile hike in the country on a warm day. Give them a couple of minutes to create lists on Post-it notes and review their responses as a group. Some will suggest taking nothing, or just a bottle of water; others will suggest rain jackets, bear spray (I live near the Rocky Mountains in Canada) and all sorts of other things. We then review the pros and cons of these items. They are useful if you need them, but a burden to carry. We then repeat the exercise, changing some parameters such as making it a ten-mile hike, or multi-day trip, in the mountains, in the winter time. Now the lists get longer as people prepare for more eventualities.
For each situation we review the 4Cs—the Costs, Consequences, Context and Choices. What we bring (and how we prepare for risk management) varies based on the Cost of bringing/using it, the Consequence of not having it (rain coat — get wet, warm jacket — cold/hypothermia). We also examine the Context we are talking about, preparations for elite ultra-marathoners who are hardy, capable, and resourceful or a kids' group who need more protection. Finally, the Choices we make should be an informed balance of Cost versus Consequence in the frame of the Context.
Another tool to relate the need to tailor the process appropriately is to ask the team to consider the decision rigor they put into their purchases. They way we consider buying a Coffee (US$2), a Couch (US$2,000), a Car (US$20,000), or a Condo (US$200,000) vary as the figures involved escalate.
For a coffee, we probably just find something close, maybe at our favorite coffee-brand store. For a couch, people will shop around and likely buy the one they like the best without much further research. When it gets up to car territory, safety, economy and resale factors are routinely examined. For a condo purchase the stakes are so high that most people engage professional help from home inspectors and condo document review companies. We need to do the same for our projects, asking what is appropriate for the endeavor.
Finally, if the team is new to risk management, then a discussion on trade off between business value and risks might be necessary. We undertake projects usually for the potential upside (or for compliance projects to avoid the downside). Getting business value out of a project is like receiving deposits into our bank account; we want them as often as possible, and preferably as large as possible. Given the uncertainty in the world, we want the biggest gains as soon as possible, before anything changes that may threaten future deposits.
In this bank analogy risks are like withdrawals; or bank fees, should they occur, set the project back, take away resources from delivering business value, and threaten the delivery of future value. So, to get the most out of a project we need to maximize business value while avoiding or reducing risks.
These exercises and discussions aim to get the team thinking about the appropriate level of risk management for the project and gain consensus and support for the strategy that is agreed upon. Without this shared understanding of “why?” we will not get people invested in the process.
2. Find Friends and Foes — (Risk and Opportunity Identification)
The next step in the process is to identify potential risks and opportunities. Opportunities are the “good” risks or fortuitous events that have a positive impact should they occur. We want to avoid risks and exploit opportunities.
Using a clock view pre-drawn on a white board or flip chart we ask team members to think of project risks associated with each of the topics represented by an hour line on the clock — 12 in total.
This is the doomsday part, we encourage the team members to think of and record as many risks as they can about that topic. We work topic by topic, but if thinking of risks triggers ideas in other areas as we progress, it is not unusual to get risks being added to previously discussed risk lines. Again, I prefer having people working individually for coming up with ideas. Then, we put them all on the wall and consolidate and remove duplicates as a group, which also sometimes identifies new risks.
Discourage people's tendencies to want to score, rank, and solve the risks. This is risk identification, which we will have plenty of time to process later.
Exhibit 1 – Doomsday Clock Risks.
A wall filled with risk sticky notes around every aspect of the project can seem like a discouraging prospect for some, but it is also a useful eye opener for why risk management is so important. This project is not magically going to work out all by itself. We have some real obstacles in front of us and we need to work as a team to overcome them.
Always within the same session, I like to run the flip side exercise – “Karma Day.” In this exercise, we generate opportunities for events and outcomes that would assist the project. Generating ideas for things that would help the project go well. Using the same clock metaphor we come up with lists of all the good things that could occur to assist the project.
Cynical team members may still continue to gripe, suggesting opportunities, “inverted issues” such as “Actually getting a 1- or 2-day turnaround on our database requests, for a change” or “support not resistance from the PMO,” but these can be really useful. Just as we later ask in risk management: “How do we avoid or reduce these risks” in opportunity management, we ask: “How do we ensure or maximize these opportunities?” If spending a couple of hours explaining the project goals and approach to supporting groups, or proactively asking them how we might best engage with them makes a difference, then this work could have a huge return on the time invested.
Without asking the team for a list of all the good things that can happen, team leads and project managers will likely be unaware of all the ways in which they could serve and support the team. The Scrum master as an “obstacle remover” is a one-sided, glass-half-empty view of the world. Why not explicitly add “opportunity implementer” to the job description and let's see if we can arrange some mutual wins by being proactive.
Exhibit 2 – Karma Day Opportunities.
These techniques are facilitated games for identifying risks and opportunities, but they do not stop us from applying more conventional approaches too, such as risk profiles, project risk lists, SWOT analysis, retrospective findings, user story analysis, and so forth. Let's not throw out the baby with the bath water; we should still use traditional approaches, but augment them with team-based approaches for better insights and buy-in.
3. Post Your Ad — (Qualitative Risk Analysis)
Having found risks and opportunities, we now need to classify and rank them. The duplicates found in Doomsday Clock and Karma Day can be removed, and related risk and opportunity ideas might be better consolidated under new headings (provided they truly are the same risks). Then we need to categorize and prioritize them.
The traditional way of doing this is to assign numerical Probability and Impact scores using something like the matrix shown below. While mathematically valid, many people find it counter-intuitive that the highest ranked risks are grouped right next to the highest rank opportunities because they represent opposite extremes of bad and good outcomes for the project.
Exhibit 3 – Traditional Risk and Opportunity Matrix.
Another issue with assigning estimated probability and impact values is that people are much better at relative estimation than absolute estimation. We get hung up on whether something has a 0.7 or 0.9 chance of happening, but could tell you that it is more likely that the lack of C# experience will bite us before a lack of MSBuild experience will. This is one reason why many agile teams use effort estimation based on points rather than ideal days, and we give directions in reference to landmarks rather than pure distances. People are better at relative estimation than absolute estimation.
So, recognizing these human traits, I prefer to get teams to gauge impacts and probabilities in a more intuitive and relative way. For this we use “Investors and Help Wanted” board concepts. Risks are things we need help with, opportunities and things we are seeking investors and supporters for. Using scales of impact to the project in terms of Impact (costs for risks) and (benefits for opportunities) on the X axis and Probability of occurrence on the Y axis the risks and opportunities can be visualized by the team as shown in Exhibit 4.
Exhibit 4 – Investors and Help Wanted Board.
In this layout, the X axis of Impact to the project in terms of benefits and costs, creates a spread where the greatest opportunities are furthest removed from the greatest risks. We do not worry too much about numeric analysis of probabilities, but instead use relative ranking to determine the vertical positions. The benefits at this stage really come from the conversations among team members about analyzing the risks and opportunities and gaining consensus as to where they belong relative to each other on the charts.
Visualizing and agreeing on a spatial reference for the risks and opportunities also engage the right hemisphere of the brain and makes us less likely to forget them. (Assigning things we want to remember to a location is a memory aid that many memory-improvement techniques use. Probably relating back to our hunter–gatherer days when our survival relied upon remembering where to find food and water, we have better recall of things assigned a physical location.) This is useful for us as the project progresses, because if we better remember the risks and opportunities at play, we are more likely to tailor our behavior and everyday decisions toward them appropriately.
Finding and categorizing risks is a start, but not sufficient. The real value comes from converting them into actionable stories for the prioritized backlog. Then tracking and adapting based on review and reflection of the systems effectiveness.
4. Today's Forecast — Quantitative Risk Analysis
The “Quantitative Risk Analysis” process attempts to quantify (put some numbers against) the risks under consideration. It helps understand the magnitude of individual risks and then, viewed together, the overall project risk profile. Before quantifying risks, let's talk about the dangers of applying math to estimates and speculations.
As Albert Einstein said, “Not everything that can be counted counts and not everything that counts can be counted.” In other words, some of the things we can quantify are not that useful, and some of the things we would like to quantify cannot easily be measured. Also, some research claims that quantitative risk approaches divert attention from precautionary or preventive measures.
However, as long as we are aware that trying to quantify risks may be problematic, we can still gain some valuable insights into their likely importance that can help us with prioritization. The usual way of quantifying risks is to express the impact of the risk occurring in monetary terms and the probability of it occurring as a percentage. We can then calculate the Expected Monetary Value of our risks.
For example, in a software project we may have a risk that our in-house reporting engine may not be up to the performance needs of the project. If the cost of swapping it out was US$80,000 and we thought it 50% likely that we would need a replacement reporting engine, then we could calculate the Expected Monetary Value of the risk as:
Expected Monetary Value = Impact (US$80,000) x Probability (50%) = US$40,000
Calculating the Expected Monetary Values of our risks then allows us to prioritize them. The general idea is that, much like an agile project's prioritized backlog of features, we want to tackle them (find ways to avoid the risk or make it smaller) in that order to minimize risks to the project as soon as possible.
The team games we use in this step are more to socialize the risk management concepts with the team to better illustrate why backlog items may shift, than taking action. That comes in the next step (risk response planning), which is all about deciding what to do about the risks and putting work in our project backlog of things to do. For now, we are just concerned with quantifying the risks.
Agile has the concept of the “next-best dollar spent” that reminds us to always be looking where we can add the most value next. This may be in developing a new feature from the backlog that will generate an increment of Return on Investment (ROI) once it is released to the business; or it may be to invest some time in avoiding or reducing a risk that threatens to negatively impact the project through expensive rework, delays, or additional costs.
The popular TV show “Dragons’ Den” in which entrepreneurs pitch investment proposals to investors, can be a useful metaphor for modeling the next-best dollar spent. By comparing features from the backlog with risks to mitigate, we can get the team more comfortable with the seemingly shifting priorities that may come from the product owner or business representative when they consider the next-best dollar spent and prioritize the backlog accordingly.
5. Backlog Injector — Plan Risk Responses
- Junction Function — choose the risk response path
- Dollar Balance — Risk/Opportunity EVM to ROI comparison
- Report Card — Customer/Product owner engagement
- Inoculator — inject risk avoidance/mitigation and opportunity stories into backlog
The Risk Response Planning step is where we decide what to do about the risks we have identified, ranked, and measured. The options generally available to us are:
- Avoidance – eliminate the cause of the risk
- Mitigation – reduce probability of the occurrence
- Transference – insurance, outsource, etc.
- Acceptance – accept and communicate to stakeholders
A fifth “Denial” option is a widely practiced risk response approach, but is not a valid option. Generally, it is best to avoid risks by finding ways to eliminate the root cause of the risk. Failing that, make them smaller or pass them to a party who is better able to handle them (for instance, outsource that work to a specialist in that field). Finally, the least preferable option is to accept the risk. For example, perhaps we just need to wait for service pack 2 from the vendor to fix the issue and until then we accept the risk of performance slowdowns.
Exhibit 5 – Risk Response Options.
We need to explain these options to the team and make sure they understand the order of preference and how the risk response options impact the residual risk to the project.
Residual risk is the remaining risk to the project even after we have taken the best risk response option we are able to. Perhaps in our reporting performance example, we chose to run the reporting engine on a high-performance server. This helps somewhat, but we still have the residual risk that the higher spec machine is not sufficient.
Secondary risks are new risks occurring as a result of our risk-response strategy. Maybe we decided to do the report processing on the company's new scalable cloud platform. This may sound like a good idea, but if the company cloud platform is new and untested, then maybe this secondary risk is more significant that the original one we were trying to avoid or reduce. So we need to quantify secondary risks to ensure they are indeed smaller than the risk we are responding to.
Once the team is familiar with the concepts of risk-response options, residual risks, and secondary risks, we can engage them in the initial step of putting action items in the backlog. Backlog prioritization is a business/product owner role but they will need help determining the priority of risk response actions; this is where steps to normalize the risks and feature values are required.
Not all risks can be avoided or reduced and we just reviewed how some risks might have to be accepted. When accepting a risk, there is no backlog action to create or balance against new functionality. However, if we can avoid a US$50,000 EMV risk, then this risk avoidance is worth US$50,000 to the project and should be inserted in the backlog above features worth US$49,000 to the business.
We need to compare the value of risk response actions to the value of prioritized features and insert them in the appropriate place. When doing so, we also have to be aware of residual and secondary risks. So, the value of a risk response action = Net Monetary Value (EMV) = EMV of Residual Risk + EMV of Secondary Risk.
For example, simply avoiding a risk with an EMV of US$40,000 that has no residual risks or secondary risks is worth US$40,000. Yet reducing the impact of a US$60,000 risk by trying an alternative approach that only addresses half of the problem and in itself carries a US$10,000 EMV is only worth US$60,000/2 = US$30,000 + US$10,000 = US$40,000.
So, we need to normalize all the risk responses to values to take into account residual and secondary risks before asking product owners to prioritize these actions in the backlog.
The Report Card is a list of recommended risk responses, normalized by net EMV, for consideration by the business representative/product owner. It is prioritized by Net EMV.
Exhibit 6 – Normalized Risk Values.
With this information, the product owner can now discuss adding these risk response actions into the backlog.
“Inoculator” is the name given to the process of injecting risk avoidance/mitigation and opportunity stories into the backlog. It is done by the product owner, but with consultation and guidance of the development team. It is this ability and frequent opportunity (every iteration planning meeting) to be able to reduce remaining risks and capitalize on opportunities that set agile methods apart from other, slower review cadence approaches that inspect and adapt less frequently.
By avoiding and reducing risks closer to their identification, the horizon of risk that the project is exposed to shortens. By making changes earlier in the life cycle, the costs of changes are reduced. On the flip side, capitalizing on opportunities is like getting investments done early; they have longer to accumulate. These are the compounding benefits of early and rapid risk and opportunity management.
Getting the risk response actions into the backlog is how these tasks are scheduled and undertaken. We want to make sure that all our risk management work is not supplemental to the project plan, but baked right in. All too often, risk management is an activity done upfront or alongside the project, but never really integrated into the day to day activities of the project. By inserting these new stories into the backlog, we drive risk management actions from the analysis to action.
Exhibit 7 – Inserting Risk Responses into the Backlog.
6. Risk Radar — Monitoring and Controlling Risks
The last step in the process is monitoring and controlling the risk management process by making sure our strategies are effective, continually looking for new or escalating risks and ways to improve.
Risk Burn-Down Graphs
Risk burn-down graphs are a great way of showing the project's cumulative risk position and trends over time. They are stacked area graphs of risk severity that allow trends, along with new and escalating risks to be easily identified.
Exhibit 8 – Risk Profile Graph.
Risk retrospectives are periodic reviews of the risk and opportunity log and risk management processes being used on the project. Just as we review the evolving product and team processes throughout the project, so should we be evaluating the effectiveness of the risk management plan and processes being used by the team.
The types of questions we could/should be asking when we regularly review our risk management approach include:
- Are we eliminating or reducing our risks?
- How is our remaining Risk EMV burning down?
- What is our Risk EMV Reduction Velocity per iteration?
- When will our remaining Risk EMV be zero?
- Do we have any new or escalating risks?
- What are the root causes of our risks and can we eliminate any of them?
- Which risk avoidance or elimination strategies are working and which are not?
- For risks that we chose to transfer, how are the third parties managing them? What can we learn from them, or would we be better bringing them back internally?
- How are our team risk management capabilities developing?
- Where do we still need mentoring and support?
Rinse and Repeat
Finally, reviewing is not enough — we need to update our risk management artifacts, update our risk lists and EMV scores, and groom the backlog with new features and new risk responses; and always rebalancing the priorities. Update the risk information radiator graphs (like our risk burn-down graphs), and make sure people are not only looking at the impacts of new work in terms of estimates, but potential risks, too.
Risk management, like estimation, should not be just a project management activity. We can greatly raise a project team's ability to manage risk and therefore avoid project failures through socialization, collaboration, and practice. If nothing else, these team activities make the basics of risk management more accessible to a larger pool of project stakeholders, and in doing so provide more eyes to find and avoid risks before they can impact the project—which, at the end of the day, is the heart of effective risk management.
Commoner, B. (1987). Contemporary moral controversies in technology: Comparing apples to oranges: Risk of cost/benefit analysis,” pp 64–65. United States: Oxford University Press.
Lehrer, J. (2012). Annals of ideas: Groupthink: the Brainstorming Myth Retrieved from http://www.newyorker.com/reporting/2012/01/30/120130fa_fact_lehrer
© 2012 Mike Griffiths
Published as part of the 2012 PMI Global Congress – Vancouver, Canada