Uncover gaps in your requirements using requirements risk management techniques
Cheryl Lee, CBAP, PMI-PBA, PMP
Ori Schibi, MBA
Cheryl Lee, President, BA Konnectors
Ori Schibi, President, PM Konnectors
We are expected to ask the right questions when eliciting requirements to identify gaps. Requirements risk management is an emerging concept in many high performing organizations used to provide guidance on what the right questions are. The concepts covered in this paper include:
- Understanding the concept of requirements risk management and the difference between project risk and requirements risk.
- Appreciating and leveraging the benefits of conducting a thorough requirements risk management process.
- Knowing how to apply the concepts of requirements risk management to identify gaps in requirements.
- Identifying opportunities to collaborate with the project manager to ensure requirements risks that are also project risks are communicated and managed in a timely manner.
This paper will help readers explore the handoffs between project and requirements risk and arm attendees with information to begin addressing the #2 reason why projects fail—inaccurate requirements, immediately within their workplace—using requirements risk management techniques.
What Is Requirements Risk Management?
Inaccurate requirements gathering has been cited as the #2 reason why projects fail according to a 2014 PMI Pulse of the Profession® report—second only to changing organizational priorities (PMI, 2014). Requirements risk management is an emerging concept in many high performing organizations used to identify gaps in requirements, enable testing and development teams to prioritize their work more effectively, and facilitate the transition of product requirements into the operational environment and thus reducing the probability of project failure.
Project risk management, which is primarily known to be the accountability of the project manager (PM), is a familiar concept but many are foreign to requirements risk management, which is the accountability of the business analyst (BA) to complete, as product requirements are being elicited and defined. Project risks are uncertainties that may impact the project where as requirements risks are uncertainties that may impact the product or solution. However, project risk and requirements risk are very closely linked to each other and the PM and the BA needs to work together to ensure both project risks and requirements risks are being identified and managed accordingly.
Examples of Project and Requirements Risk
To further clarify the differences between project and requirements risk, let us articulate a few examples using the risk statement: “If condition occurs, then consequence could happen.” The condition is the risk event and the consequence is the impact the risk event will have on the project or product. When we think about a project's focus, what usually come to mind are the triple constraints: time, cost, and scope, specifically project scope (see Exhibit 1 – The triple constraints of a project).
Exhibit 1 – The triple constraints of a project.
When articulating the risk statement, if the consequence or the “then” portion of the risk statement is to one or more of the triple constraints, then it is a project risk; where if the consequence is to the product or solution, then it is a requirements risk.
Examples of Project Risk Statements
- If the business subject matter expert is tied up on a higher priority project, then we will have to reschedule the requirements workshop.
- If we cannot identify an expert on the existing application, then we will need more time to understand the current state of the application.
- If compliance/risk is not engaged prior to requirements sign off, then it may lead to rework if changes are identified post requirements sign off.
- If my current project is not completed by next week, then I cannot begin working on this new project.
- If we are unable to schedule a requirements walkthrough for next week, then the approval process will be delayed.
Examples of Requirements Risk Statements
- If there are more than 300 applications processed each month, then the system may crash.
- If the reports are not available by 9am, then teams cannot begin their work.
- If user IDs are not maintained, then unauthorized users may access the system.
- If the account number is not found in the system, then we will not be able to process the application.
- If the upstream system passes system ABC incorrect data, then nurses will be using incorrect reports to triage patients.
How Do We Conduct Requirements Risk Management?
Many BAs are likely informally conducting requirements risk management when they are eliciting requirements. As a BA, our primary focus is to ask the right questions to ensure there are no missed requirements. Some of those right questions include playing devil's advocate and asking the what if scenarios. For example, we may not be close to the system limitation of 300 applications processed each month, but what happens if and when we are a year from now? This next section will walk through formalizing the requirements risk management process.
Requirements Risk Identification
The BAs at the limited number of organizations that do perform requirements risk management struggle with the time it takes to conduct the analysis and assessment. This is often because the requirements risk assessment is left until right before we are ready to obtain approval of requirements and are treated like an afterthought or something that provides no value other than to tick a box on the project documentation checklist to pass audit. It is much more time consuming and less valuable to do this than to build it into the elicitation process. It is known that most BAs are already having these discussions with their stakeholders as they are eliciting requirements, so why not tackle them at the same time rather than have to discuss them again later? Formalizing the process will put more structure around the discussions to decrease the chances of missed requirements.
When identifying requirements risk, the BA is looking for potential exposures or areas where there could be exceptions to the process, or failure points. The same way testers look to break the system or product, BAs are looking to break the requirements and identify fixes for them before it happens in the real world. Examples include:
- Have all paths through a process been identified?
- Have error conditions been identified?
- Do we know who updated a transaction and when?
- Can we easily correct incorrect transactions?
- Is there a potential for unauthorized users to access the application?
The following are some suggestions on how to elicit for requirements risks:
A requirements risk assessment from a comparable project can be used to identify potential exposures. Reviewing past lessons learned can also be a valuable source of potential requirements and project risks. If it was a problem before, then it could be a potential problem this time.
If there are multiple handoffs between teams or systems, each handoff is a potential exposure. If the process is handed off to a third party vendor, what happens if the communication link is hacked? Is there a potential for something to be missed between the handoffs from one team to another?
Although the BA is accountable to ensure the requirements risk management process is being completed, it is not their sole responsibility to identify all the potential exposures in the product or solution. Similarly, although it is the PM's accountability to ensure a project risk assessment is being conducted; it is not the PM's sole responsibility to identify all the project risks in a project. As BAs are interviewing or conducting requirements workshops with stakeholders, BAs should be identifying possible failure points and documenting these exposures in a requirements risk log to walk through the process of ensuring they will be addressed either through the project or through a business workaround, should it happen in the production environment.
Non-Functional Requirements Analysis
A fundamental item to remember when documenting requirements risk is, a requirement risk is not simply what happens if we do not implement the requirement. If it was listed as a requirement it is a need; therefore, there is no need to imagine what would happen if we do not do it. The only exception to this is when performing non-functional requirements analysis. If we have a non-functional requirement that says “The system shall be available 7:00 a.m. to 10:00 p.m. (EST), seven days a week”, what happens if the system is not available during that time frame? The rationale for this analysis is to ensure there is a business workaround or a process to get the system up and running again if there happens to be an unexpected outage. Naturally, if it is known that your organization already has procedures in place for this, it is not a potential risk and therefore should not be logged in your requirements risk log.
Similar to conducting interface analysis, when you are analyzing diagrams or models, you can look for handoffs between teams, as each handoff is a potential for error.
Observing processes can sometimes help identify any current exposures that may need to be fixed. For instance, while observing sales procedures in a call center, a customer service rep writes down a credit card number on a scrap piece of paper to facilitate processing of the payment. The customer service rep then leaves the scrap paper on the counter while going on a quick coffee break. This is an exposure to the organization, as someone else can potentially obtain the credit card number off the scrap paper.
A checklist of possible exposure categories can be developed within your organization and used to help brainstorm any potential requirement risks.
Assumptions, Constraints, and Dependencies Analysis
Similar to having project level and requirement level risks, there are also project level and requirement level assumptions, constraints, and dependencies. Each of these assumptions, constraints, and dependencies can also be converted into corresponding project and/or requirements risks. A project level assumption, constraint, or dependency is typically converted to a project level risk; a requirement level assumption, constraint, or dependency is typically converted to a requirement level risk.
Assumptions are factors that we believe to be true but yet to be confirmed. The risk is what would happen if they turn out to be false. Constraints are limitations or restrictions on the project or product. If we are approaching the limitation, then we should have a corresponding project or requirements risk; however, if we are at the limitation then we actually have an issue, not a risk. Issues and risks are often confused. Recall that risks are uncertainties that may impact the project or product. Issues on the other hand can be considered things that have happened, or things that make you go “uh oh, now what?” This is one of many reasons why framing all risks using the risk statement that was introduced earlier should be used as a best practice, because if something happened, it cannot be articulated with an “if” in front of it. Lastly, dependencies are linkages between entities. There are potential risks for linkages that are not adhered to. Exhibit 2 shows examples of assumptions, constraints, and dependencies at the project and requirement level and their corresponding risk events.
Exhibit 2 – Project level and requirement level assumptions, constraints, dependencies and their corresponding risk events.
Requirements Risk Assessment
Once a requirements risk has been identified, it should be documented in a requirements risk log to work through the remainder of the analysis. Sample headers in a requirements risk log can be found in Exhibit 3. The remainder of this section will walk through the process of completing a requirements risk assessment.
Exhibit 3 – Sample headers in a requirements risk log (Schibi et al., 2015).
Scoring Impacting and Probability
Similar to performing qualitative risk analysis for project risk management, requirements risks should be evaluated for the impact the risk would have to the organization if the risk were to occur and the probability of it occurring. Each organization should have their own scoring system and definitions for High, Medium, Low impact and probability. Unfortunately, some organizations specify, as part of their risk management procedures, that risk impact probably should be evaluated using a rating of high, medium, and low, but do not provide any guidance as to what high, medium, or low means, leaving it up to project team members to arbitrarily assign values. The impact and probability definitions and scores described in Exhibit 4 can be used as a guideline to evaluate requirements risks.
Exhibit 4 – Guidelines for defining requirement risk impact and probability ratings (Schibi et al., 2015).
The product of the impact and probability ratings provides an overall risk score. Once again, many organizations lack guidance on what to do if a risk item has a score of 25 versus a score of 1. The following can be used as a guideline:
- High Risk (score of 15–25): Critical to address these types of risk. Recommended response - define both a control and risk strategy
- Moderate Risk (score of 5–9): Cost/benefit analysis will need to be conducted to determine whether the action plan is appropriate for the threat level. Recommended response – define either a control or a risk strategy
- Low Risk (score of 1–3): Monitor to ensure threat levels do not change. Recommended response—define a risk strategy
A risk trigger indicates that a risk has occurred or is about to occur. It can be thought of as something that gets your “spidey senses” tingling (see Exhibit 5). Using the example “If there are more than 300 applications processed each month, then the system may crash,” the trigger can be any number of applications less than or equal to 300. Ideally, we would want to identify a trigger that would provide us with sufficient lead time to execute our contingency or backup plan. We may select a trigger of 250 applications processed within a month to trigger the action plan to begin initiating a new project to purchase and install a new server to handle the additional volume, assuming we can complete the purchase and installation of a new server before we hit 300 applications per month.
Exhibit 5 - Spider-Man's Spidey senses tingle when a risk trigger has been detected (Schibi et al., 2015).
Requirements Risk Responses
The requirements risk responses are different from the project risk responses. Where as the responses to project risks are avoid, mitigate, transfer, and accept, the responses to requirements risks are to introduce a control or a risk strategy.
A control is something that is put permanently in place to address the requirements risk. Permanent meaning the process, or whatever is implemented to address the requirements risk, exists beyond the lifetime of the project. This typically means additional product scope and product requirements need to be introduced and implemented; hence, identifying gaps in requirements. Following through with the same example “If there are more than 300 applications processed each month, then the system may crash,” a potential control is to implement an alert to notify production support when the application volumes reach 250 applications per month. This alert will require additional requirements to be defined and tested accordingly. If the requirement risk assessment was not conducted, we may have missed the fact that application volumes would need to be monitored beyond the project altogether, causing a major system crash when the additional volumes are realized. Subsequently, this also means additional project scope for the PM to manage.
A risk strategy on the other hand is something that is put temporarily in place to address the requirements risk. This is additional project scope or project requirements that are completed to address the requirements risk. Using the monthly applications example, a potential risk strategy could be to perform load testing to determine the potential impacts around the 300 application level to determine the true impacts of reaching the full load and to test whether 250 applications can be used as an appropriate risk trigger as system performance may already be degrading at that point.
Residual Requirements Risks
Residual risk is any leftover risk after the control and/or risk strategy has been implemented. It is risk that will be handed over to the operational environment to manage going forward. Let us assume the sponsor decided not to add the alert at this time but approved the load testing, which determined that there was no system performance issues until applications hit the 300 mark, and there were only about 50 applications being processed monthly; thus, it is not worth the investment for the alert at this time. The residual risk is that the volumes may still reach 300 in the future. By documenting it in a requirements risk log, you can use it to feed into the transition document to indicate that the project team is transferring this risk over to the operational team to manually monitor going forward.
Workarounds can be compared to contingency in project risk management. Workarounds are used to address any residual risk. A workaround can be a systematic or a manual solution. It is a means for the operational environment to manage the requirements risk in the operational environment. In the example we are following through, we can create an operational procedure as part of monthly cleanup activities to indicate that someone will manually assess application volumes to determine whether we are at or near the 250 applications per month, and to initiate a project request for additional servers if the threshold is reached.
Why Do We Conduct A Requirements Risk Assessment?
Conducting a requirement risk assessment has many benefits if performed correctly. Some of these benefits will be elaborated on in this section of the paper.
To Uncover Gaps in Requirements
The number one reason why we conduct a requirements risk assessment is to identify gaps in our requirements. A requirements risk assessment assists with uncovering both product and project requirements. As mentioned earlier in this publication, one of the ways to address requirements risks is to introduce controls, which are in essence additional product requirements or product requirements that would have been missed otherwise which subsequently translate into additional project requirements. Another way to address requirements risk is the introduction of additional risk strategies, which are in essence additional project requirements, or additional work to be executed by the project team in order to reduce or eliminate the requirements risk that may have also been missed otherwise. Project requirements fall under the responsibility of the PM to manage, so the PM and the BA should be partnering on the requirements risk assessment.
Highlight residual risk to the business
Any residual requirements risk will need to be presented to the sponsor for signoff with the requirements. This is to acknowledge that they are accepting the residual risk into the operational environment and agree that the workarounds are sufficient rather than adding additional controls or risk strategies to manage the requirements risk within the project. The requirements risk assessment can be used as a negotiation tool to either request more funding during the analysis phase to implement additional product and/or project requirements, or pushed to the operational teams to manage via workarounds.
Furthermore, the residual risks will also need to be presented to the operational team during transition of the project into the operational environment. The transitioning of the project is typically the accountability of the PM; therefore, it is recommended that the PM and the BA work together on this activity to ensure the residual risks are appropriately transitioned over to the operational teams.
Risk based testing to allow testing efforts to be focused on specific areas of high risk and impact
Testers are often faced with the dilemma of wanting to find the most defects while performing the minimal number of tests. A completed requirements risk assessment will allow the testers to focus on the riskier requirements first and these should also be tested the most, in comparison to less risky requirements.
May assist with the prioritization of development activities
Identifying the riskier requirements may also assist with the prioritization of the development activities, as developers may want to work on the riskier requirements first since these are likely the ones to cause the most defects, and thus require additional time to fix.
Identify project risks that may have otherwise been missed
It may have been noticed that the difference between some project and product risks is simply in how they are articulated. There are many requirements risks that are also project risks, which again, is another reason why it is recommended that the PM and BA work together on managing project and requirements risks. For instance, the requirements risk “If the software package cannot be configured to display 2 decimal places with no rounding, then our accounting may not be accurate” is also a potential project risk by modifying the risk statement – “If the software package cannot be configured to display 2 decimal places with no rounding, then additional time and cost will be required to custom code the application to meet our needs.” The PM and the BA are looking at the risk from a different angle; the PM from a project perspective and the BA from a product/solution or requirements perspective.
Not all requirements risks are also project risks. Consider the example “If there are more than 300 applications processed each month, then the system may crash”; assuming we are only at 50 applications processed each month and not expected to reach 300 until well beyond the project implementation date, it is not a project risk worth logging and investigating. It is however still a valid requirements risk that will need to be managed and possibly transitioned over to operations if not managed within the project.
Final Thoughts On Requirements Risk Management
There is no doubt that there is value in completing a requirements risk assessment even if your organization does not mandate that you do so, to minimize the probability of any missed requirements and to transition any residual risks. The requirements risk assessment is meant to generate discussion on potential failure points and exposures to ensure they will either be addressed via controls and/or risk strategies, and/or managed via workarounds.
Also as demonstrated in this paper, the BA should attempt to involve the PM in the requirements risk management process or at least have regular checkpoints to review results of the assessment to ensure that any requirements risks that are also project risks are managed in the project risk log; any additional project requirements resulting from risk strategies or added product requirements resulting from controls are managed appropriately; and to ensure requirements risks are appropriately transitioned over to the operational teams as part of project closing activities.
Project Management Institute. (2014). PMI pulse of the profession®: Requirements management–A core competency for project and program success. Newtown Square, PA: Author.
Schibi, O. & Lee, C. (2015). Effective PM and BA role collaboration – Delivering business value through projects and programs successfully. Plantation, FL: J. Ross Publishing.
© 2015, Cheryl Lee, CBAP, PMI-PBA, PMP & Ori Schibi, MBA, PMP
Originally published as a part of the 2015 PMI Global Congress Proceedings – Orlando, Florida, USA