Risk management @ risk
what are the opportunities PMI and INCOSE should work on first?
David A. Maynard
Manager, PMI Risk Management Community of Practice
Instructor, Purdue University
The aim of engineering program risk management is to ensure that program deliverables are produced on time, within budget, and to expected quality levels. However, the award-winning MIT-PMI-INCOSE “Guide to Lean Enablers for Managing Engineering Programs” states that programs today are “plagued” with cost and schedule overruns, and falling short of customer/stakeholder expectations.
There has been no shortage of effort. Today, we have more risk management standards and guides for more industries and specialized professions than ever before! But does more mean better? The International Organization for Standardization (ISO) now offers two definitions for “risk.” Unfortunately, they conflict.
Recent research indicates that there is confusion and disagreement about how to perform risk management, and insufficient application of basic best practices.
In support of the PMI-INCOSE Strategic Alliance, the PMI Risk Management Community of Practice and the INCOSE Risk Management Working Group are collaborating to identify and develop opportunities for improvement.
Introduction and Background
Project managers, program managers, and systems engineers play leadership roles in the design and implementation of key organizational initiatives. The types of initiatives vary and can include complex engineering programs such as the building of infrastructure, improvements to healthcare delivery, managing equipment acquisition for national defense, and delivering many types of competitive new products. At times, however, these professionals apply their own distinct practices independently, without consideration for the effect on the other's ability to perform their activities. This can result in mix-ups, delays, increased cost, poor quality deliverables, customer/stakeholder dissatisfaction, lawsuits, and even death or serious injury.
This paper provides an overview of the PMI-INOCSE Strategic Alliance, describes the challenges faced by today's engineering program risk managers, presents the results of relevant work completed thus far under the PMI-INCOSE Strategic Alliance, and proposes a strategy for the INCOSE Risk Management Working Group and PMI Risk Management Community of Practice as they move forward to explore and develop opportunities.
About the PMI-INCOSE Strategic Alliance
In September of 2011, PMI and INCOSE formed a Strategic Alliance to better integrate the project/program management and systems engineering functions. Adding strength to the alliance, INCOSE and PMI have partnered with MIT’s Lean Advancement Initiative (LAI) to help facilitate research, identify strategies and key practices, and disseminate the information.
A formally chartered PMI-INCOSE Alliance Working Group was established shortly after the announcement of the alliance. The working group (WG) is co-chaired jointly by representatives from PMI and INCOSE, and has membership consisting of INCOSE staff, PMI staff, MIT staff, and leadership from the INCOSE Lean Systems Engineering WG, the INCOSE Risk Management WG, and the INCOSE Core Competencies WG.
The PMI-INCOSE Alliance WG has made a concerted effort to disseminated information about the alliance, its activities, and its work products through press releases, articles, and postings to the web. Links to information sources and free downloads of the publications are posted on the INCOSE, PMI and MIT websites.
Description of INCOSE
The International Council on Systems Engineering (INCOSE) is a not-for-profit organization founded in 1990 to develop and disseminate the interdisciplinary principles and practices that enable the realization of successful systems. INCOSE’s mission is to share, promote and advance the best of systems engineering from across the globe for the benefit of humanity and the planet. INCOSE has over 8,000 members in more than 70 chapters worldwide, and nearly 90 organizations from industry, academia and government on their Corporate Advisory Board.
Description of PMI
The Project Management Institute (PMI) is the world's largest not-for-profit membership association for the project management profession. Its professional resources and research empower more than 700,000 members, credential holders and volunteers in nearly every country in the world to enhance their careers, improve their organizations’ success and further mature the profession. PMI has a globally recognized standard and certification program, extensive academic and market research programs, chapters and communities of practice, and professional development opportunities reinforce PMI’s worldwide advocacy for project management.
About the PMI-INCOSE Collaboration on Risk Management
The leaders of the PMI Risk Management Community of Practice (CoP) and the INCOSE Risk Management Working Group (WG) made initial contact regarding the PMI-INCOSE Strategic Alliance in December of 2011. Several teleconference meetings were held to discuss ideas on how they might support the PMI-INCOSE alliance, and presentations outlining their ideas were prepared.
In January of 2012, at a meeting of the INCOSE Risk Management WG at the INCOSE International Workshop in Jacksonville, FL, the Manager of the PMI Risk Management CoP and the Co-Chairs of the INCOSE Risk Management WG solicited feedback from the INCOSE Risk Management WG members. There was unanimous agreement to move forward to support the alliance and collaborate on risk management-related aspects. Several members stressed the importance of carefully defining the scope and problem(s) before beginning the effort.
According to the INCOSE Systems Engineering Handbook, Section 1.5 “One of the System Engineer's first jobs on a project is to establish nomenclature and terminology that support clear, unambiguous communication and definition of the system and its functions, elements, operations, and associated processes” (INCOSE, 2011, p 4). Likewise, current best practices for project, program, and risk management emphasize communication.
Since the audience for this paper includes members of two communities who may not be familiar with each other's terminology, naming and definitions the core concepts used in this paper are provided.
Definition of “Program”
The Project Management Institute (PMI) defines program as “a group of related projects, subprograms, and program activities that are managed in a coordinated way to obtain benefits not available from managing them individually. Programs are comprised of various components – the majority of these being the individual projects within the program. Programs may also include other work related to the component projects such as training and operations and maintenance activities” (PMI, 2013).
Definition of “System”
The INCOSE Systems Engineering Handbook defines a system as “a combination of interacting elements organized to achieve one or more purposes” (INCOSE, 2011, p 362).
Definition of “Program Management”
According to PMI’s “Standard for Program Management – Third Edition,” program management is the application of knowledge, skills, tools, and techniques to a program to meet the program requirements and to obtain benefits and control not available by managing projects individually. It involves aligning multiple components to achieve the program goals and allows for optimized or integrated cost, schedule, and effort. There are five program management domains: (1) Program Strategy Alignment, (2) Program Benefits Management, (3) Program Stakeholder Engagement, (4) Program Governance, and (5) Program Life Cycle Management (PMI, 2013).
Definition of “Systems Engineering”
Systems engineering is a discipline that concentrates on the design and application of the whole (system) as distinct from its parts. It involves looking at a problem in its entirety, taking into account all the facets of all the variables and relating the social to the technical aspect. Systems engineering utilizes an interdisciplinary approach that focuses on defining customer needs and requirements for system functionality early in the development cycle. It considers both the business and technical needs of all customers with a goal of providing a quality product that meets these needs. The scope of systems engineering includes 25 processes grouped into the following four areas:
- Technical Processes (e.g., requirements definition, design, integration and test, verification)
- Project Processes (e.g., project planning and control, decision management, risk management)
- Agreement Processes (acquisition and supply)
- Organizational Project-enabling Processes (e.g., project portfolio management, human resources)
The 25 processes within the above groupings are performed over the following seven system life cycle stages: (1) Exploratory Research, (2) Concept, (3) Development, (4) Production, (5) Utilization, (6) Support, and (7) Retirement (INCOSE, 2011, pp 1-3, 24-27, 363).
Assessment of the Current Situation – Risk Management @ Risk
The following excerpt from “The Guide to Lean Enablers for Managing Engineering Programs” provides a rather candid assessment of the current overall situation for engineering programs in general:
“We have come to accept that big programs mean big problems, big bills, and big delays. In addition, we accept that there is constant bickering between functional silos; conflicts among customers, contractors, and suppliers that lead to frequent irritations, animosity, and open hostility; lawyers and bureaucrats run the programs; and no work other than writing reports gets done. Conveniently, the excuses for doing so are endless (e.g., no time for managing the program better because everyone is busy fixing problems, requirements change all the time, regulations and compliance replace efficiency, new technologies fail, suppliers do not stick to their promises, and qualified people are impossible to find)” (Oehmen, 2012, p. v).
Challenges in Risk Management
The challenges facing risk managers, and the risk management profession as a whole, are formidable and multifaceted. While the above general situation originates, to a large extent, from outside the risk management domain, there are also challenges to the risk manager that come from with his/her own profession.
The field of risk management has grown from many different roots, and there is not ideal uniformity in language, practices, and application. This has caused some dissention and confusion within the field, and could be raising doubts about the effectiveness of risk management when programs that employ risk management still fail to meet objectives. Exhibit 1 is intended to provide a sense of the extent of variation in basic risk management concepts. This table contains a sample of “standard” definitions of the term “risk” from a variety of principal sources.
Exhibit 1 – Various Definitions of the Word “Risk” Likely to be Encountered in an Engineering Program
The definitions in Exhibit 1 are typical of the variation associated with the term “risk” that can be expected in most any cross-functional engineering program. The communication issues caused by such variation are not hard to imagine. The total amount of variation, extrapolated to include other basic terms used in risk management, as well definition of processes, methods, techniques, and so on, is substantial, and could be problematic for some programs.
Variation and Diversity in Risk Management
Few areas of established professional practice are as diverse and dynamic as risk management. The reasons are simple and quite understandable:
- Risk management-related concepts and practices have evolved and become established in many different industries and government agencies. Some examples are:
- The insurance and banking industries
- The construction, nuclear power, and automobile industries
- The aerospace and defense industries
- The medical device, pharmaceutical, and healthcare industries
- The Food and Drug Administration (FDA)
- The Department of Defense (DoD)
- The Department of Energy (DoE)
- The Occupational Safety and Health Administration (OSHA)
- Risk management-related concepts and practices have evolved and become established in many different professions. Some examples are:
- Safety, reliability, quality, industrial, and human factors engineering
- Information technology (IT) and software engineering
- Systems engineering
- Business management (at the enterprise, portfolio, and project/program levels)
- Law enforcement and disaster management
- For the most part, each industry, government agency, and profession (only some of which are listed above) developed their individual risk management-related practices and publications separately.
Differences Between PMI and INCOSE Risk Concepts
Exhibit 2 is intended to show visually how INCOSE and PMI currently name and define fundamental concepts having to do with positive and negative (desirable and undesirable) events, situations and outcomes that could be encountered in a systems engineering or project/program environment. It corresponds to rows 1 and 6 in Exhibit 1.
Exhibit 2 – Illustration of Foundational Risk-Related Concepts and Definitions (INCOSE and PMI)
Key Project/Program Risk Management Standards
In 2009, the International Organization for Standardization (ISO) released ISO 31000, “Risk Management -Principles and guidelines.” This document provides a set of guidelines and models designed to help mitigate and minimize risks in business applications, and organizations of most any type (ISO, 2009, pp. 1-2). It is designed to help organizations manage risks and their impact. ISO 31000 was developed with participation from 30 countries, and represents a new and powerful global standard for risk management.
PMI’s publication A Guide to the Project Management Body of Knowledge (PMBOK® Guide) presents a set of terminology and guidelines for project management certification (PMI, 2013). The Fifth Edition is also published as ANSI/PMI 99-001-2008 by the American National Standards Institute and as IEEE 1490-2011 by the Institute of Electrical and Electronics Engineers. The PMBOK® Guide states that “project risk is an uncertain event or condition that, if it occurs, has a positive or negative effect on one or more project objectives such as scope, schedule, cost, and quality,” and that “project risk has its origins in the uncertainty present in all projects.”
Risk management is one of 25 processes defined in the INCOSE Systems Engineering Handbook. Section 5.4, Risk Management Process, of version 3.2.2 of the handbook, contains a description of a risk management process along with additional information and guidance useful in implementing the process. The Safety, Reliability, and Security Engineering sections of the handbook also pertain to risk (INCOSE, 2011, pp. 213-225, 312-315, 323-325).
It is pointed out here that all three of these standards describe iterative processes where risk management is ongoing during the entire project life-cycle. This iteration has been shown to be important by Floricel and Miller's study, which found that even with a thorough and careful risk identification phase, unexpected events, conditions, and outcomes occurred in every project (Floricel, 2001).
Summary of Research
Risk management is recognized as one of the most critical project management practices. For example, Royer states that “experience has shown that risk management must be of critical concern to project managers, as unmanaged or unmitigated risks are one of the primary causes of project failure” (Royer, 2000, p. 7).
A report by PM Solutions, “Strategies for Project Recovery,” examined a spread of 163 small, medium, and large organizations having an average of $200 million in projects each year. Respondents reported a 37 percent “at risk” rate for their projects, which equates to an average of $74 million “at risk” for each project (Krigsman, 2011).
In a recent paper by Eileen Arnold, “Call for an Effective Alignment of Program Management and Systems Engineering Risk Management Practices,” 12 recommendations were made (Arnold, 2013, p. 14). For the purposes and context of this paper, her recommendations have been consolidated and paraphrased to the following seven areas of focus:
- Align the language of risk management.
- Establish a cross-discipline risk management process based on an alignment of standards and certification bases.
- Align stakeholder input to include risks from project management, product management, finance, communication, customers, etc., according to the stakeholder register
- Utilize a combination of ISO 31000 (2009), ISO Guide 73:2002, PMI’s PMBOK® Guide (Chapter 11), and PMI’s Practice Standard for Project Risk Management, for enhanced alignment where appropriate.
- Create templates for a coordinated multi-discipline Risk Management Plan.
- Create a coordinated set of suggested outlines for a Project Management Plan and Systems Engineering Management Plan with tailored indications from the systems engineering, project management, program management, and other applicable disciplines.
- Align risk handling with monitor/control activities and their definition and language used.
Section 4 of the joint MIT-INCOSE-PMI publication A Guide to Lean Enablers for Managing Engineering Programs defines the “Top Ten Themes of Challenges for Managing Engineering Programs” (Oehmen, 2012, p.29). Theme 1, “Firefighting,” and Theme 9, “Lack of Proactive Risk Management,” underline the importance of risk management to the success of engineering programs. Thus there is rationale to categorize them as “high-priority.”
While it may be true that many, perhaps even most, engineering programs are failing to meet objectives, the fact that a significant number of programs do meet objectives should not be ignored. What are the successful programs doing that make them successful, and how do they make sure that they are doing those things well? The answers to these questions may very well open up a plethora of opportunities for organizations, programs, and project teams who want to (and perhaps must) improve their chances of success.
Best Practices in Risk Management
Over the past few decades, much has been learned about the importance of systems analysis, and the cultural, psychological, and other human factors affecting risk, particularly in the field of safety engineering (Leveson, 2011, pp. 350-374). It has been established that communication, integration of risk management with other organizational processes, consideration of both internal and external factors, and analysis for unintended consequences are also important (Bahill, 2011, pp. 1-2). The recognition of these factors as elements of risk management is a testament to the need for systems thinking and that attention to process enablers is necessary for effective risk management.
Until recently, risk management standards have focused primarily on risk management process, methods, techniques and tools. With the release of ISO 31000, this changed. Exhibit 3 illustrates the scope of the ISO 31000 model for risk management. As can be seen, it incorporates elements for principles, framework, and process, with additional (not typically specified) process elements for “establishing the context” and “communication and consultation.”
Exhibit 3 – Illustration of the ISO 31000 Principles-Framework-Process Model for Managing Risk
Thus ISO 31000 encourages a systems approach to risk management and provides guidance for implementing risk management systems tailored to the specific organization/program and the objectives at hand, while considering the context, and applicable internal and external factors. The ISO 31000 model for risk management allows for the integration of diverse risk management terminology, practices, methods, tools, and techniques, and can accommodate PMI and INCOSE best practices, as well as the enablers and challenges detailed in The Guide to Lean Enablers for Management Engineering Programs recommended by MIT, INCOSE, and PMI.
Lessons from Systems Engineering
At the enterprise level, embracing systems engineering is strategic in that it is, fundamentally, a systematic way to address risk in engineering programs – especially large, complex engineering programs. Systems engineering processes, practices, methods and tools are designed to increase the chances of success and reduce uncertainty. Exhibit 4 (INCOSE, 2011, p. 18) shows how the proper application of systems engineering increases the probability of engineering program success. All 25 systems engineering processes are important, but we would like to emphasize here that requirements management is critical. The ROI for developing good requirements at the beginning stages of a program is high, while inadequacies in requirements are a major risk (Wheatcraft, 2011, p. 1).
Exhibit 4 – Engineering Program Data Shows that Allocating 15% of Program Cost for Properly Executed Systems Engineering Reduced Uncertainty Significantly and Eliminated Cost and Schedule Overruns
Lessons from Lean Program Management
The Guide to Lean Enablers for Managing Engineering Programs presents the results of an analysis comparing successful engineering programs to unsuccessful engineering programs (Oehmen, 2012, p. 6). The data, shown in Exhibit 5, indicate that the successful programs apply the Lean Enablers (chart at left), and that the successful programs consistently meet objectives in all areas (chart at right). Conversely, programs that do not apply the Lean Enablers are unsuccessful, and consistently fail to meet objectives. The correlations are statistically significant.
Exhibit 5 – Programs that Apply the Lean Enablers Consistently Meet/Exceed Objectives
Conclusion – Pioneering a Systems Approach to Risk Management
The PMI-INCOSE Risk Management Collaboration leadership team has identified PMI and INCOSE risk management practitioners as their primary stakeholders. Outreach activities have been conducted. These include teleconferences, presentations to local PMI-INCOSE chapter meetings, and a panel session at the 2013 INCOSE International Symposium. We thank those who participated and appreciate the excellent feedback, problem descriptions, and ideas for improvement. These have been taken into account in formation of the following strategy:
- Commit to pioneering a systems approach to engineering program risk management. Utilize the ISO 31000 model (Principles-Framework-Process for Risk Management) as a guide to enabling the systems approach. Leverage the ISO 31000 model to address problems related to inconsistencies in risk management-related terminology and practices, and to facilitate effective integration of risk management into organizations.
- Embrace the Lean Enablers for Managing Engineering Programs, recognizing that failure to properly implement the enablers is a source of risk to engineering programs. Focus first on Theme 9, “Lack of Proactive Risk Management,” as outlined in the “Top Ten Challenges for Managing Engineering Programs” section of the guide and the seven areas of focus listed in the “Summary of Research” section of this paper.
- Embrace proven and established best practices for systems engineering, project/program management, and risk management published by INCOSE and PMI, recognizing that failure to properly implement these practices is a source of risk to engineering programs.
- Leverage the resources and talent available through the PMI Risk Management Community of Practice and the INCOSE Risk Management WG to develop a plan to implement this strategy. Apply Lean Program Management to the collaboration effort, and maintain focus on the stakeholders, especially PMI and INCOSE members.
While the solutions to the specific problems identified in this paper are critical to the intent of the PMI-INCOSE Collaboration on Risk Management, achieving them will not be rapid or easy. Risk management is not limited to a few processes, but includes the interaction with many other internal and external processes. The authors of this paper therefore suggest a two-step approach. First, we recommend an immediate “take-away” solution followed up with a longer-term study, action, and refinement.
Short Term – Engineering program risk management, at the enterprise/company level, should take into account the high-priority challenges identified in this paper. Project managers and systems engineers should work with their core program team during the initial planning stages to address the prioritized items as a team. Notes should be taken, and the information collected as a pre-cursor to the risk management portion of the project risk management plan. One possible approach is to create a checklist that offers yes/no answers to a series of questions. A preponderance of “yes” or “no” answers would indicate either agreement or disagreement on that particular recommendation. In the case of a disagreement, the team must work out their differences before proceeding with the project team. A disagreement in the fundamentals of managing risk at this stage of the project will be of critical impact.
Long Term – Over the long term, a more ambitious plan is required. The joint members of INCOSE Risk Management WG and the PMI Risk Management Community of Practice, empowered by the PMI-INCOSE Strategic Alliance WG should work together to develop a “best union” of the two sets of risk management knowledge – PMI’s and INCOSE’s. This “best union” would represent a set of risk practices and processes that would be best used in an engineering program. The “best union” risk management documentation would be the result of many hours of joint research, brainstorming, opportunity exploration, discussion, analysis, debate, and consensus-making.
It is vitally important that program/project managers and systems engineers utilize the same language, tools and philosophy when discussing project and program risk on an engineering program. This is currently lacking, as has been shown in this paper, and in the joint work of PMI, INCOSE and MIT. Both INCOSE and PMI have thriving risk management communities of practice that are continually researching new and better ways of practicing risk management. A collaborative effort by these two communities, empowered by the PMI-INCOSE Alliance WG to transform the strategy and high-priority challenges identified in this paper into tangible products, is necessary.
Arnold, E. (2013, June). Call for an effective alignment of program management and systems engineering risk management practices. INCOSE 23rd Annual International Symposium, Philadelphia, PA.
Bahill, T. (2011, June). Diogenes: A Process for Identifying Unintended Consequences. INCOSE 21st Annual International Symposium, Denver, CO.
Floricel, S., & Miller, R. (2001). Strategizing for anticipated risks and turbulence in large-scale engineering projects, International Journal of Project Management.
INCOSE (International Council on Systems Engineering). (2011). INCOSE Systems Engineering Handbook. (Version 3.2.2). San Diego, CA: INCOSE.
ISO (International Organization for Standardization). (2007). ISO/IEC Guide 51, Safety aspects -- Guidelines for their inclusion in standards. Geneva, Switzerland: ISO.
ISO (International Organization for Standardization). (2009). ISO 31000, Risk management - Principles and guidelines. Geneva, Switzerland: ISO.
ISO (International Organization for Standardization). (2009). ISO Guide 73, Risk management - Vocabulary. Geneva, Switzerland: ISO.
Krigsman. (2011). New research identifies five important reasons that projects fail. Strategies for project recovery. Retrieved from http://www.zdnet.com/blog/projectfailures/cio-analysis-why-37-percent-of-projects-fail/12565.
Leveson, N. (2011). Engineering a Safer World: Systems thinking applied to safety. Cambridge, MA: MIT Press.
Oehmen, J. (Ed.). (2012). The guide to lean enablers for managing engineering programs, version 1.0. Cambridge, MA: Joint MIT-PMI-INCOSE Community of Practice on Lean in Program Management.
Royer, P. S. (2000). Risk management: The undiscovered dimension of project management. Project Management Journal, 31(1), 6-13.
Project Management Institute. (2013). A guide to the project management body of knowledge (PMBOK® guide) – Fifth edition. Newtown Square, PA: Author.
Project Management Institute. (2013). The Standard for Program Management. Newtown Square, PA: Author.
Wheatcraft, L. (2011, June). Triple your chances of project success: Risk and requirements. INCOSE 21st Annual International Symposium, Denver, CO.
©2013 Jack Stein and David Maynard
Originally published as a part of 2013 PMI Global Congress Proceedings – New Orleans, Louisiana