The value of organizational project management (OPM) maturity--understanding, measuring, and delivering benefits
Glenn Strausser, MSP, MBA, PMP
The benefits of improving Organizational Project Management (OPM) maturity in organizations that conduct projects are often implied or taken as an obvious result of a project management improvement initiative. Although an organization will be better positioned and capable to deliver on project commitments as they move up the maturity ladder, research shows that quantifiable benefit realization is often not automatically achieved. This paper discusses the value of using a program management approach to systematically manage and deliver OPM benefits and to effectively establish the business relevance of the OPM improvement effort and the organization that executes it. It will also discuss metrics that focus on the business case for OPM improvement, review cases where this approach was applied and the degree of success it has had in realizing business benefit targets. It will also compare public research findings with Siemens’ results as well as the roles of maturity assessments, expected monetary value of projects, global program standards, and OPM3® organizational enablers.
With over 50% of Siemens revenue generated by project businesses that cover a wide spectrum of products, solutions, and service deliveries, OPM maturity at Siemens has global operational significance. In 2000, the executive board implemented an organizational project management improvement initiative to systematically and continuously improve performance on Siemens’ projects. This initiative became known as [email protected] As part of the [email protected]’ initiative, more than 170 organizational project management maturity assessments have been conducted as well as many at the project and program level. Due to the engineering nature of all Siemens projects, successful project performance not only depended on good project management, but also sales, purchasing, quality, senior management and engineering. To enable and assess excellence in these areas, the Siemens Maturity in Project Management (MPM) model is collaborative model using elements of OPM3®, CMMI®, Six Sigma and [email protected] best practices.
Improvement at the organizational level often requires a change in organizational culture and the environment of the enterprise. Change at this level is not possible without a commitment from executive leadership and organization support. More importantly, the objective of the initiative must be focused beyond establishing the new capabilities and processes needed. It must be driven towards the achievement of organizational strategic goals—the benefits. Any effort that does not deliver benefits to the organization is not likely to receive support from the executive leadership or the organization that must embrace it, sustain it, and continuously improve upon it. In a nutshell, the strength of project management is in capability delivery. Program management, as its complement, is focused on benefits to be delivered by using those capabilities and on effectively transforming the organization’s culture to sustain the benefits. The establishment of an OPM improvement program, alignment of the capabilities to organizational objectives, interpretation of the maturity models and coaching in this transformation is the valued and critical role of the OPM consultant.
OPM3® Standardized, Measured, Controlled, and Continuously Improved (SMCI) stages of maturity can be considered achievable using an improvement project approach to implement the capabilities defined by the model. However, without a program approach (i.e. focusing on organizational benefits and change), there is little reason for the organization to accept change and it is likely not to be sustainable. Further, in today’s ultra-lean resource environments, OPM improvement initiatives are in competition with many other initiatives, both voluntary and regulatory. When allocating scarce resources, the only fundable initiatives are those whose benefits are in direct alignment to the organization’s strategy. In order to expect a successful result from an OPM improvement effort, a program business case based on strategically aligned benefits must be defined to balance against the cost of the program and to gain organizational support for change.
Understanding the Business Case for OPM Maturity
The business case for OPM improvement can be summarized by answering this question: What is the value lost by the organization for not delivering on project commitments (e.g., planned cost, time, or scope)? Longer projects almost always cost more. Longer projects also utilize resources beyond the planned date, restricting them from working on other projects in the portfolio, which may in turn cause additional delays, adds risk, and weaken the portfolio. For project businesses that are contract based, an often used as a lost value metric is project margin (or profit) erosion—the difference between planned and actual project margin. In new product development projects, time-to-market can either make or break a product line business. The effects of being late to market such as sales volume changes, lost market share or net present value (NPV) are well known by marketing professionals. Internal projects, even though not typically associated with generating revenue directly and often overlooked, are key enablers to a business. Delay of key capabilities can add to the cost of productivity, loss of a competitive advantage and eventually loss to the bottom line. The general term used to quantify the summation of costs or value lost due to deviations from the plan is non-conformance costs (NCCs). In the extreme case, entire project costs may be considered as an NCC if the project objective was not met and costs cannot be recovered. This is a strong case for organizational project selection and review processes to accurately predict a project's outcome early enough to keep it successful or its recovery or termination costs low.
Exhibit 1 shows the relationship of non-conformance costs to statistical delivery performance for organizations at various levels of OPM maturity. Closing the gaps between actual and planned schedule/costs/scope and recovering lost opportunities is the value to be exploited by improving OPM maturity. These summary lost value metrics are ideal for the executive dashboard of an OPM improvement program.
An interesting note is that NCCs are the net effect between negative costs to the project (e.g., risk occurrences, rework, and lost opportunities) versus positive effects (e.g., risks averted and opportunities that were enhanced or realized). Very often, as an organization improves in maturity, not only are negative aspects eliminated, but positive opportunities gained are now visible. In a Siemens case study by Sopko and McDevitt (2009), the subject organization had experienced a margin recovery that was twice the measured margin erosion prior to the improvement effort. Another key aspect of that study was that as an organization achieves an organizationally defined or standardized stage of maturity, the benefits become sustainable since they are commonly implemented in practice throughout the organization and benefits of OPM improvement tracked by the PMO.
Another key metric at the heart of OPM maturity benefits is the project estimated monetary value (EMV). EMV is the value a project delivers to the organization upon completion. Projects and programs are investments—an organization would not engage in them unless it was seeking a value greater than their costs. The project EMV, and ultimately the portfolio or program value, is affected by deviations from the performance baselines. The task for the organization's project and program sponsors is to define the estimated monetary value (EMV) for each project for the project teams. The ability to plan and control projects effectively and accurately is directly related to the implementation of OPM best practices (i.e., OPM maturity). As an organization standardizes the quantification of project estimated monetary value (EMV), it becomes more understandably clear that deviations from project commitments have quantifiable and often negative effects on project value. If the project team is aware of project EMV and how project performance factors affect it, the project team is able to actively seek ways to optimize project EMV. EMV can be aggregated at the program or portfolio level to give an indication of portfolio effects of optimizing projects and in the end, benefits the organization. Many projects do not receive the resources needed, even at the beginning of the project, due to the missing analysis of EMV. An excellent analysis of project value metrics, how they are affected by deviations in project schedule, and optimization techniques is described by Devaux (1999).
Note—Maturity scales are shown for both the Siemens MPM model (Levels) and for OPM3® (SMCI stages). The “target” is the planned project goal in terms of costs, schedule or scope. The curve represents the actual statistical performance of the organization in accomplishing these objectives.
Exhibit 1: Non-Conformance Costs (NCCs) and Their Relationship with OPM Maturity.
Defining the Benefits
Benefits may be tangible or intangible in nature. Measurement of benefits isn't always obvious and, in some cases, the very first project an OPM improvement program may undertake is to develop effective benefit metrics as well as a performance baseline. Examples of tangible benefits can be:
Project margin improvement/recovery
Increased market share capture
Improved cash flow
Improved customer satisfaction
Improved time-to-market execution
Lower non-conformance costs (NCCs)
Improved safety, health, or environmental performance
Improved service delivery
Improved operational capability or availability
Intangible benefits, such as an improved workplace environment, more effective leadership, enhanced career opportunity, and improved use of resources are also important as these often affect the morale, retention of key resources, and attractiveness of the organization for the best and brightest which the market may offer. Brand recognition and good will are classic intangible benefits. In some programs, the intangible benefits may be the key measures of success. Another positive outcome of improved OPM maturity is increased trust from the customer whether internal or external. Consistently high quality delivery results in greater support for internal improvements and better customer satisfaction for external projects—and potentially more projects.
Using benefits profiles to formally describe and define the benefits that the organization desires to achieve enables the formation of realization plans. If the program stakeholders do not agree at the onset as to what benefits are to be achieved, it is difficult to determine what the future state of the organization should be in terms of processes, operations, technology, and information flow to effectively deliver and sustain those benefits.
Program Management—Delivering the Benefits
Have you ever experienced the implementation of a new enterprise project management system, product, process, or company-wide IT application that met all expected performance expectations—even completed on-time and budget—yet doesn't realize the full organizational benefit that it was intended to deliver for months or even years after the project is completed? In simple terms, this may be a case of project success, but program failure. Why should OPM improvement initiatives behave any differently?
Project management has been around for a long time. It's proven, mature, and is often a default management model for unique, temporary initiatives or endeavors. However, not everything is a project. If one is challenged with an initiative that has a vision with no clear path to the objective, which is coupled with the delivery of organizational benefits in its description—it's a prime candidate for program management.
Why a program? Benefits from OPM maturity improvement are not automatic. In research conducted by Thomas and Mullaly (2008), the degree of correlation of tangible benefits achievement to the organization's project management maturity stage was not evident in thirty-two of sixty organizations surveyed—53%. To seasoned project management professionals, this does seem counter-intuitive at first glance. We have a new capability/service/product/process. Why can't we achieve the benefit it was intended to enable? It is conceivable that improvement projects may deliver needed OPM capabilities (which may improve their maturity level or stage) only to fall short in the organization's ability to effectively use them and to achieve the intended benefits. Are these improvement initiatives being managed as programs or projects?
A 2005 survey conducted by KPMG International cited a number of findings regarding the shortfalls in benefits delivery by projects.
Nearly one in three organizations does not always define financial benefits in their business cases.
The most popular definition of success today is meeting promised project benefits—in other words, keeping commitments.
For the majority of survey participants, project success appears to equate to achieving an acceptable level of failure or reducing lost benefits.
In only 23% of cases is delivery of project benefits tied to individual executive performance plans and rewards.
Inadequate benefits management processes prevent the articulation of program and project success and this creates a largely unquantifiable degree of benefits leakage.
Only 2% of organizations claimed they achieved targeted benefits all the time in the past 12 months.
Hobbs (2007) also noted in his survey of ~ 500 PMOs (those typically responsible for OPM processes and maturity improvements) that benefits management was only cited as an important function in 28% of the responses. One could make the assumption that the lack of both a benefits focus and the continuous value delivery of an OPM improvement program could contribute to the short life expectancy of PMOs. If the PMO is not actively managing the delivery of benefits from improving OPM maturity, then how is the business relevance of the PMO determined? If the improvement endeavor is viewed as a project, then you must expect the question to arise: Aren't we done? With that answer goes both the PMO and the OPM improvement program's fate.
Another shortfall of focusing on capabilities or maturity levels alone is the fact that the improvement initiative appears to only have a cost side to the business case with intangible benefits as the only upside. A business case for a project is typically a comparison of project costs versus project scope whereas a program business case is a comparison of program costs versus organizational benefits. Capabilities such as a project planning process or project management software may be critical to the foundation for OPM best practices, but without formal alignment to organizational benefits, the efforts to create and implement these capabilities may be cut short due to cost reduction efforts.
As explained by Sopko and McDevitt (2009) and Strausser and Zeller (2010), Siemens Corporate Research (SCR) has cited program management as one of the key success factors for achieving benefits from OPM maturity improvement. Without a formal alignment to organizational strategy, improvement projects and the initiatives that sponsor them tend to fall short in delivering expectations. Even with the aid of a corporate directive to improve OPM maturity and an internal maturity model using global standards and best practices, the rate of improvement does not always meet expectations. Business leaders were not only interested in the capability development aspects of [email protected] and MPM, they needed to understand the benefits to their organizations. As in many organizations, there are many initiatives and usually not as many resources or funding. Specifically, executives were most interested in how these initiatives were actively helping in meeting their strategic business objectives—how they were being graded.
A particularly noteworthy organizational enabler from OPM case studies at Siemens was the paradigm shift from a project (execution and delivery) approach to one that embraced program management (e.g., benefits realization and business change). Defining, measuring, controlling, and achieving organizational benefits placed OPM maturity improvement on the executive dashboard. The focus on business benefits, organizational change, and an iterative maturity analysis and coaching approach are proving effective in establishing a critical support infrastructure, standardizing best practices, business value alignment, and enabling the organization to establish and sustain an improved level of organizational project management maturity.
OPM3® defines organizational enabler (OE) best practices that support the creation of a program to achieve OPM maturity (BP 6980) as well as a business change program (BP 7035). Both program and project management play critical roles in OPM improvement. This is also critical to the value proposition of the PMO. The PMO that does not recognize the difference between programs and projects will often find its relevance increasingly challenged as OPM capabilities are achieved. Without the continuous value proposition that is inherent in programs, the result may be an early termination of the OPM improvement effort, even when a significant amount of benefits remain to be reached.
To identify and define a program SCR adopted the UK Office of Government Commerce's (OGC) Managing Successful Programmes (MSP). A comparison of the PMI and UK OGC program management standards is given by Strausser and Zeller (2010). A few key practices used in program definition can significantly improve the effectiveness of OPM improvement value delivery. These are:
Determining capability gaps
Benefit profiles are a recommended practice in MSP that determines the definition of a benefit, how it is to be observed, to what parts of the program will the benefits be attributed or mapped, and how the benefit will be measured. Defining and understanding the benefits are important in constructing the future state of the organization so that it can consistently deliver these benefits. The document that defines the needed architecture in terms of required processes, organization, technology, and information flow that is capable of delivering the benefits. This is also termed the program architecture in PMI's The Standard for Program Management. Capability gaps between the current state of the organization and the blueprint (i.e., the future desired state) is the role of OPM maturity models. These artifacts can set a solid foundation for a program that will aid in a more effective definition of the business case, the component projects, and their linkage to achieving the program's goals and objectives. An organizational governance model for programs as defined by Managing Successful Programmes is shown in Exhibit 2.
Exhibit 2: Program management governance roles.
Measuring the Benefits
A question we often ask in our maturity assessments is: “Now that you have a standardized project planning process that takes six weeks instead of four, is this good for your organization?” We should not be doing anything that does not help us meet our objectives (e.g., making our lives better, our business more profitable, preserves our resources, or achieving our goodwill intentions). How will you know this unless you are measuring the effects?
Implementing an OPM process or achieving a Level 3 maturity are not benefits, they are capabilities. In our opinion, one of the more important capabilities in OPM3® measured stage is assessing the existence of measurements that determine the effectiveness of the implemented processes—satisfying the process customers requirements. Is the new process capability solving the problem it was intended to fix? This single capability in OPM3® will enable the determination of whether the OPM improvement team has implemented something that benefits the organization, or simply created more overhead work. In this capacity, the OPM improvement program must ensure the organizational adoption of the new process (business change), transition it to consistent implementation, and either achieve the benefit for which the capability was created, or fix that capability. NCCs are metrics that indicate a deficiency in a process. When NCCs are attributed to an element in the project work breakdown structure and the processes associated with it, we have a measurement of OPM process effectiveness. In summary, a PMO that measures process effectiveness and improves upon it makes everyone's life easier and will likely enjoy a long and prosperous life!
Benefits can be measured in many ways and the metric and technique will vary by organization. In the case of Siemens I IA, three benefit areas targeted by the CFO were: customer satisfaction, project delivery performance, and project margin improvement. Customer satisfaction was determined by independent surveys conducted as a follow-up to projects. Project delivery performance was measured by timeliness with regard to customer schedule commitments. Project margin was calculated by comparing planned project margin (profit) versus actual project margin at project close-out.
Results in Practice at Siemens
In the Siemens Industry case study mentioned, the ratio of organizational benefits to program costs was on the order of ~7:1. This is in line with other research conducted by SEI and the Department of Defense, Data & Analysis Center for Software (2007) using the CMMI model and global assessment data. Remarkably, due to the sustainability of project margin recovery, a planned reduction in staff was offset, saving a number of jobs. The organization improved in customer satisfaction, project delivery, project margin, and revenue in spite of one of the worst economic downturns in its history and a very aggressive market. On top of that, Siemens underwent one of the largest reorganizations in Siemens history. This measured success through a turbulent and changing enterprise environment is a testimony to the ability of a program to manage through organizational changes and stay focused on the strategic benefits. Exhibit 3 shows the benefits targeted by the business excellence improvement program at Siemens Industry—Industry Automation (I IA) from 2007 to 2009.
Exhibit 3: OPM improvement program results after initial phase of the [email protected] Program at Siemens Industry – Industry Automation (I IA) (Sopko and McDevitt) (2009))
Projects deliver capabilities. Programs deliver benefits. Benefits encourage organizational change and support. For an OPM initiative to be successful, it must have the right organizational environment and alignment of the OPM capabilities and practices it delivers to strategic benefits for it to succeed. Most importantly, senior management is typically not overly supportive of capabilities alone, especially in today's lean organizations, unless they are delivering benefits that help them achieve their organizational leadership goals. Alignment of capabilities to these goals defines the organizational relevance of the initiative and the organization that leads it. It's necessary for the organization to adopt new capabilities, embed them through organizational change management, and transition them towards the intended benefits. This often requires processes and infrastructure support beyond what project management alone delivers. It takes a focus on benefits and organizational change.
Devaux, S. (1999). Total project management—A manager's guide to integrated project planning, measuring, and tracking. New York, NY: Wiley.
Hobbs, B. (2007). The multi-project PMO: A global analysis of the current state of practice. A White Paper prepared for the Project Management Institute. Newtown Square, PA: Project Management Institute.
KPMG International (2005). 2005 Global Programme Management Survey, KPMG LLP, United Kingdom
Office of Government Commerce. (2007). Managing Successful Programmes (3rd ed.). Norwich, UK: The Stationery Office.
McGibbon, T., Ferens, D., & Vienneau, R. (2007). DACS Report Number 347616, Ft. Belvoir, VA: U.S. Department of Defense, Data & Analysis Center for Software.
Project Management Institute. (2008). A guide to the project management body of knowledge (PMBOK® guide)—Fourth edition. Newtown Square, PA: Author.
Project Management Institute. (2008). The standard for program management—Second edition. Newtown Square, PA: Author.
Project Management Institute. (2008). Organizational Project Management Maturity Model—OPM3® —Second edition. Newtown Square, PA: Author.
Siemens. (2009). Global project management—Global Standards for project success, Version 5. CT O PM/[email protected] Core Team. Munich, Germany: Siemens AG.
Sopko, J., & McDevitt, K. (2009). Accelerating organizational project management maturity at siemens, PMI Global Congress, Orlando, FL.
Sopko, J., & Strausser, G (2008). Case study—Program management and the PMO—critical success factors in improving organizational PM maturity at Siemens. PMO Symposium, San Antonio, TX.
Strausser, G. & Zeller, C. (2010), Business Change Management Using Program Management, PMI EMEA Congress, Milan, IT.
Thomas, J., & Mullaly, M. (Eds.) (2008). Researching the Value of Project Management, Newtown Square, PA: Project Management Institute.
© 2010, Joseph Sopko and Glenn Strausser Published as a part of 2011 PMI Global Congress Proceedings – Dublin, Ireland