Bridging OPM3® and CMMI®--a case study
The current interest in improving business and Information Technology (IT) processes as evident by the popularity of Six Sigma, LEAN, Organizational Project Management Maturity Model Knowledge Foundation (OPM3®), CMMI® and ITIL® in many of the world's top corporations also presents a problem when these different approaches and tools converge in an effort to identify and improve Project Management processes.
This case study follows one organization's opportunity and approach to leveraging the best of CMMI® and OPM3® to create a bridge providing a seamless approach and communication mechanism across the Lines of Business (LOBs) and IT to improve Project Management Maturity, which studies suggested equated to an improvement in project delivery.
As a preface to the case study, it is important to describe at a high level what CMMI® and OPM3® are and are not in the Project Management maturity space. CMMI® is the latest version in a long history of Capability Maturity Models developed originally for the United States Department of Defense and managed by the Software Engineering Institute at Carnegie-Mellon University and is the industry leader in maturity measurement. CMMI® takes the best of the multiple Capability Maturity Models and presents a maturity model across the product lifecycle, including IT. What CMMI® is not is a maturity measurement for Organizational Project Management, including Program or Portfolio Management. OPM3®, now going into its 2nd edition is Project Management Institute's (PMI®) standard for measuring organizational project management maturity across the domains of Project, Program and Portfolio using Best Practices and Capabilities. What OPM3® is not is a product lifecycle maturity model. What the two do share is the foundation of Project Management and the focus on Project Management through the maturity continuum.
In early 2003, the Enterprise Project Management Organization (EPMO) Executive of a top financial services corporation in the United States charted a Design for Six Sigma (DFSS) project to develop and implement a process and related tools to measure Project Management maturity through identifying gaps and developing a roadmap to close these gaps in an effort to move the organization to a “world class” status. The EPMO was responsible for all Project, Program, Portfolio, Project Risk and Project Financial Standards, Methods and Tools for the enterprise, including IT. A cross enterprise project team of experienced project managers was created with a stakeholder committee representing key senior executives from the Lines of Business and IT. From this group this problem statement was created; “There is no definition or measurement system to determine what is “world class” or to gauge our progress against that standard for managing projects. There is a lack of information to accurately forecast and/or predict progress against a “world class” standard. The cost of poor quality of Project Management is estimated at $325MM over the next two years”. This case study follows that corporation's DFSS Methodology which used the Six Sigma DMAIC Framework.
Phase One – Design
The first realization for the project team was that the IT Group had undertaken an improvement initiative using CMM® and had achieved Level 3 and was in the process of migrating to CMMI® and at the same time PMI® had released OPM3® which the EPMO along with the Project Management community were leaning towards using. This presented an obstacle for the team, because the initial approach was to use CMMI® to align the entire enterprise on one model, but the project team knew that based on PMI®'s track record with standards that in a few years, OPM3® would become the de facto standard. Based on these facts, the project team embarked on a detailed examination of both models and an attempt to create a bridge between them by creating a new assessment model, process and tools.
The project team was expanded with key resources and stakeholders from both the IT Group and the Project Management community and the project team conducted in-depth Voice of Customer (VOC) sessions with key stakeholders across the enterprise. The VOC findings were loaded into a Kano Analysis to prioritize customer requirements based on their impact to customer satisfaction and based on Kano Analysis the project team's success would be measured by if and to what degree the maturity assessment solution accomplished all the items identified as “Must Be's “ and to a great extent the One Dimensionals.
Exhibit 1 – Kano Analysis
Based on Kano Analysis the project team conducted additional VOC sessions with the key stakeholders and the produced the following Critical to Quality (CTQ) measures with Upper Specification Limits (USL), Lower Specification Limits (LSL) and Targets.
Exhibit 2 – CTQ Matrix
Phase Two – Measure
The purpose of the Measure Phase was to define the requirements for the Maturity Measurement Assessment Process and Tools.
The key decisions for the Measure Phase were:
- To use the Structured CMMI® Appraisal Method for Process Improvement (SCAMPI®) as the basis for the To-Be Model since there was experience within the IT Group with using CMM®/CMMI®.
- The To-Be Model would include process steps to create a framework that further assessment methods could be developed for areas such as Project Portfolio Management (PPM), Project Financials, Resource Management, Project Management Information Systems and Project Risk Management.
- The To-Be Model would be a 2 phase approach based on both CMMI® and OPM3®. It would use OPM3® Self Assessment Method (SAM) Project Level Questions for the subjective level assessment and SCAMPI® inspired objective level assessment.
The key deliverables for the Measure Phase were:
- Level 1 Process Map and SIPOC based on the review, decomposition and selection of the best of SCAMPI® and OPM3®.
Exhibit 3 – Level 1 Process Map
- A Second House of Quality (HOQ) to prioritize the work on the process steps, with the CTQs down the left side and the Level 1 Process Steps across the top.
Exhibit 4 – QFD2
- A Pugh Matrix to validate the project team's design concepts of using the available assessment methodologies with the conclusion that the hybrid approach of SCAMPI® Continuous and OPM3® was a more effective process design concept than using SCAMPI® or OPM3® stand alone.
Exhibit 5 – Pugh Matrix
Phase Three – Analyze
The purpose of the Analyze Phase was to design the components of the Maturity Assessment Process and Tools.
The key deliverables for the Analyze Phase were:
- Design to represent the maturity measurement translation between CMMI® Continuous Model and OPM3® Standardize, Measure, Control and Improve (SCMI) Model
- A CMMI® Key Process Area (KPA) to OPM3® Best Practice Mapping Template including; a.) CMMI® KPA Listing, b.) PMBOK® Guide Knowledge Area Listing, c.) PMBOK® Guide Knowledge Area to CMMI® KPA and Specific Practice, d.) PMBOK® Guide Phase, Process to CMMI® KPA and Specific Practice and e.) OPM3® Best Practice, SAM Question and PMBOK® Guide Process to CMMI® KPA and Specific Practice.
- A Subjective Questionnaire Template using the OPM3® SAM Project Questions with the addition of a Confidence Level Indicator.
- Design of the Comprehensive (Objective) Tool to verify adherence and validate the Subjective assessment through the use of H2O, Hear it twice and Observe it once.
- Develop mechanism to score Level 2 Capabilities
- Design of the Assessment Reports and Outputs
- Creation of the Communication, Education and Deployment Plans
Phase Four – Improve
The end of the Analyze Phase was the official conclusion to achieve DFSS Green Belt certification, but the project team decided to continue with the Improve Phase to build and pilot the Maturity Assessment Process and Tools.
The key deliverables for the Improve Phase were:
- CMMI® Continuous Model and OPM3® SCMI Model Capability Comparison
Exhibit 6 – Capability Level Comparison
- CMMI® KPA to OPM3® BP Mapping
Exhibit 7 – Mapping Examples
- The Subjective Questionnaire and Scoring Matrix
Exhibit 8 – Subjective Questionnaire Example and Scoring Matrix
- The Comprehensive Assessment Tool. The tool provided the PMBOK® Phase, KP and associated artifacts that should contain that information. The project team looked at the artifacts and validated them against the CMMI® Level 2 Capability Requirements. Each identified Level 2 Capability was given a score of 0.10 and if all capabilities were achieved the score would be 2.0.
Exhibit 9 – Comprehensive Assessment Tool Example
A Project Management Executive, who was also a member of the stakeholder group and a key proponent of the project, requested that his area be the pilot. He had developed his own theory of where his team was on the maturity continuum and needed to validate it against what his own managers and team members thought.
The project team conducted both the Subjective and Objective Assessments over a 5 day period and following are the findings.
Exhibit 10 – Pilot Assessment Findings
The assessment validated the Project Management Executive's theory that his team thought they were more mature than in reality. Based upon this pilot, an improvement program was integrated with an in-flight Project Manager Development Program to address the gaps.
Sample Additional Assessments
Financial Services Project Management Group
Exhibit 11 – Sample Assessment Findings 1
Based on the findings from the Subjective Assessment, the following improvement initiatives were initiated.
- Project Management Processes, Tools and Techniques – Generation One
1.1. Developed by team of practicing Project Managers
1.2. Created Six Sigma Improvement Path linked to the Enterprise Methodology
- Basic Project Management/Six Sigma Training
2.1. Training Developed by team of practicing Project Managers
2.2. Training conducted by Processes, Tools and Techniques Development Team and Senior Managers
- PMO Management by Fact (MBF) Program
3.1. PMO Process Excellence Project – Enterprise certified in Bronze, Silver and Gold
3.1.1. Resource Management
3.2. Project Management Process, Tools and Techniques Black Belt Project to achieve Level 3
3.3. PMO Black Belt Project
3.3.1. Portfolio Management Implementation Green Belt
3.3.2. Resource Management Integration Green Belt
3.3.3. `Project Planning, Tracking and Reporting Green Belt
Global Provider of Healthcare Products and Services
Exhibit 12 – Sample Assessment Findings 2
Based on the findings from the Subjective Assessment the following improvement initiatives were initiated.
- Process Development and Documentation for these areas:
1.1. Scope, Estimation and Schedule Development
1.2. Project Resource Management
1.3. Scope Verification and Deliverable Acceptance
1.4. Monitoring and Controlling
1.5. Project Communications
1.6. Project Financial Management
1.7. Project Closure
- Policies and Procedures for these areas:
2.1. Organizational Readiness
2.2. Project Manager On-Boarding
2.3. Project Escalation Synopsis
2.4. Project Knowledge Repository
2.5. Project Quality Assurance
- Training on Processes, Policies and Procedures
- Conducted Project QA Audits on 42 projects in the portfolio. QA findings indicated a 27.4% improvement in understanding and adherence to the Processes, Policies and Procedures.
No matter which Project Management Maturity approach your organization undertakes the goal should be transforming the organization and it will take hard work, senior executive sponsorship and dedication. If your organization's goal is to be assessed or “certified” at certain level and you have the best intentions of transforming the organizations, please be very careful. This is not only about process improvement, but also people improvement and if you do not focus on the people involved with the processes your chances of success will be limited and possibly fail. In short, follow these simply steps for success; a.) Anticipate – do your research and prepare an in-depth organizational change impact analysis on doing a Project Management Maturity Initiative and what your specific areas and goals are for this effort, b.) Adopt – pick a model or models if you want to experiment and use them, c.) Adapt – make the maturity assessment yours and adapt it to your organization and its culture and d.) Achieve – put together achievable roadmaps for improvement to keep the momentum going. Remember, this is not a day trip, but a journey.
Project Management Institute, (2000), A Guide to the Project Management Body of Knowledge (PMBOK®) (2000 edition.), Newtown Square PA: Project Management Institute.
Project Management Institute, (2003), Organizational Project Management Maturity Model Knowledge Foundation (1st edition), Newtown Square PA: Project Management Institute.
Software Engineering Institute, (2002), CMMI® for Systems Engineering, Software Engineering, Integrated Product and Process Development and Supplier Sourcing (CMM-SE/SW/IPPD/SS, V1.1) Continuous Representation CMU/SEI-2002-TR-011 (version 1.1), Pittsburgh, PA: Software Engineering Institute.
Software Engineering Institute (2001), Standard CMMI® Appraisal Method for Process Improvement (SCAMPI®), Version 1.1: Method Definition Document (CMU/SEI-2001-HB-001), Pittsburgh, PA: Software Engineering Institute
Chrissis, M. B., Konrad, M., Shrum, S. (2003) CMMI® Guidelines for Process Integration and Product Improvement. Pearson Education, Inc. Boston, MA: Addison-Wesley
PMI®, OPM3® and PMBOK® are registered trademarks of the Project Management Institute, registered in the United States and other countries.
SEI®, CMM®, CMMI® and SCAMPI® are registered trademarks of the Software Engineering Institute, registered in the United States and other countries.
ITIL® is a registered trademark of Office of Governance Commerce, Government of the United Kingdom.
© 2007, Brad Clark, PMP
Originally published as a part of 2007 PMI Global Congress Proceedings – Atlanta, Georgia, USA