Project management maturity case study
what you can do before OPM3™
This paper will address the following questions:
- □ Where do the concepts on project management and process maturity come from?
- □ How is maturity defined?
- □ What are the arguments in favour of becoming mature?
- □ How should you measure and plan for maturity development?
- This is illustrated with a case study supported by a proprietary process and tool for maturity evaluation and planning
- □ What are the challenges in engaging the interest of the organization at the correct level of management?
- □ What role can Project Managers take in leading the way to organizational maturity
- How they can get the message across?
After over six years of discussion and preparation, PMI®'s Organizational Project Management Maturity Model (OPM3™) was published in 2003.
Although the model is applicable to all industries, a large number of organizations will feel overawed by its complexity and terminology. This paper describes a model, and the supporting tool, that provide an easy introduction to maturity: this has been named the “Cultural Project Management Effectiveness Model” (CPMEM®).
Based on the application of the CPMEM to a real-life organization, the paper explains the way in which the attitudes and approaches of an organization have differing impacts on the effectiveness with which several Knowledge Areas are applied. The corresponding options for enhancement are described and the various scenarios are analyzed using the same tool to show how each option would affect the overall outcome, if correctly applied.
The paper concludes with an analysis of how CPMEM can be used in a number of situations: the analysis of a single project, the analysis of a project management environment, and as a stepping-stone towards a full OPM3 assessment and maturity improvement cycle.
As the need for measurable improvement in project management has become more widely recognized, a number of different maturity models have appeared on the market. It is not the intention of this paper to carry out an in-depth comparison of these models, but rather to extract the defining characteristics that determine their range of applicability, ease of use and potential appeal to business management, so as to explain the reasons behind the approach used in the case study.
There is a growing literature on maturity models, stretching back to the original Quality Management publications (Deming, 1986; Juran 1988). This now embraces the concepts of “best practice” in specific technical domains: the most widely-quoted of such models has now evolved into the Capability Maturity Model (Integrated) - CMMI® - (Chrissis et al., 2004), and, within the last year, PMI's Organizational Project Management Maturity Model (OPM3™) has been released (Project Management Institute, 2003). CMMI® started with a captive audience, due to the fact that it was developed by Carnegie-Mellon University for the US government and became a mandatory requirement from the Department of Defence in a number of procurement domains. OPM3, on the other hand, has been developed directly from within the project management profession and still needs to prove its worth and become recognized as a “must-have” building-block of any successful business environment.
Each of the models provides a mechanism by which to measure the level of maturity of an organization. The CMMI appraisal is designed and structured so as to assess the answer to the following question: are you addressing the required goals, and are you doing this to a defined level of capability? Many other models (e.g. Kerzner, 2001) use a similar approach - i.e. to ask: are you trying to do the right thing, and how good are you at it?
Model Philosophy – Pushing and Pulling
Up until the appearance of OPM3, models were based on a “push” approach to increasing quality: “Enforce the right rules, to avoid the wrong outcome”. OPM3 and subsequent models such as CPMEM have adopted the “pull” strategy: “To ensure the right result, make things possible and expected”. One other main difference is in the way in which the models support improvement planning:
- □ In CMMI, the relationship between answers and the maturity rating is direct and explicit, so that the action plan can be roughly based on the question “how can we score better on each of the questions that are holding us back”?
- □ In OPM3, the relationship is more subtle since the questions map onto a set of Best Practices and the resulting score relates to these. An additional tool is provided which allows the organization to assess how to enhance the scores for any of these Best Practices by developing missing Capabilities. OPM3 is therefore more explicitly action-directed.
Each approach has its strengths and weaknesses, as explained below.
Marketing the Idea of Maturity
What is Maturity?
Capability Maturity Models (CMMs) focus on improving processes in an organization. Increasing maturity is a sign of increased control, predictability, measurement and effectiveness of all of the processes. For CMMI and PMMM (Crawford, 2002) maturity depends on implementing and executing a set of processes to a given level of capability.
For the PMGS Cultural Project Management Effectiveness Model (CPMEM®), (PMGS, 2003), cultural project management maturity is a measure of how well an organization provides all of the support (technical, physical, intellectual, emotional, etc.) in an integrated way so as to ensure that all projects have an optimal chance of meeting their expectations in a predictable and repeatable manner. Process maturity is just one factor of this cultural project management effectiveness.
The OPM3 definition of maturity is the level of an organization's ability to implement organization strategy through the successful, consistent, and predictable delivery of projects. It explicitly integrates the concept of increasing process capability into its model.
For CMMI, one obvious motivation is “in order to sell to the DOD”! In addition, there is the well-justified feeling of “if many other people are using it, it must be good for me” – which of course then leads to a snowball effect. This tends to mean that management realizes that it is mandatory and empowers the technical delivery areas to adopt it.
The ProjectFramework™ from ESI (ESI, 1999) provides a model and assessment tool by which a Project Management Office or similar project management level structure can assess its own effectiveness with respect to the concepts and processes of the PMBOK® Guide. It does not easily reach the parts of the organization on which strategic decisions depend.
The situation is different for OPM3: in this case, there is no captive customer for the product. Whereas the areas of the business that are involved day-to-day in project management are likely to be strong supporters of the concepts, the value – or even the need – for such a model is much less obvious to organizational management. OPM3 preaches to the converted in their language but, if it is to be adopted, it requires the active engagement of management – who are likely to consider it “technobabble” and not waste time or money on it.
What is therefore missing is a bridge between the well-defined professional processes as defined in the PMBOK® Guide and the way in which the organization is managed from the top. The CPMEM is targeted as a response to this need: the way in which it bridges the gap is explained below.
CPMEM®: An Articulate Model
The aim of the CPMEM approach is to map the way management behaves onto the way projects perform in their organization. It should therefore ask questions in the language that management understands and provide the results of the analysis in manner that reflects project management concerns. For this reason, the input side – i.e. the information-gathering process of the CPMEM – is open-ended so as to accept updates to tailor it to the specific management environment of the organization being analyzed, whereas the PMBOK® Guide provides a fixed reference for the analytical output.
Articulating the CPMEM model
Just as with PMI®'s OPM3 model, and in contrast with many other models (SEI's CMMI, etc.), CPMEM® does not stipulate specific levels of maturity, capability or effectiveness that need to be achieved. The CPMEM® approach is to provide the client with a rating in each of the 9 Knowledge Areas, so that the client can decide the degree of improvement they actually require in their specific environment. The subsequent maturity planning can then also make use of CMMI and OPM3 – especially in the areas of process capability improvement. For CPMEM, progress is measured on a continuum rather than in discrete steps, thereby allowing the organizations greater flexibility in setting its targets.
Project management culture is considered to be composed of a number of “cultural elements”, which can be clustered or grouped into “cultural areas” for ease of classification. The effectiveness with which a client implements each cultural element can be analysed by means of a questionnaire which, to make it more manageable, is structured according to cultural areas. Each cultural element has an impact to a greater or lesser degree on the effectiveness with which each Knowledge Area (KA) from the PMBOK® Guide is implemented.
The technical core of the CPMEM model is a system for mapping the responses to the culture-related questions onto the Knowledge areas and the Tools of the PMBOK® Guide, as well as providing the reverse mapping, from the results onto the culture-related questions. This is shown schematically in Exhibit 1.
Exhibit 1: Mapping Cultural Elements onto Knowledge Areas and Tools
The CPMEM® consultancy engagements (PM-AWARE®, PM-ALERT®), with their support tool, start with a question and answer session on how an organization envisages and performs their projects, and delivers a report on the level of effectiveness with which each Knowledge Area is addressed within the organization. In contrast with most other maturity models, CPMEM allows any value in the range 0-100% rather than a binary Yes/No reply to most of the questions. This allows a more realistic and accurate evaluation of status and progress. The set of responses then forms the basis for a scenario analysis of the options for enhancing the organization's project management effectiveness, followed by the creation of an action plan.
The steps in the PM-ALERT approach correspond to its name:
The output of this phase is a filled-in set of questionnaires
- Result-oriented meeting with key members of the client organization in order to determine their priorities in the management of projects.
- Understand how they view and support projects.
The output of this phase a “populated” tool, based on the questionnaire and discussion on the output of the tool
- Identify the areas which the organization considers to be either key to success or important for avoiding failure.
- Identify the areas in which the organization is the least effective and should therefore focus on.
The output of this phase is an agreed list of changes to one or more cultural project management elements
- Translate the Evaluation into the list of cultural project management elements on which to focus.
- Evaluate the overall improvement those potential changes would make to the overall cultural project.
The output of this phase is a set of successful “cultural project management” change projects
- Define, plan and execute the project carry out the agreed actions, in order to enhance the client's cultural project management effectiveness.
The CPMEM® questionnaire has been completed for the informatics organization of a large, multinational company. This was carried out with the help of this organization's project quality department.
The assessment is carried out in a set of cumulative steps:
1) Assessment of which aspects of project management effectiveness are the most important from the organization's point of view. This is based on a brief questionnaire on organizational aspects that can most influence a project's chances of success (see Exhibit 2); this then provides a list of priorities or “weights” for the Knowledge Areas from the PMBOK® Guide.
Exhibit 2: First few questions of the “weight setting” assessment
➣ For this case study, as shown in figure 3, the answers from the importance assessment have been mapped into the PMBOK® Guide Knowledge Areas, the corresponding weights have been calculated and plotted on a radar diagram. In this case, it is seen that the area that the organization considers most important to ensure project success is effective scope management. However, at this point is the organizational review, we do not have any indication as to whether or not they actually are effective in the area!
➣ Note also that a similar assessment is carried out with respect to the organizational aspects that can most damage a project's chances. For the case study, it was found that the organization recognized that poor risk management would be their major concern.
Exhibit 3: Results of the weight-setting assessment: scope management is viewed as the highest priority for success, risk management is the greatest protection against failure
2) Assessment of the organization's approach to, and support for project management. This is carried out with a comprehensive questionnaire (see Exhibit 4) that is split into a number of management focus areas (“cultural areas”) such as project planning, managing uncertainty, project tracking, external relationships, finance, people, use of tools.
Exhibit 4: Sample of Life Cycle assessment questions
3) The answers are then mapped onto the Knowledge Areas and these results adjusted based on the weights from the previous step. The results are plotted on a radar diagram (Exhibit 5). For the case study, the areas of risk and scope are those with the highest scores, with risk marginally the higher. This implies that, although in the previous step, management recognized the importance of these two areas, the way in which projects are carried out does not reflect this concern – and this ineffectiveness is a result of those management actions and attitudes that have been investigated through the questionnaire. This type of conclusion is the basis on which a strong argument for culture change can be based!
Exhibit 5: Weighted analysis of assessment responses: risk is seen as the most critical KA
4) Evaluation of the results of the previous steps, in two complementary ways:
□ by reviewing the questions (and their answers) that have the greatest effect on the Knowledge Area that has been assessed as the most critical and
□ by reviewing the way in which the tools are used that have the greatest effect.
➣ For the case study, the management-related questions with the greatest potential to increase the risk-related scores are shown in figure 6. It can be seen that the following actions would have the greatest impact
□ Actively managing the planned risk responses – just how many organizations do we know that reduce Deming's “plan-do-check-act” cycle to a set of “plan-discard-complain-accept” crises!
□ It should also be noted that these areas for improvement may well be in categories seemingly unconnected with the specified knowledge areas: you will see that questions relating to people, external suppliers and quality have a strong impact on the effectiveness of project risk management
➣ The evaluation of the responses to the use-of-tools questions in the case study gave rise to the priority list in Exhibit 6.
Exhibit 6: Priority list of tools to address, with their current level of usage
□ It can be seen that sensitivity analysis is not used at all (score = 0) and that this could therefore be a useful area for training and awareness in the project teams.
□ Similarly, in third position, Earned Value (not a specific risk management tool) is used some of the time (score = 2) but that a more consistent use would lead to an effective reduction in uncertainty
5) in this way the evaluation immediately reveals the management actions which would have the most worthwhile effect on the effectiveness of projects as vehicles for strategic effectiveness.
These tables and charts allow a fact-based discussion with management, and the evaluation tool allows successive “what-if” iterations to be carried out. For example, in the case study, it became clear that some functions involved in projects did not report their time – thereby making the “tools” option of extending the use of Earned Value premature; other options were therefore examined to identify how to gain an equivalent advantage without resorting to this technique; in this case a training in the general use of the risk management tools was developed. However, it was also agreed that a feasibility analysis project would be launched in order to investigate the viability of extending the use of Earned Value for projects in the informatics division. One other very positive outcome of the discussion with management was the immediate addition of an executive risk owner for all major projects, in order to address the issue shown up by question 13.1 in Exhibit 7.
Exhibit 7: Priority list o cultural element of address, with their current scores
Conclusion: Maturing into Maturity
A great majority of people within the project management profession recognize that their work would be easier, more valuable and more worthwhile if the overall environment in which they worked were more closely aligned to the needs of managing projects. However, management needs to be convinced that there is something in the concept of project management maturity that can answer their day-to-day needs. It has been shown in the case study that questions based on the language of the organizational managers can serve to provide assessments which are meaningful to managers in both the business and project domains.
What also needs to be taken into account when communicating with management is that organizations have different levels of readiness:
- □ Readiness to listen: there is a level below which project management is, at best, not even considered. Whereas in CMMI, level zero is rather sensitively described as “not yet level 1”, a brief discussion on reallife difficulties in the profession will bring up convincing arguments that level zero would be better described as “not nearly as bad as it could be”, since there are hidden levels below zero that need to be addressed before any form of structured improvement can be envisaged (Piney, 2000);
- □ Readiness to be measured: in general, before people will agree to be measured, they must want to improve and believe they can improve; measurement should address only those areas of potential improvement, and not more. There is a simplified form of the organizational assessment described in the case study can provide project-by-project measurement of PMBOK®-Guide-related effectiveness both during and at the end of projects based on typical project status-review questions (“PM-AWARE® service” – PMGS, 2003). This relates the measurement to the everyday work of the project manager to the Knowledge Areas and delivers practical guidance on areas of strength and weakness;
- □ Management identification: if you expect management to listen to talk about “maturity”, you need to use the language that management talks. In this way management can see a direct correlation between their behaviour and increasing project management success. The case study has shown that this it is possible to ask the questions in a way that management associates with, as well as evaluating the answers in a way that allows direct action planning for focussed improvement directly relevant to the project environment;
- □ Confidence in the outcome: you can only hope for an organization to base strategic initiatives on projects and aim for increased organizational project management maturity in line with OPM3, once project management has been shown to be consistently effective and can therefore be reliably considered to be a strategic asset;
- □ Recognition of strategic value: management values and develops strategic assets. Once management considers projects in their organization as strategic assets, OPM3 will be seen as the tool of choice by which to plan this development. However, as explained above, you may need to work your way up to OPM3 by using less sophisticated tools in order to develop this understanding.
One of the barriers to acceptance of project management maturity at executive management level has been that it has been promoted by project managers with a focus on their own needs! This is not surprising, since project managers are the first people to suffer from the effects of lack of maturity, so it is project managers in general -along with the excellent teams of volunteers within PMI - who have developed and promoted these ideas. What is now required is for project managers to change their approach and language in promoting these ideas: it should no longer be “excuse me, please, management, but I need this to do my job”, it should be “please be aware, management, that here is a management tool for strategic advantage that you could apply in order to make yourselves increasingly successful”.
This change may go hand-in-hand with a new management paradigm: after Theory X and Theory Y where the view of management with respect to their employees is seen to drive leadership styles (McGregor, 1960), and Theory Z, where the social context is considered all-important (Ouchi, 1981), OPM3 will help deliver the era of a Theory Ω in which Management recognizes the key role that their organization's approach, capabilities and culture have on promoting best practices and business success.
Therefore, do not tell them what maturity can do for you – rather explain what maturity can do for them! And explain it in a way with which they can identify.
Ahern D.M., Clouse A., Turner R. (2003) CMMI® distilled. New York: Addison Wesley.
Chrissis, M.B., Konrad, M., Shrum, S. (2004) CMMI® guidelines for process Integration and Product Improvement. New York: Addison Wesley.
Crawford J.K. (2002). Project management maturity model. NY: Marcel Drekker, Inc.
Deming, W. Edwards. (1986). Out of the crisis. MA:MIT Center for Advanced Engineering.
ESI. (1999). Project Framework. VA: ESI, Arlington.
Juran, J.M, (1988) Juran on planning for quality. New York: Macmillan.
Kerzner, H. (2001). Strategic planning for project management using a project management maturitymodel. NY: John Wiley.
McGregor, D. (1960). The human side of enterprise. NY: McGraw-Hill.
Ouchi, W. (1981). Theory Z. How american management can meet the japanese challenge. NY: Perseus Publishing.
Piney, C. (2001). Project management immaturity model. Available from firstname.lastname@example.org.
PMGS. (2003). CPMEM® White Paper. PMGS, 16, rue Ampère, 95300 Pontoise, France. Available from email@example.com
PMI (2000). A guide to the project management body of knowledge (PMBOK® guide). PA: Project Management Institute, Four Campus Blvd, PA.
Project Management Institute (2003) Organizational project management maturity model – knowledge f/oundation (OPM3™). PA: Project Management Institute.
©2004, Crispin (Kik) Piney
Originally published as part of 2004 PMI Global Congress Proceedings – Anaheim California