Introduction
There is a growing number of “maturity models” being provided to organizations, either directly or indirectly, to assist with the assessment of how “mature” an organization is (Cooke-Davies, 2004a). At the same time, there is an intense interest inside organizations in the topic of how best to “measure” project performance, particularly on the part of those concerned with governance, portfolio management, and enterprise-wide project management (Egberding & Cooke-Davies, Unpublished). Thus, in theory at least, it should be possible to assess how “mature” a project-based organization is by looking at a combination of what aspects of project performance or project management practice it measures, and what the results of those measurements show.
Unfortunately, before an organization’s suite of metrics can show stakeholders how mature the organization is, there are a number of significant problems to be overcome. Four of them in particular are worthy of mention:
- There is no generally agreed definition of what a “mature project-based organization” looks like. Different “maturity models” embody both different concepts and different suggestions as to the route to maturity (Cooke-Davies, Schlichter, & Bredillet, 2001)
- The whole field of “capability and maturity studies” is a semantic minefield – with specific technical meanings for certain words being very different from their normally accepted breadth of use in common speech. (Cooke-Davies, 2004a)
- The question of “project success” is hugely complex, and although there is general agreement on some points, there is wide disagreement on others. (Morris & Hough, 1987; Baker, Murphy, & Fisher, 1988; Pinto & Slevin, 1988; De Wit, 1988; Crawford, 2000; Cooke-Davies, 2004b)
- Both measures and maturity models are based on an implicit “theory of projects” that may not be the most appropriate one (Melgrati & Damiani 2002).
The linked questions of maturity and measurement, however, are important to project managers employed in major organizations, since many of their employers are taking the axe to project management departments through downsizing, outsourcing, elimination or dispersion to business units. The communications gulf between the boardroom and project management departments has never been wider or deeper (Thomas, Delisle, & Jugdev, 2002).
What this means is that despite these complications and uncertainties, it is important for project management research to shed such light as it can on both aspects of the two inter-related questions: whether there is a set of measures that can show how mature a project-based organization is; and what benefits such “maturity” might confer on its stakeholders. Since there are fragmentary results from many different pieces of research that provide clues to this complex question, but none that can provide either a complete answer or even certainty about whether such an answer is possible, this paper will draw widely on different pieces of research in order to answer the question, “What are the relevant questions about maturity and metrics for a project-based organization to answer, and what are the implications for project management research?”
Four relevant questions
Perhaps the most helpful way of discussing the topic is by examining each of the relevant questions in turn, and reviewing both what research has to offer on the subject, and what questions still remain. There are four such questions that this paper will examine:
- What is meant by the term “maturity,” in the context of organizational project management?
- Does “maturity” mean the same thing in different industries and for different kinds of projects?
- Is “maturity” an asset to project-based organizations? If so, how?
- How can “maturity” be assessed? What aspects of projects and their management should be measured?
1. What is meant by "maturity"?
In general usage, maturity means fully developed, or perfected.
In Collins Dictionary, the adjective “mature,” from which the noun “maturity” is derived, has a number of different meanings in common usage. It can, for example, mean (1) fully-developed or grown up; (2) of plans or theories, it can mean that they are fully considered or perfected; (3) of insurance policies or bills, it can mean due or payable; and (4) of fruit, wine or cheese, it can mean ripe or fully aged.
The last two of these do not offer any obvious link to the world of project management (unless it is through projects that are described as “pear-shaped” when they are in serious trouble), so it is in either meaning (1) “fully developed” or (2) “perfected” that the word is used in the term “maturity models.”
The two could almost sound synonymous, but in fact they embody both differences and similarities. They are different insofar as the former is based on an organic metaphor – the idea that, in the course of time, biological processes will unfold and bring to fruition a design that is “enfolded” in the genetic structure of the organism that is maturing. The latter, on the other hand, implies an external designer or thinker, who is outside of the system being designed, but is capable of assessing the extent to which they are “fit for purpose” – meaning, of course, the purposes of the designer or thinker.
They are similar in that each makes the assumption that a “perfected end-state” exists, whether it “unfolds from within” or is “designed from without.”
This assumption has been made explicit in the development of “capability-maturity” models, which are very obviously founded on a control/systems epistemology, and which provide the wellspring from which project management maturity models have flowed.
In capability-maturity models, the term maturity has a special technical meaning.
The family of capability-maturity models has been developed by the Software Engineering Institute of Carnegie-Mellon University under the original leadership of Watts Humphreys (Paulk, Curtis, Chrissis, & Weber, 1996). Drawing heavily on the concept that every process has a natural “capability” that can be assessed using statistical process control, the original software model embodied a simple principle: if organizations wish to develop predictability and repeatability in their Information Systems/Information Technology (IS/IT) production processes, then they need to develop a number of process areas, each of which consists of families of related processes. In turn, each of these processes needs to develop through a series of stages of maturity from informal at the lower end of the scale to highly routinized and with continuous improvement embedded at the higher end. As each process develops in this way, its “capability” will improve. To prevent the model from becoming excessively complex to understand, the process areas and process maturity stages are combined into a series of five levels of organizational maturity, into one of which any organization can be categorized.
Thus maturity is used in capability-maturity models in the very technical sense to mean “the extent to which an organization has explicitly and consistently deployed processes that are documented, managed, measured, controlled, and continually improved. Organizational maturity may be measured via appraisals.” (CMMI Product Team, 2002, p. 582).
Implicit in the model is the assumption that there is a known and predictable group of processes that represent the “perfected end-state” for any organization that wishes to develop new products (in the case of CMMI) effectively and efficiently. A second inherent assumption is that these processes operate predictably as a controllable “system” that is capable of delivering the outputs (new products in the case of CMMI) reliably and predictably. Put another way, the assumption is that the “designers” and “managers” of the system are free to design the ideal system, and the role of everyone else is to operate within the processes and workgroup practices that together make up the system.
The relationship between “capability” and “maturity” in these technical models is worth exploring, since “capability” is also a word that has a specific technical meaning that differs from its use in common speech. The technical meaning has its roots in the Quality movement, and can be traced to the writings of authors such as Walter Shewhart and J. Edwards Deming. The principle is simple: “a stable process … is said to be in statistical control …. A system that is in statistical control has a definable identity and a definable capability.” (Deming, 1986, p. 321, italics in original text). Thus the efforts of quality improvement are, first of all, to make the process stable and thus bring it under statistical control, and then to work on improving the capability of the process. A process can thus be said to mature as it passes through the stages from unstable, to stable, and then to enjoying improved capability.
This has been embedded in capability-maturity models through the concept of a capability level – an attribute of a process area – and the overall maturity level of an organization is assessed by considering which process areas achieve which capability level. As an organization increases in its maturity, so it will implement additional process areas and improve the capability level of each of them. Each successive maturity level includes process areas that do not feature in lower levels.
Thus, in the field of capability-maturity models it IS possible to assess the maturity of an organization by a combination of what process areas are measured (indicating the maturity level that is being striven for) and what level of capability is achieved by each (indicating the capability level of each process area). Thus, for example, if an organization is measuring its practices and processes for project integration management, it is clearly seeking to operate at maturity level 3 (using CMMI), whereas if it is simply measuring its practices and processes for project planning, monitoring and control, it is seeking to operate at maturity level 2.
It is perhaps unfortunate that the terms used to define specific “capability levels” (incomplete, performed, managed, defined, quantitatively managed & optimizing) are nearly identical to those used to define specific maturity levels (initial, managed, defined, quantitatively managed & optimizing). This has given rise to some confusion in the area of project management maturity models, as will be seen in the next section.
In the field of project management, maturity is being used with a number of different meanings.
Given the role that project management plays in the development of software and of new products, it isn’t surprising that many of the concepts of maturity that are incorporated in capability-maturity models are imported wholesale into the realm of project management maturity models.
The two models that have received the greatest attention in the research literature so far have been the Berkeley PM Process Maturity Model (e.g., (Ibbs & Kwak, 1997; Ibbs & Kwak, 2000; Kwak & Ibbs, 2000; Ibbs & Reginato, 2002) and the PM Solutions Project Management Maturity Model (e.g.,(Burns & Crawford, 2002; Pennypacker, 2002; Pennypacker & Grant, 2003). Like other project management maturity models, each of these assesses the maturity of processes derived from PMI’s A Guide to the Project Management Body of Knowledge (PMBOK® Guide) areas, using a scale of maturity that combines and blurs the distinction between “capability levels” and “maturity levels.”
They thus imply that organizations, regardless of their maturity, will each be measuring the same things (performance of the same group of processes), but that what will distinguish the maturity of an organization is the score that is revealed by the measurement.
Other project management maturity models follow more closely the principle of adding incremental process areas as an organization increases in maturity (e.g., (Kerzner, 2001; Hillson, 2001; Gareis, 2001; Office of Government Commerce, 2002), although these inevitably develop their own definitions of the process areas that relate to each maturity level.
OPM3™ – PMI’s new organizational project management maturity model – adopts its own technical meanings for maturity and capability.
No discussion of organizational project management maturity would be complete, without mention of OPM3, PMI’s organizational project management maturity model.
OPM3 defines organizational project management maturity as “the extent to which an organization practices organizational project management. In the OPM3 maturity model, this is reflected by the combination of ‘Best Practices’ achieved within the Project, Program and Portfolio domains.” (Project Management Institute, 2003, p. 173) – a definition that has something in common with capability-maturity models, but recognizes that it is applied to the field of project management.
In OPM3, however, the word “capability” is used somewhat differently. In OPM3, “a capability is a specific competency that must exist in an organization in order for it to execute project management processes and deliver project management services and products. Capabilities are incremental steps leading up to one or more Best Practices.” (Project Management Institute, 2003, p. 171).
Thus, in common with the Berkeley and PM Solutions models mentioned above, the same measures are used by organizations at all levels of maturity, and it is the scores obtained that indicate how mature an organization is.
Implications
The definition of maturity in the capability-maturity family of models leads to the clear conclusion that more mature organizations measure different things than immature ones, and can also expect the measures to show improving results as the organization increases in maturity. The definition of maturity in many of the more popular project management maturity models, however, does not make this distinction. The same things are measured at all levels of maturity; it is simply the results that improve with maturity.
2. Does “maturity” mean the same thing in different industries and for different kinds of projects?
Some industries have been practicing project management for longer than others.
Modern project management has its roots in the second world war, and developed in a limited number of engineering-based industries during the 1950s, 1960s and 1970s (Morris, 1994). As such, it might be reasonable to expect that these “industries of origin” might have developed a more advanced model of project management than industries such as Pharmaceutical Research and Development, which adopted project management disciplines and practices somewhat later.
There are differences between the practice of managing projects in different industries
A recent study of how practices vary among project-based enterprises in four different industries (construction, IT, defence and pharmaceutical Research & Development (R&D)) showed that while there are clearly common practices, the environments in which they are deployed are significantly different. This, in turn, leads to a difference in the way project management topics are handled in the different industries (Morris, 2003).
Comparisons have been made between the project management maturity in different industries using the Berkeley and PM Solutions models. Using the Berkeley model, Ibbs and Kwak (Ibbs & Kwak, 1997; Ibbs & Kwak, 2000: (Kwak & Ibbs, 2000) concluded that there were minor differences in maturity between the most mature (Engineering construction) and least mature (Information systems) sectors. Using the PM Solutions model and a different methodology, however, Pennypacker and Grant (Pennypacker & Grant, 2003) concluded that the differences in maturity among different industry sectors were not significant.
A separate study conducted on behalf of a number of the world’s leading pharmaceutical research and development companies found differences between the way project management was practiced in different industries. The most highly developed project management models (which might be said to equate to a measure of project management maturity, although the methodology did not involve a maturity model) were found in the Petrochemical and Defence industries. Other industries (Pharmaceutical R&D, Construction, Telecommunications, and Financial Services) displayed some interesting differences in practice, but did not score as highly as the two leading industries (Cooke-Davies & Arzymanow, 2002).
The PMBOK Guide seeks to describe those project management practices that generally apply in most contexts.
Although PMI’s PMBOK Guide describes those project management practices that are “applicable to most projects most of the time. [This] does not mean that the knowledge and practices described are or should be applied uniformly on all projects; the project management team is always responsible for determining what is appropriate for any given project.” (Project Management Institute, 2000, p. 3). It explicitly recognizes that project management is applied in different “application areas,” each of which may have different characteristics, and which might be expected to influence project management maturity.
Other “bodies of knowledge,” many of which encompass a broader range of knowledge and practices than PMBOK Guide (Morris, 2000), similarly recognize the importance of context to the particular way in which projects are managed.
OPM3, for example, recognizes the additional and related areas of program management and portfolio management. It would seem to follow that a mature “supplier organization,” undertaking projects for external clients and seeking thereby to make profits, will have different considerations for the management of its project portfolio (e.g., what bids have we been successful in winning?) than a mature “product development organization” undertaking projects to develop new products that it can exploit through its marketing and manufacturing departments (e.g., where should we allocate our scarce resources for maximum benefit?).
Different types of project seem to call for different project management practices.
Aside from different industry contexts, there are many different types of project that are undertaken. A large segment of the literature has concentrated on specific types of projects, such as construction. (Miller & Hobbs, 2000), engineering. (Cooper, 1994), new product development (Cooper, Edgett, & Kleinschmidt, 2001), or software (Hartman & Ashrafi, 2002).
The question persists as to how meaningful it is when discussing the maturity of the practices of managing projects to ignore such apparently significant differences.
Interestingly, in a report describing “value-enhancing practices in the construction industry” (ECI Benchmarking Steering Committee, 1999) alongside some industry-independent practices such as pre-project planning, team building, and project change management, there are others that have a definite “engineering construction” flavor about them, including strategic alliances and design/information management, and a couple that relate explicitly to the construction industry – constructability and safety.
Implications
As more organizations wrestle with the concept of maturity, there is a challenge to the research community to broaden its vision from those aspects of managing projects that are common to “most projects, most of the time” to encompass an understanding of how maturity varies among industries and among project types.
3. Is “maturity” an asset to project-based organizations? If so, how?
This paper has so far suggested that more work is needed on the definition of maturity, and on understanding how it varies among industries and project types. Such work will be of value, however, only if organizations perceive maturity to be an asset. So the answer to this third question is possibly the most important of the four that are being considered in this paper.
It is claimed that project management maturity confers business benefits.
Various claims have been made about the benefits that organizations have obtained from using particular maturity models (e.g., (Suares, 1998; Rosenstock, Johnston, & Anderson, 2000; Peterson, 2000). The implications are that mature organizations are able to:
- Manage all the projects undertaken by an organization effectively (Suares, 1998)
- Improve continually the performance of all projects undertaken by an organization (Peterson, 2000), and
- Improve dialogue between the project management community and organizational top management (Peterson, 2000)
In the introductions to two of the more recent project management maturity models, Project Management Maturity Model (PMMM) and OPM3, the benefits that are to be expected from using the models to improve maturity also include:
- The creation of an organization-wide ability for managing projects based on standard, defined project management processes that can be tailored to meet the specific needs of individual projects.
- Roles and responsibilities for carrying out all project-related activities are defined and are clear throughout the organization.
- Provide the organization with project information from previous projects on which to evaluate project schedules and budgets, ensure they are realistic, and review project performance. (Office of Government Commerce, 2002, pp. 3-4)
- “Enables the organization to advance its strategic goals through the application of project management principles and practices. In other words it bridges the gap between strategy and individual projects.” (Project Management Institute, 2003, p. xiv).
In an interesting application of maturity assessments, Ibbs and Reginato (2002) suggest that, as an organization grows in project management maturity, it obtains a better project management performance at a lower cost.
These all sound like excellent benefits, although in sounding a warning note that maturity models may not be the “silver bullets” that some hope for, Thomas and Jugdev (2002) examine maturity models (MMs, in the language of their article) from the viewpoint of four different resource-based models in order to assess whether or not the possession of a higher maturity level in project management confers a competitive advantage on an organization. The article concludes that MMs possess some but not all of the characteristics of a strategic asset and thus cannot, in and of themselves, confer competitive advantage.
This conclusion is based in part on their observation that although “MMs are a component of project management [they are] not a holistic representation of the discipline.” (p. 11) The extent to which maturity models are indeed a representation of the discipline will be assessed below in the examination of Question 4.
In the meantime, the benefits that project management maturity is claimed to provide, all relate to improvements in project success, so it is to that topic that the paper now turns.
From the point of view of business organizations, projects are a means to an end, not an end in themselves.
Businesses don’t exist in order to undertake projects – they undertake projects in order to accomplish one or more business goals in pursuit of their chosen strategic intent.
In today’s turbulent times, it seems as if every organization holds on to some form of “strategic intent” as a compass to guide it through rough waters. This may be a collective intention arrived at through a deliberate process of analysis and dialogue that is then formally written down in an “organizational strategy” document, or it may represent the emergent characteristics of the combined actions of the whole organization expressed as a purpose or a series of goals. But just as an individual person seems to need meaning in order to make sense of his or her life (Frankl, 1959), so an organization seems to need a strategic intent in order to survive in a turbulent business environment (de Geus, 1997).
The strategic intent, of course, varies from organization to organization, but it will always encompass two kinds of goals: improvement of the current products or services and the processes and technologies for delivering them; and innovation and introduction of new products or services, processes or technologies (Anbari, 2003). Regardless of industry or market sector, each of these two kinds of goals requires projects to be established, executed, and completed so, in a very real sense, projects can be considered as an integral part of “strategy in action,” and the successful implementation of every strategy can be said to depend to a greater or lesser extent on the successful completion of projects.
The project management community is starting to relate project success to business success.
Many of the practitioner-focused textbooks on project management define project success criteria in terms of the time, cost, and product performance (expressed as quality, or scope, or conformance to requirements) compared to the plan. In a seminal article on the subject of success, however, De Wit (1988) distinguished the success of project management (for which measures of time, cost, and quality might be broadly appropriate) from the success of the project, which will depend on a wider range of measures.
The importance of this distinction is emphasized by Munns & Bjeirmi (1996), who draw attention to the short-term goals of the project manager (in delivering the required product or service to schedule and within budget) as opposed to the long-term goals of the project (to deliver the promised business benefits). Kerzner makes a similar distinction between “successful projects” and “successfully managed projects.” “Successful implementation of project management does not guarantee that individual projects will be successful … Companies excellent in project management still have their share of project failures. Should a company find that 100 percent of their projects are successful, then that company is simply not taking enough business risks” (Kerzner, 1998, p. 37).
De Wit, as it happens, is following Baker, Murphy and Fisher’s classic analysis of 650 completed aerospace, construction, and other projects (1974), which was subsequently developed further by the same authors (1988). They concluded (p. 903) that “if the project meets the technical performance specifications and/or mission to be performed, and if there is a high level of satisfaction concerning the project outcome among key people in the parent organization, key people in the client organization, key people on the project team and key users or clientele of the project effort, the project is considered an overall success.” A definition that includes elements of both project management success (technical performance specifications; satisfaction of key people on the project team) and project success (meets mission to be performed; satisfaction in parent and client organization).
This tendency to blur the distinction is also followed in work subsequent to Baker, Murphy and Fisher by authors writing both before and after De Wit’s article. Morris and Hough (1987), for example, in their seminal work on major projects, make a convincing case for the popular perception that an excessively large number of “major projects” are perceived by the public to fail. They then argue—on the basis of both a comprehensive survey of the literature and also eight meticulously conducted case studies—for three or possibly four dimensions to project success criteria: project functionality, project management, contractors’ commercial performance, and, possibly, in the event that a project was cancelled, whether the cancellation was made on a reasonable basis and the termination handled efficiently. Project functionality, as defined by Morris and Hough, includes an assessment of both project technical performance which forms a part of “project management success,” and other aspects of performance, which presages the much more recent language of benefits management.
More recently, a survey of 127 Israeli project managers (Shenhar, Levy, & Dvir, 1997) concluded that there are four dimensions to project success: project efficiency (broadly De Wit’s “project management success”), impact on customer, business success, and preparation for the future. The latter three fall within De Wit’s category of “project success,” and are remarkably similar to Baker, Murphy and Fisher’s conclusions.
A backdrop to the discussion on success criteria is provided by an understanding of the different parties to the project that have a legitimate interest in its success or failure. Baker, Murphy, and Fisher (1988; pp. 903ff) emphasize the importance of perceptions and name the “client” and the “parent” in addition to the project team. Morris and Hough (1987, pp. 194ff) refer to “sponsors,” contractors, owners, regulators, financiers and governments as well as citizens and environmentalists. DeWit (1988) reviews the breadth of possible project “stakeholders,” as does Geddes (1990). Authors generally acknowledge that each stakeholder group can have different criteria for the success of the project, thereby introducing greater complexity to the subject.
Project success can be related to business success through three distinct levels in an organization.
As has been seen earlier, OPM3 introduces the important concept of three separate domains – Project, Program. and Portfolio. Interestingly, a recent article identifies these same domains as steps on a ladder of maturity (Anderson & Jessen, 2003) from project level, to program level, and finally to portfolio or business level.
These two independent pointers also link into three questions that have been posed elsewhere about project success: “What factors lead to project management success?”, “What factors lead to a successful project?”; and “What factors lead to consistently successful projects?” (Cooke-Davies, 2002). Each of these corresponds to one of the steps on Anderson’s ladder, so it is worthwhile for a moment to think about them not so much from the point of view of maturity, but from the point of view of project and business success.
Level 1. Project management success – was the project done right?
This is the measure of success that has dominated the practitioner-oriented literature on project management. In the folklore of the project manager, it is about managing time, cost, and quality. In reality, project objectives are rarely this simple. There will often be a business case to be borne in mind or a gross profit to be made; there may be health, safety, and environmental objectives to be accomplished; if the project is a technical one, or a “platform” for new product development, there may be scientific or technical goals to reach.
Nevertheless, the principle is simple: once the objectives of a project have been clearly defined and the constraints spelled out, then the project manager and her or his team can use their best endeavors to deliver the project so that it meets the objectives within the constraints. If anything changes, which is likely given the inherent uncertainty that is involved in any new endeavor, then techniques such as project risk management and project change control can be called into play as appropriate. As a guided missile seeks its target, adjusting its trajectory as appropriate along the way, so the project team seeks to achieve the project objectives.
It is far from being the whole story, however, and for the second level of success it is necessary to turn to the second of De Wit’s levels – what he calls “project success” (1988).
Level 2. Project success – was the right project done?
This level of project success is perhaps the one that is of most interest to the “owner” or “sponsor” of the project. It is about “value for money” in its broadest sense. The assumption is that the project will be successful only if it is successfully operated, and if it delivers the benefits that were envisaged by the people and organizations (i.e., the stakeholders) that agreed to undertake the project in the first place. An analysis of six recent project management “bodies of knowledge” identified sixty core elements that are central to the way a project manager thinks about his or her work (Cooke-Davies, 2001). When these are clustered into eleven topic areas, and related to each other through a “systemigram,” it becomes clear that “anticipated benefits” are the touchstone not only for formal “stage gate” reviews of projects, but also for the continuous “informal assessment” of the likely success of projects carried out by owners, sponsors, or senior management, and for influencing decisions about priorities and resource allocation.
Of course, as Figure 1 shows, benefits are not delivered or realized by the project manager and project team; they require the actions of operations management. This calls for a close cooperation between the project team on the one hand and the “sponsor,” “customer,” and/or “user(s)” on the other, and involves dialogue with a wider cross-section of the organization than is necessary for project management success. There is the added complication that the extent of project success is unlikely to become clear during the life of the project itself, whether success is measured quantitatively in terms of financial benefits or qualitatively in some less tangible form.
Figure 1: The involvement of both project management and operations management in the achievement of “project success”
It is not being suggested here that project success is somehow a “better” level at which to establish success criteria. Both project success and project management success are important to any project. If a project achieves project success without project management success, there is the inevitable conclusion that even greater benefits could have been realized. On the other hand, if project management success is achieved without project success, then the owner or sponsor has failed to obtain the benefits that the project was designed to provide. And that brings us to the third level of success.
Level 3. Consistent project success – Were the right projects done right, time after time?
As the focus moves from project management success, through project success, to consistent project success, a completely new set of criteria comes into play, as adjudged by different groups of stakeholders.
At this level, a discussion of success criteria is one that embraces the whole organization, and that is inevitably influenced by its chosen market and strategic intent. For operations-driven organizations (such as financial services companies, or mass manufacturers), consistent project success in such areas as effective overall IT expenditure and new product development can lead to competitive advantage. For project-based organizations (such as engineering contractors, defense suppliers or turnkey IT systems providers), consistent project success can lead to profitable expansion.
In recent years, there has been a growing interest in project portfolio management for new product development (e.g., Cooper, Edgett, & Kleinschmidt, 2001), specifically for R&D (e.g., Matheson & Matheson, 1998) and generally for project spending in organizations (e.g., Artto, Martinsuo, & Aalto, 2001). But many organizations, particularly in traditional project-based industries, do not adopt a portfolio approach. Thomas and Jugdev (2002) have emphasized that long-range competitive advantage is enjoyed by those organizations that make the best use of their strategic assets (i.e., resources) and all organizations, regardless of industry, must necessarily be concerned about the effective and efficient use of their scarce resources.
Implications
Before project management maturity can be said with confidence to confer business benefits on an organization, there are several patches of ground that still need to be cleared. The existence of OPM3 notwithstanding, research has yet to identify beyond reasonable doubt the factors that lead to project success and through successful projects to business success in different industries and for different types of projects. In view of these uncertainties, therefore, it is now appropriate to focus on the last of the four questions posed in this paper, and consider to what extent it is now possible or sensible to assess an organization’s maturity.
4. How can “maturity” be assessed? What aspects of projects and their management should be measured?
Maturity models, as they stand today, do not assess an organization’s project management maturity with any certainty.
In a searching examination of the practical and theoretical shortcomings of project management maturity models, Thomas and Jugdev summarize a number of criticisms of them (Thomas & Jugdev, 2002) and conclude that “MMs are a component of project management, but not a holistic representation of the discipline.” (p. 11). Their article was published, of course, before the latest generation of maturity models such as OPM3 and PMMM was launched, but the newer models do not substantially affect the thrust of their argument, which is directed as much against capability-maturity models when they are applied to the management of projects as it is against project management maturity models.
The earlier discussion in this paper, particularly in answer to questions 2 and 3, adds further arguments in support of the claim that maturity models are not a holistic representation of the discipline. But there are two further points to be considered.
First, there is the question of precisely what constitutes “the management of projects.” Unless the scope of the topic can be agreed, it is unlikely that it will be possible to agree on what the “management of projects” might look like in its “perfected end-state,” so this discussion is fundamental to whether or not maturity models as they stand today are capable of assessing an organization’s maturity with any certainty.
Perhaps the obvious place to start is with a review of the “bodies of knowledge” that are produced by several of the world’s project management professional associations. Both the longest established and the most widely distributed is undoubtedly the PMBOK Guide produced by the Project Management Institute. First produced in 1976, and most recently updated in 2000, this document had over 270,000 copies in circulation in September, 2001 (Crawford, 2002).
The Association for Project Management in 1986 developed the framework for what was to become the APM Body of Knowledge, which is now in its fourth edition, having been updated in 2000 (Dixon, 2000) on the basis of research carried out by the Centre for Research in the Management of Projects (Morris, 2001). A much broader range of topics is covered by the APM, in line with the findings of research into project success.
Following the development of “bodies of knowledge” by various European professional associations, the International Project Management Association in 1998 published in French, German and English the “International Competency Baseline” (Caupin, Knoepfel, Morris, Motzel, & Pannenbaecker, 1999), offering a coordinated set of definitions to the terms used in the Swiss, German, French and United Kingdom documents. Other professional associations (e.g. Australian Institute of Project Management [AIPM], Project Management South Africa [PMSA]) have their own “bodies of knowledge” and/or competency standards, usually resembling some combination of those that have already been discussed.
Most recently, a three-year joint academic/government/industry study in Japan has resulted in the production of an innovative standard for project management known as P2M (Project Management Professionals Certification Center {PMCC} 2002). This is remarkable for the thoroughness with which it re-examines and re-defines the practices, processes and competencies that are necessary to deliver innovation through strategies, programs and projects.
It has been argued forcefully and cogently (Crawford, 1998; Crawford, 2001; Morris, 2001; Crawford, 2002; Morris, 2003) that the absence of global standards works to the detriment of the practice of managing projects in multi-national or global organizations. Precisely the same argument applies to maturity models -- the absence of a generally accepted definition of what is involved inevitably inhibits the value of any maturity model to the whole of an organization.
The second additional argument has to do with the epistemology that underpins all maturity models. At the conclusion of their paper on the need for a new project management epistemology, Melgrati and Damiani (2002) argued that a fundamental change should take place so that “the project is not viewed as an objective and inevitably rational system, but rather as a concrete system where what matters are the impulses of those who have decided that they want change, where, for example improvisation prevails over planning, and where the essence of the method and the profession lies [in] the capacity to coordinate a substantial rationality constantly jeopardized by bureaucratic formalism and by the claims to omnipotence and atemporality of permanent organizations.”
Just such an alternative epistemology is offered by the “complex responsive processes” strand of complexity theory (Stacey, Griffin, & Shaw, 2000; Stacey, 2001; Streatfield, 2001; Shaw, 2002; Stacey, 2003), which emphasizes the importance of human interactions at the expense of any concept of organization as a “system,” temporary or permanent.
If this alternative view is a valid one (and the references cited provide a convincing prime facie case), then human attitudes, interactions and relationships lie at the heart of organizational performance, and it is likely to prove epistemologically challenging to include them in the scope of a project management maturity model that is inherently based on control/systems thinking. Even if it isn’t, a weaker form of the argument states that maturity models are essentially process models, and the management of projects encompasses more varied categories of human thought, action, and interaction than simply processes and workgroup practices.
At present, organizations measure aspects of the management of projects at each of the three levels of success.
The final question in this paper is an empirical one — what do organizations actually measure? During 2002 and 2003, a survey was conducted by Marlies Egberding, Frank Davies and the present author among 71 respondents employed by 26 organizations in 6 countries (See Figure 2 below). Each of the organizations is explicitly committed to improving corporate results through projects, and so the sample can be regarded as appropriate for organizations that are concerned with improving their project management maturity.
There is no room for more than the most cursory analysis of the results of this survey, which will be written up more fully elsewhere. It is cited here simply to provide insights into the actual measures that organizations currently use to measure the success of their projects.
The first 39 respondents were interviewed for between one and two hours each, seeking information about the metrics that they either provided or were provided with, and as a result of the insights gained during these interviews, the remaining respondents were provided with written questionnaires.
Figure 2: Industries, organizations, interviewees and metrics in the survey on metrics in use
The questionnaire was developed using the Goal/Question/Metrics method, and respondents were selected as holding one of four types of jobs: manager of multiple projects, financial manager, project manager, or provider of project support. There was a reasonable spread of each of the four job roles from each industry grouping (See Figure 3).
Figure 3: Number of metrics provided by each job role in each industry grouping
Each of the job roles provided details of metrics that related to each of the three organizational levels, although project managers naturally provided a higher percentage at the project level (see Figure 4).
Figure 4: Number of metrics provided by each job role at each organizational level
As a final “reason check,” each of the metrics provided by level was consistent with the kind of success criteria that were described above (see Figure 5).
Figure 5: Number of metrics of each type that are appropriate to each organizational level.
Implications
Maturity models are not an adequate way of measuring organizational project management maturity, at least as they now stand. An examination of the metrics that are used by organizations aspiring to achieve organizational project management maturity, however, suggests that the distinction between the project, program (sponsor), and portfolio (business) levels of organizational focus is a valid one that represents a real advance in the discussion about maturity.
Concluding observations
In spite of the four significant problems that currently bedevil discussions about organizational project management maturity, the field offers interesting opportunities for project management research to contribute to bridging the gap between an organization’s top management and the community of project managers that it employs. These discussions will need to be broadened to include both line and staff managers, and will benefit from being informed by research into the nature of maturity, the benefits it can be shown to bestow, the differences in maturity profiles between different industries and different types of projects, and the suite of metrics that will best help an organization to establish clear links between business success, project success and a mature project management capability.