PROJECT MANAGEMENT'S GROWING popularity has stimulated interest in how companies compare in their application of project management processes, tools and techniques. Though many other industries (e.g., automotive, semiconductor, engineering-construction) have benchmarked their operations for years, the project management profession has lagged in this regard. (For information on these benchmarking initiatives, see Construction Industry Institute, “CII Benchmarking Now Underway,” CII News, November 1996, and “Assessment of Owner Project Management Practices and Performance,” Special CII Publication, April 1990; “The Competitive Semiconductor Manufacturing Survey: First Report on Results of the Main Phase, The Competitive Semiconductor Program,” Report CSM-02, Engineering Systems Research Center, University of California at Berkeley, April 1993; and Womack, Jones, and Roos, Daniel, The Machine That Changed the World, Harper Perennial, 1991.)
A Study Team was consequently formed and worked for over a year to develop and apply a comprehensive benchmarking procedure. Our objectives were to (1) develop a tool that could be used for an indefinite period of time for ongoing benchmarking, and (2) apply that tool to collect and compare project management process information for a number of leading organizations. Ultimately, we expected that this information could be used to improve the project management effectiveness of those organizations.
This article summarizes that research, which was sponsored by the Project Management Institute Educational Foundation and Northern California PMI Chapter, and conducted by a team from the University of California at Berkeley. A companion article, “Measuring Project Management's Return on Investment,” published in the November 1997 PM Network, supplements the results presented here.
Research Methodology
In this study projects were viewed as having six lifecycle phases and eight functional management areas, consistent with the Exposure Draft of A Guide to the Project Management Body of Knowledge (PMBOK Guide) that existed when the study began. These phases and functional management areas were used to analyze project management maturity (PMM) in detail.
As explained in our previous article, the evaluation tool was extensive and specific. Exhibit 1 shows the distribution of the 149 multiple-choice questions contained in the benchmarking tool. The numbers in the cells of this spreadsheet show how many questions for each functional management/phase combination were contained in the benchmarking tool.
Thirty-eight companies participated in this study: 15 engineering-construction companies (E-C); 10 information management and movement (IMM) companies; 10 information systems (IS) companies; and three high-tech manufacturing (HTM) companies. The E-C companies included contractors as well as owners who purchase engineering-construction services. Companies that participated included ATT, Bechtel, Chevron, East Bay Municipal Utility District, Eichleay Engineers, Hewlett-Packard, IBM, Kodak, Lucent Technologies, Proctor & Gamble, and Sun Microsystems. All data was collected in a confidential manner so that only the Berkeley researchers knew the details for any particular company.
Exhibit 1. The benchmarking tool contained 149 multiple-choice questions. The numbers in the cells of this spreadsheet show how many questions pertained to each functional management/phase combination.
Reader Service Number 5093
Results of PMM Benchmarking
Average Project Management Costs. One key question that the study team wanted to explore was the cost of project management services as a percentage of total project cost. These costs were computed as a percentage of a company's or department's project-based revenues or sales.
Though difficult to answer precisely, an order of magnitude estimate from this study is that PM services cost on average about 6 percent of a project's total revenues. Exhibit 2 shows the details.
Results of Benchmarking PMM. A substantial amount of information was generated in this study, and one of the more important lessons is that this wealth of information can be used selectively to improve a company's operations.
For instance, consider Exhibit 3, which describes the PMM levels for the 38 companies. EC-12 is an engineering-construction company that has an overall PMM of 3.68, considerably higher than the average of 3.26. Yet when that company's performance is studied in more detail it can be seen that it is relatively weak in terms of Procurement Management (2.00 vs. a median of 3.14 for all companies) and Initiating Projects (1.00 vs. 3.50 for all companies). Conversely, it is relatively strong in Time Management (4.06 vs. 3.39) and Closing Out Projects (5.00 vs. 3.25). The inference is that EC-12's management should concentrate any new project management investments in these first two areas.
The overall PMM of the 38 organizations and four industries ranged from a low of 3.06 for IS to a high of 3.36 for E-C. (For purposes of brevity in this article we will refer to Information Systems as an industry rather than the more proper term application area.) The median PMM was 3.26 for all companies. Because the rating scale ranges from 1 to 5, there is still a substantial opportunity for improvement of project management practices for all four industries. Exhibit 4 contains the details.
Though there is no vast difference in the median scores for the different industries, IS is less mature than the other industries and its higher standard deviation indicates less consistency.
Benchmarking Functional Management PMM Levels. Also of interest were the details of how companies and industries compared. One way to examine the data was to investigate the PMM levels by each of the eight functional management areas. Exhibits 5 and 6 contain the details.
In general, IS had the lowest PMM score and HTM the highest, though only three HTM companies participated in this part of the study.
Cost Management was strong for these companies, as indicated by its high median PMM level (3.48). On the other hand, Risk Management's PMM level had the lowest median score. It also had a relatively low standard deviation, indicating that all companies were consistently in need of substantial improvement. This was the only category where the PMM rating for all companies was below 3.0.
Quality Management's PMM level had the highest standard deviation of all functional management categories, indicating a very high dispersion among companies, and Human Resource Management had the lowest.
All industries and companies have substantial room to improve, since the maximum maturity level is a 5.00 score.
Benchmarking Phase PMM Levels. Lifecycle Phase Maturities were also studied. See Exhibits 7 and 8.
Once again IS was the least mature and EC and HTM the most mature by our benchmarking system.
For all industries taken together, Define and Organize was the most mature phase and Project-Driven Organization was the least mature. The standard deviation for Initiating Projects was higher than for any other lifecycle phase, and lower for Define and Organize. As before, all companies and all industries have substantial room to improve.
Findings
This research provides comparative studies on project management practices across industries as well as companies within an industry. By comparing and correlating the organizational aspect of project management process maturity data, the following findings emerged:
Exhibit 2. Project management services, according to the estimate generated by the study, cost on average about 6 percent of a project's total revenues.
Exhibit 3. Looking at the PMM levels for the 38 companies, we can ascertain which areas of a company's project management are strong and which need attention so that a company's management can concentrate any new project management investments in the weaker areas.
Exhibit 4. The median PMM was 3.26 for all companies. Because the rating scale ranges from 1 to 5, there is still substantial opportunity for improvement of project management practices for all four industries.
Exhibit 5. Here's how the various functional management areas stacked up in terms of PMM.
Reader Service Number 5042
■ The project management benchmarking methodology that was developed for this study works for a range of companies and industries. It provides a means for identifying and measuring different PMM Levels. Having that information is important to understanding improvement opportunities and competitive positioning.
■ The actual cost of project management is higher than previously suspected; 6 percent of total project revenues.
■ Even organizations with the highest PMM scores can improve their overall project management processes.
■ In general, the IS Industry had low maturity levels and E-C and HTM had high maturity levels.
■ Cost Management and Defining and Organizing Projects were areas of strength; Risk Management and Project-Driven Organization were areas of weakness.
■ As exemplified by the case of company EC-12, benchmarking allows an organization to identify its project management process maturity in specific areas. This, in turn, permits carefully targeted investments in training, system development, hiring, and so forth.
Recommendations. This study was a first attempt to benchmark project management process maturity on a widespread basis. Project lifecycle phase and functional management area maturities were evaluated using a benchmarking device that was developed for a broad cross-section of companies and industries. In doing so, this study permits a factual and quantitative way to measure project management practices and performance.
Recommendations of this study include:
■ The project management benchmarking methodology should be applied to other industries and companies to further our understanding of project management.
■ This benchmarking methodology should also be reapplied to companies that participated in the first year's study to determine the impacts of improvements that have been implemented during this past year.
■ Companies should benchmark their operations using factual, impartial techniques such as those contained in this benchmarking methodology. Much anecdotal information and supposition exists in the general literature but, as was repeatedly seen in this study, much of that supposition is wrong.
Summary and Conclusions. The results from this study can be used by any organization to evaluate its project management process relative to peer organizations. With this level of detail a company can determine where to target any new project management investments as opposed to a broad-brush investment that may not specifically address areas that are truly deficient.
Many perceptions and beliefs about project management were not confirmed by the data. The need for factual, rationally collected information is crucial to properly understanding project management and to properly managing our organizations.
A complete copy of the final report can be obtained from PMI Publications (The Benefits of Project Management: Financial and Organizational Rewards to Corporations, C. William Ibbs and Young-Hoon Kwak, 1997).
Acknowledgments. This research is sponsored by the Project Management Institute Educational Foundation and the Northern California PMI Chapter and conducted by a University of California at Berkeley research team. Integrated Project Systems of San Carlos, Calif., donated the project management benchmarking questionnaire. The questionnaire was then amended to meet the specific needs of this particular study by the UCB research team. The authors wish to thank the PMI study team members who supported this study, including Dan Ono, Jim McFarlin, Mike McCauley, Paul Nelson, Ray Rowley, Bill Ruggles, Ahmet Taspinar, Bob Thompson and Cathy Tonne. We also appreciate the time and resources of the 38 companies and agencies that participated in our study
Exhibit 6. PMM in the various functional management areas is compared across industries.
Exhibit 7. Looking at the correlation between the six project phases and PMM, overall Define and Organize was the most mature phase. By industry segment, IS was the least mature, with E-C and HTM the most mature.
Exhibit 8. For all industries taken together, Define and Organize was the most mature phase and Project-Driven Organization the least mature. The standard deviation for Initiating Projects was higher than for any other lifecycle phase, and lower for Define and Organize.