Tuning up your PM engine
value management from the field
Over the past few years, Global 2000 companies from a variety of industries — information technology, manufacturing, pharmaceutical, new product development, government, professional services — have initiated projects to create measurement programs to measure the value that project management provides to their organizations.
Called pmValue Measurement Program Initiatives, the goals are to:
- Provide tangible metrics to senior management on the value of implementing systematic project management methods in order to reinforce the business case for project management improvement across the organization.
- Boost customer and project team morale by sharing statistics that show the value their work adds to the organization and the improvement they can achieve.
- Track on-going project management performance and the business impact of project management to the organization.
- Initiate metric-based efforts that will streamline the project portfolio.
This presentation will review the process used to develop the measurement programs and, in particular, a pmValue Scorecard. Emphasis will be placed on the lessons learned in that process.
Senior Executives want to know one thing about any project management improvement initiative: “What's the value?” Today more than ever, every dollar invested must be justified, and any initiative must deliver positive and tangible results. The good news is that tangible measures of project management value can be established by asking the right questions and developing an appropriate measurement system.
Previous industry research measured the value of project management in terms of efficiency (time, cost, scope) and finance, and was generally limited to a specific project return on investment. Today, however, there are broader valuation practices that extend to the organization as a whole, and that lend a much clearer, accurate, and useful view of the effects of implementing project management in organizations. What should you measure to determine the benefits of implementing a project management improvement initiative? In the past, and in some cases today as well, the measure most often used is return on investment.
ROI is a measure of profitability computed by dividing the amount of operating income earned by the average investment in operating assets required to earn the income. ROI measures how effectively assets are used to earn income. Financial measures alone, however, don't present a clear picture of the value. They are too unreliable as either a clear gauge of success or a clear sign of the future measures. Unlike financial measures, most operational measures are leading indicators. When they improve, improvement in the financial measures follow, albeit often indirectly, and with a time lag. Therefore, to drive success, financial measures must be supplemented with operational ones. Many, if not most, Global 2000 organizations are taking a balanced scorecard approach to measure and manage their organizations. This is the approach that several Global 2000 organizations chose to take in implementing a pmValue Measurement Program.
Developing a pmValue Measurement Program
The goal of the pmValue initiative in each organization was to develop pmValue Measurement Program. The initiative consisted of a three-phase, six-step development and implementation approach designed to bring the Measurement Team from an introduction to project management-focused measurements through design, development, and implementation of a pmValue Scorecard (see Exhibit 1).
Phase 1 efforts focused on educating the Measurement Team on the pmValue Measurement Program to help them understand and enable them to clearly identify the program's objectives and goals. Organizational constructs that affected the pmValue Measurement Program were identified including stakeholders, organizational mission and strategies, organizational structure, key business processes, project management maturity, prior project management improvement initiatives, current measurement systems, and data availability.
Step 1: pmValue Measurement Readiness Planning
Activities in this step ensured that the pmValue initiative was clearly understood and aligned to support the organizations' strategies — an essential element to support sustained success of the initiative. The primary project management, business unit, and organization goals that influenced the development of the pmValue measures included the following:
Organizational Goals and Objectives
- Reduced costs
- Improved quality
- Improved timing
- Improved productivity
Project Management Goals and Objectives
- More predictable project performance
- More repeatable project performance
- Improved ability to execute projects
- More effective resource management
- Improved internal and external customer satisfaction
- Better alignment of projects to business strategy
- More effective risk management
- Reduced learning curve for new project managers
pmValue Measurement Program Initiative Goals and Objectives
- Measure the business impact of project management improvement initiatives
- Compare the costs to benefits of project management improvement initiatives
- Determine if a project management improvement initiative is accomplishing its objectives
- Identify the strengths and weaknesses in project management processes
- Establish a database of key PM measures
- Understand the infrastructure required to support a measurement program
- Gain a sense of whether the project portfolio is as productive as possible
- Spread the acceptance of project management throughout the enterprise
- Help attain project management and organizational goals and objectives
This step educated the Measurement Team on the issues of project management value measurement and better prepared them to make key decisions throughout the initiative concerning the program's objectives and measures and the implementation approach.
Phase 2 efforts plan the initiative and engage the team to identify measures, develop the pmValue Scorecard, and plan the implementation. After putting a pmValue Initiative Plan and Schedule in place, subsequent steps in this phase continued to build on the team's understanding of the project management measurement program and engage the team to develop the pmValue Scorecard and pmValue Implementation Plan.
Step 2: pmValue Initiative Planning
In this step the team was aligned around the measurement program's objectives, scope, development approach, timeline, deliverables, and implementation strategy. It collaboratively developed a pmValue Initiative Plan and pmValue Initiative Schedule (see Exhibit 2)
Step 3: pmValue Measures Development
In the pmValue Measures Development step, the team created and prioritized the initial list of measures for the Scorecard. It is the initial pass at identifying and prioritizing measures with the primary activity in the step being a collaborative development workshop. A comprehensive list of measures was developed keeping in mind that they needed to be logically linked to the goals described above. The measures also needed to meet the criteria for good measures, which means that the measures selected:
- Provide meaningful information
- Are supported by valid data that is cost effective to capture
- Are acceptable by stakeholders
- Are repeatable
- Are actionable
- Align to organizational objectives
A sample of the list of prospective measures developed by the pmValue Measurement Team is shown in Exhibit 3.
The Measurement Team took the list of prospective measures and prioritized and selected measures that would comprise the pmValue Scorecard. A simple prioritization process was used: criteria were developed for ranking the list of measures on a scale of 1-5, and each of the Measurement Team members ranked the list; average rankings were calculated and a prioritized list was developed for review in Step 4 (see Exhibit 4.)
Step 4: pmValue Scorecard Development
In this step the team reviewed the prioritized measures information developed to date and developed Measure Packages and a cohesive pmValue Scorecard. The team first engaged in measures review, prioritization validation, and Measure Package definition. That information was then used to construct the pmValue Scorecard for review and acceptance by the Measurement Team in preparation for implementation.
A comprehensive definition of each measure is included in a Measure Package to support the initial implementation and ongoing collection of data. Each Measure Package includes the following elements:
- Measure (What): The data to be collected must be clearly identified.
- Objective (Why): The measure's objective must be clearly defined. Why is it being collected? How will it be interpreted? What will it tell us? The core team must understand the objective of each measure.
- Data Capture (How): The mechanism for collecting the data capture must be identified.
- Timing (When): The timing data collection must be defined. pmValue data collection timing must be properly timed to match the type of data and objective. pmValue measures are not intended to track individual project progress, so there would most likely not be a need to collect data monthly. Typically a quarterly or longer interval will support the objectives of the initiative
- Data Location (Where): The location of the data must be identified
- Data Contact (Who): The person responsible for maintaining the data must be identified. From whom the data will be collected? What is the reliability of this source?
Information from the Measure Packages is used to create a pmValue Scorecard, which is a collection and reporting tool for keeping score and reporting progress (see Exhibit 5).
Step 5: pmValue Measurement Program Implementation Planning
The implementation planning efforts in Step Five defined the framework around pmValue measurement processes and data collection that will be used to support ongoing measures program implementation. Key activities in this step included development of an implementation strategy and process.
The pmValue Measurement Program process shown below (see Exhibit 6) describes a systematic approach to project management performance improvement through an ongoing process of establishing project management measures, collecting, analyzing, reviewing, and reporting performance data; using that data to drive performance improvement; and using lessons learned to continuously improve the pmValue Measurement Program process.
Phase Three includes an initial implementation of the program and the transition to ongoing execution of the program.
Step 6: pmValue Measurement Program Implementation
The pmValue Measurement Program Implementation is an ongoing effort to execute the program as documented in the Implementation Plan, using the Measures Packages to reinforce the data requirements, collection timing, and data contact responsibilities. Step 6 begins with the preparation for the initial collection-analysis-reporting cycle and continues through transition of ongoing program execution and continuous improvement responsibilities.
- Organizational strategies and objectives set the foundation for effective measurement programs. It's essential to understand how the critical elements of the organization's strategies and objectives are linked to the measures that comprise the pmValue Scorecard.
- You need to have a very clear idea of the measurement program stakeholders and what their needs and expectations are regarding the program (there are often huge differences in expectation among stakeholders — setting those expectations through clear communication of program goals is a key to success).
- Clearly identify pmValue Measurement Program goals and objectives. Without this clarity, selecting the right set of critical few measures will be difficult.
- In most best-in-class organizations, measurement initiatives are introduced and continually championed and promoted by top executives. When measurement initiatives are introduced from the bottom up, getting senior management buy-in is crucial and may take significant effort. Be prepared to make that effort.
- Develop a clear understanding of measurement terminology, which tends to be confusing and inconsistent, but needs to be understood and agreed upon by the Measurement Team and program stakeholders.
- Communication is crucial for establishing and maintaining a successful measurement program. It should be multidirectional, running top-down, bottom-up, and horizontally within and across the organization.
- The driving force to create a new or improved measurement program is usually a threat to the organization (often a crisis or strong competition). For organizations that are strategically developing measurement programs to enhance their competitive advantage, rather than reacting to their business environment, a sense of urgency must be nurtured and driven by individuals who understand the value of measurement and can evangelize the need for developing a measurement culture. Again, this takes enormous effort and communication.
- It's critical, and very difficult, to limit the number of measures in the pmValue Scorecard. Selecting those critical few measures sharpens the stakeholders' understanding of the issues. Too many measures confuse and complicate (the Measurement Team can't try to please everyone—selecting too many measures will ultimately kill the program).
- Pilot the pmValue Measurement Program before full implementation. And implementation should come in phases — implement a critical few high-value measures at first and identify more detailed measures when the organization has developed a measurement culture is ready to collect and analyze more complex measures.
- Successful deployment of a measurement program requires a successful system of accountability; that is, all stakeholders need to buy into measurement by assuming responsibility for some part of the measurement process (sponsorship, analysis, data collection and monitoring, evangelism, etc.).
- Benchmark against industry standards if possible.
- Identify a central area of responsibility for the pmValue Measurement Program.
- You need to determine what counts as a project (what exactly will be measured).
- Reinforce the fact that pmValue measurement is measuring performance change due to project management. Measures, therefore, are process-focused, not project-focused (you are not trying to measure the progress of a particular project).
- The measures selected are highly influenced by the project management maturity of the organization. Level one organizations generally need to focus on process compliance and simple cost and/or schedule measures. As the organization matures in their project management capability, more sophisticated measures can be used.
- Analysis is one of the most important steps in pmValue measurement, yet it is often one that is neglected. The insight gained from effective analysis (particularly determining root causes of the results measured) is what makes measurement a valuable business tool.
- Feedback is one of the best assets for continuous improvement. Seek it and use it.
Pennypacker, J.S. (Ed.). (2003). Justifying the value of project management. Havertown, PA: Center for Business Practices.
Oswald, J., & Pennypacker, J.S. (2002). The value of project management: The business case for implementation of project management initiatives. Proceedings of the Project Management Institute Annual Seminars & Symposium.
Pennypacker, J.S. (2001). The value of project management: A Center for Business Practices research report. Havertown, PA: Center for Business Practices.
© 2004 James S. Pennypacker
Originally published as a part of 2004 PMI Global Congress Proceedings – Anaheim, California