Measuring What Matters
Why you need to change your perspective on project success
Measurement is key to demonstrating the impact and value of projects. Measurement provides project visibility and delivers the data to strengthen the future business case for a program or project management office (PMO). However, if project success is limited to the “iron triangle” metrics of scope, schedule, and cost, it often condemns projects (and the PMOs overseeing them) to being “red flagged” as a poor return on investment for organizations. But what if PMOs change the way projects are perceived by emphasizing outcome-based benefits such as operational efficiencies or customer satisfaction?
Project Management Institute (PMI), the world’s leading authority on project management, and professional services firm PwC have teamed up to address how success being measured by project-based organizations impacts PMOs. Our global research indicates that innovation in measurement is a key feature of PMO maturity. Our PMO maturity index highlights a group of 230 PMOs which make up the top 10% of organizations— called the “Top 10 Percent” in this report— which are bringing in a greater variety of measures and using technology to increase the number and variety of metrics and the frequency of measurement. The Top 10 Percent organizations outperformed organizations overall in 2020 in revenue, customer satisfaction, and acquisition (see Figure 1) and Environmental, Social, and Governance (ESG) indicators.
The report also highlights what PMOs can do to reimagine their approach to measurement, which will help to increase the visibility of evolving maturity levels and increase the perceived value and impact of PMOs and project managers across organizations.
Measuring What Matters
Starting the Conversation on Measurement
When projects come into existence, the planning process undoubtedly starts with what needs to be done, when it will be done and how much it will cost. These dimensions frequently form the foundation for tracking progress and reflecting on project success as well. The iron triangle (see Figure 2) is easy to understand, quantifiable, and trackable. However, what it doesn’t do is provide a robust evidence base for the wider impact of projects. The conversation about project outcomes, which happens during the planning process, needs to start driving measurement at project initiation—not afterward. The ability to link performance metrics to outcomes from start to finish is critical in positioning PMOs as valued contributors to strategy delivery.
The Top 10 Percent organizations are more likely to be aligning key performance indicators (KPIs) to wider organizational strategy, communicating the impacts of projects, and making the most effort to improve the line of sight for the C-suite (and engage them in the PMO) during the COVID-19 pandemic (see Figure 3). We believe that this is driving a more outcome-focused approach to measurement. Our research confirms these organizations have a more innovative approach to measurement using more metrics and technology. In order to qualify in the Top 10 Percent, we know they are also more likely than not, to be communicating the impacts of projects and aligning KPIs to wider organizational strategy. That said, there is still more to be done even by the Top 10 Percent to improve the approach to measurement and the use of technology to facilitate measurement.
Our research identifies these four key approaches to improving how the impacts of projects and programs are measured:
- Increasing the number of measures and frequency of measurement.
- Increasing the effectiveness of measures.
- Involving the right stakeholders.
- Increasing the use of technology.
Increasing the Number of Measures and Frequency of Measurement
In our global survey, only 12 project professionals out of 4,069 told us that their organization had no formal metrics in place to monitor the success of projects. The rest, on average, had 7.1 metrics in place for every project—but of these, typically four were linked to the iron triangle (see Figure 4). Therefore, only three additional measures were being used to tell a compelling story of project success. To amplify the impact of projects, and by extension the PMO, additional measures such as societal or environmental impact or return on investment must be the headline act. Being on time, in scope and on budget are the basics; they do not evidence the real change in outcomes that projects can provide.
Overreliance on the iron triangle is also affecting the image of project managers. When participants in our global survey were asked to describe how those outside of the profession would describe project managers, scheduler was the second most frequent word used, selected by 54%. However, among those in the Top 10 Percent, scheduler was just one of many descriptors used to the same degree to describe project managers. In addition, many more participants in the Top 10 Percent used words like changemakers, essential, and realizing visions to describe project managers, compared to organizations overall. This suggests that increasing the scope of measures can also influence the perceived value of project management capabilities and help combat the talent crisis.
The frequency of measurement also poses a challenge. Our findings showed that only 41% of PMOs were consistently measuring and reviewing their performance, and that just over half were regularly communicating project milestones and impacts to the C-suite. Both of these elements are key to improving PMO maturity in our index—using measurements to assess the performance of projects and PMOs has to be done on a regular basis.
Increasing the Effectiveness of Measures
There are two aspects to consider when it comes to the effectiveness of measures: firstly, selecting the right measures and secondly, how to implement them.
Selecting the right measures: A measure that’s useful for one project may not necessarily be the best measure to gauge success in the future. Scope, schedule and cost will always be constants, but the outcome measures need to evolve. Measures have to be actively reviewed by the right people—there needs to be purposeful, collaborative reflection on what the measures are saying to tell the story of project impact in the best way and inform future projects. This reflection has to become commonplace—part of daily team meetings, part of project reviews and part of monitoring PMO performance.
How to implement measures: Customer satisfaction is a score that can be calculated in a number of ways, but the meaning is universally understood. It can also be implemented and quantified through a carefully designed follow-up survey, as Sam Farid, Global IT PMO leader for Fonterra explains (see case study). Conversely, cultural shifts (the willingness of employees to adopt a new process) or operational efficiencies present a bigger challenge; the metric needs to be defined and then presented in a way that’s easy to understand. Like customer satisfaction, the threshold for success needs to be defined, but what that threshold should be is likely to be less obvious. For example, tracking adoption and monitoring behavioral change needs further context through employee and customer consultation, but PMOs can also add further value by linking this to wider project data through project and program management (PPM) software.
Involving the Right Stakeholders
In order to select the right measures, the right stakeholders need to be involved. Our research reveals only 63% of organizations with a PMO actually engage the PMO in the development of success metrics, and only 39% engage the C-suite. Measures being developed without collaboration between the PMO and senior leadership risk the development of metrics that fail to align to the organization’s strategic vision. The Top 10 Percent organizations further refine measurement by involving a greater variety of stakeholders, including customers, consultants, and external stakeholders, to ensure metrics focus on the outcomes that really matter.
Increasing the Use of Technology
One third of the participants in our global survey stated that their organizations were using technology to effectively measure the impact of projects and programs. Strategy execution management technology is helping these organizations capture more metrics beyond the iron triangle—and on average, 8.3 metrics, compared to just 6.5 metrics among those not using technology to assist measurement.
While working remotely during the pandemic, 79% of project managers in organizations using technology to aid measurement said it was easy to effectively measure project/program impacts. This compares to 63% in organizations not using technology to assist in measurement. Over one third of participants in PMOs that are using technology for measurement stated that their PMOs are very successful, versus 21% within PMOs that aren’t using technology for measurement.
It’s not just PMOs that benefit. Where technology is being used to measure the impact of projects and programs, the organization as a whole saw better performance in 2020 versus 2019 in terms of revenue, environment, social and governance (ESG) indicators, customer satisfaction metrics, and customer acquisition levels.
However, even where organizations are seeing a benefit to the use of technology for measurement, they still face similar barriers to investment in technology. Overall, 54% told us budget constraints are a key blocker to investments in technology; this constraint could come from a range of sources—the purchase price, maintenance, or training. Cost, as a barrier, is true regardless of how much organizations are already using technology to facilitate measurement.
Using Technology to Enable Measurement
CASE STUDY: Fonterra Co-operative Group
Sam Farid headed up the revival of Fonterra’s centralized IT PMO four years ago. As part of this initiative, the PMO invested in implementing a project portfolio management (PPM) software, with a key focus, at the time of interview, on increasing the visibility of project and portfolio information utilizing the software.
The PPM software brings a range of benefits, including the management of:
- Tracking and reporting
- Planning and collaboration
Bringing this functionality together creates a single source of truth, and brings together real-time data on demand to enable better investment decision making. “It’s about trying to get people to move away from their spreadsheets,” says Sam. “Spreadsheets create a nightmare for a single source of truth, so we’re trying to get people to move away from their own measure to a centralized system.”
Understanding if the tools, processes, and frameworks provided by the IT PMO are helping to create value is key to measuring the success of the PMO. Sam and his team measure success through customer satisfaction: “If they are not happy, they are not satisfied—then it’s no good!” Customer satisfaction is measured and responded to in a number of ways:
- Annual survey: Sent to project managers and key stakeholders in the wider PMO community to understand the level of satisfaction with the PMO. They evaluate different areas of the PMO, for example onboarding and communication.
- Review of annual PMO roadmap: An annual PMI work plan (roadmap) is established with input from the annual survey and other key sources, such as strategy and operating model direction. The roadmap outlines the set of prioritized actions and improvements to be made over the next period, which can be up to two years to uplift customer satisfaction. This also enables the PMO to define how success will be measured in the coming period, for example, KPIs and Objectives and Key Results (OKRs).
- Post-implementation review (PIR): After every project, the project team and stakeholders complete a qualitative and quantitative review, using a defined template established by the PMO. Output from the PIRs will feed into the PMO roadmap and improvement plan.
A Call to Action
Getting measurement right is a critical part of PMO maturity, providing project visibility, evidence about the impact and value of projects, and data to strengthen the future business case for a PMO. It’s time to get creative about measurement:
- Start the conversation of project and PMO success with outcome-based measurements. The iron triangle remains important but it should not be the only measure. Collaborating across the organization to develop the measures that matter is key to changing the image of the project manager from that of a scheduler to a changemaker.
- Improve collaboration between the PMO and C Suite, and involve a greater variety of stakeholders in identifying appropriate metrics and start this conversation early. Reflect on the effectiveness of measures—they need to evolve over time depending on project and stakeholder needs.
- Invest in technology, including strategy execution management technology and benefit realization tools, that will facilitate measurement and help tell the story of project success. Organizations that are using technology to effectively measure the impact of projects are more likely to use a greater number of metrics and report better business performance than those that are not.
PwC and PMI would like to thank everyone who took part in the survey and in the qualitative interviews referenced in this report.