Project Management Institute

From making sense to making cents

part 3 -- the process

ExecutiveNotebook

by Joan Knutson, Contributing Editor

LET'S REVISIT THE TOPIC of my January and February PM Network columns, in which I discussed the four plateaus by which to measure project management: (1) Comprehension and Acceptance, which assesses that project management is being used; (2) Application, which judges success by the frequency and the accuracy with which the technical and social components of project management are applied; (3) Influence on the Business, which assesses success by quantifying the business results produced by the existence of the project management discipline; (4) Return on Investment, which appraises success through the use of a standard series of calculations.

Like any good newshound, the astute collector of data about project management capabilties must ask the five W's and an H.

The challenge is to implement a process that will gather, analyze and report the results. In measuring project management success, this article focuses on the process.

To measure project management success in any one of the four plateaus, metrics must be set. These metrics aid in evaluating progress over time, set levels of commitment, and position accountability to reach these targets. Actual metrics are acquired through assessments—individual, 360-de-gree, or organizational; through action plans and performance contracts; and/or through performance monitoring.

The process to monitor, track and manage these metrics attempts to balance feasibility, credibility and simplicity, and consists of four steps: Data Collection; Data Analysis; Results Compilation; Action Plan Implementation. For these four steps to be successful, a discipline must be established which mandates that the project management measurement process be managed and maintained.


images

Joan Knutson is founder and president of Project Mentors, a San Francisco-based project management training and consulting firm. She can be reached at 415/955-5777. Send comments on this column to editorial@pmi.org.

Data Collection. This is critical. Only sound data collected in this step can assure valid reports and credible action plans coming out of the process. It is important to generate a data collection plan before beginning the step. Here are questions that need to be answered in order to create the plan:

images Why? Which plateau are we addressing and what information are we trying to ascertain?

images What? What type of data do we want to collect— financial, attitudinal, performance of specific competencies, designated project success relative to schedule, resource, cost and/or quality?

images How? What medium will we use to collect the data—conduct interviews or focus groups, administer an assessment tool, observe and document the observations?

images Where? What is the sphere of collection—the entire project management community or a sampling of the community; internal or external or both?

images When? How frequently will data be collected and reported out?

images Who? Who will collect and process the information—an internal project management council, a project management center for excellence, or a staff position reporting to senior management?

There is no end to the creative ways in which one can collect data; examples include:

images Sending out assessment tools

images Picking several work-in-progress projects and conducting an interim audit

images Administering a “competency exam” to a select control group of project players

images Monitoring each project player's performance and development objectives, which are part of their performance management contract

images Establishing personal performance contracts with project players prior to beginning a project or prior to rolling out a project management initiative and then tracking those contracts as the project progresses

images Examining various business performance records and operational data

Measuring Project Management Summary

As you evaluate the success or failure of project management in each of the four plateaus using the process described in the article, keep your eyes open for the barriers that inhibit and the enablers that encourage the application of project management. To improve the scorecard, before proceeding to implement a project management measurement system you must remove the barriers and parley the enablers.

Barriers

The effort costs time and money. Be sure that everyone buys into “the return on the investment” for doing this work.

People may not have the skills to perform project management well. Provide them with the necessary training.

People may not buy-in to the particular project management initiative being measured, because they see no incentives or consequences. “Sell” the initiative as you would “sell” the value of doing any project.

Project management is a discipline that requires discipline. Be sure the organization is ready to adhere to the rigor required.

The focus may be too much on the process of project management and not enough on the people who make project management successful. See that there is a balance between the productivity of the process and the performance of the people.

A project initiative may not have been well scoped out and/or may have been done for the wrong reasons or may have been tried in order to attain the wrong result. If this is true, don't discredit the discipline of project management because of a mistake in judgment.

People are afraid of being evaluated by quantifiable metrics. Show them that this is a benefit, not a consequence.

Other pressures make it difficult to stay the course; in other words, to continue to collect, analyze, compile and react to the data. Be sure that this measurement effort is officially scheduled and funded.

Benefits

There is a viable process in place to measure the contribution of project management to the organization, thus assuring management that they are making a wise investment. Management gains confidence that project management is an investment and not a draining expense.

Knowing what is working and what is not working allows management to make better priority decisions relative to where to spend their project management maintenance and development dollars.

Knowing what is working and what is not allows management to make adjustments and changes to how project management is being performed and to redesign, re-target or eliminate projects or project initiatives as appropriate.

Everyone involved in the project management community can focus on results, not activity or politics.

Customers and sponsors can now see project management as value-added, not as a time waster.

We, in project management, may be reluctant to implement a measurement system because we think that managers really don't care whether or not project management is successful; managers haven't asked specifically for a measurement system, therefore they are not expecting one; we are all professionals and thus we all do a good job, so there is no need to measure it; project management as a discipline is impossible to measure; or that project management is needed, so why do we have to justify ourselves—we'll continue to get funded no matter what.

In my opinion, this is faulty thinking. More and more money is being spent on project management. Project management is becoming more and more visible. It behooves us to step up and measure what project management is doing for our organizations, to laud those successes, and to continually raise the bar, stretching ourselves to become better and better.

images Querying project players, users of the outcomes of projects, senior management, external experts or customers on what percentage of improvement, and/or degree of impact (low, medium, high), and/or increased benefits or reduced expenses are a result of the intervention of project management.

Data Analysis. This step uses the data collected. Here the output performance directly related to project management is evaluated along with how this output performance has generated specific improvements in the production and/or performance within the organization. Approaches to compartmentalizing the data include:

images Control Group. Compare the same data elements from one group that is using a specific project initiative as opposed to another group that is not using that initiative.

images Trend Analysis. Plot historical data prior to the implementation of project management or of a project management initiative. Then project out what the metrics would be (at predetermined time checkpoints) if a project management intervention were never implemented. At those specified time checkpoints, collect and plot actuals. If actuals are better than projections, intervention is succeeding.

images Forecasting. This process is similar to trend analysis, but without historical data to rely on.

The goal is to convert intangible benefits to some tangible interpretation of success. If the intangible benefits are too subjective, don't throw them away. Intangible benefits can be as impressive as tangible ones. Create a list of those intangible benefits and let them speak for themselves.

Results Compilation. Results are compiled by documenting the techniques of the data analysis. Documentation may be in the form of graphic representation, such as line graphs or columnar charts. Tables or matrices also lend themselves to communicating this type of data. Lists that have been sorted and extracted relative to specific segments of data can be meaningful. And straight text, if organized well, can accomplish the purpose.

Action Plan Implementation. This measurement process is not just a one-time snapshot. It is meant to be an ongoing reevaluation and reengineering of the process and an ongoing building of the performance of the people.

Create the action plan by taking the reports compilation and isolating those metrics that are not being met or not being met as quickly as expected. Action plans are created and implemented with frequent review checkpoints until the metric is being accomplished.

When creating these action plans:

images All parties involved need to commit to accomplishing their part of the plan

images The posture needs to be that the metric is meaningful and that the expectation is to hold people accountable for meeting the metric

images If the original metric is found to be unrealistic or unattainable, it should be discarded or changed and

images The action plans are the important effort before we recycle back to Step 1 of the process—data collection.

A successful process should be (1) simple, economical and easily implemented; (2) routine, embedded into the daily operational project management process; practical, credible and account for external factors; (3) applicable to all projects; and (4) address both the hard and soft data.

THE PROCESS OF measuring project management success is classic and straightforward: gather the data; analyze it; compile the reports; and implement action plans. We are either measuring the success of a composite of projects or of a specific project management initiative. Once the technique(s) of measurement for each of the plateaus have been chosen, the process above can be implemented and refined as time goes on. images


Reader Service Number 012

This material has been reproduced with the permission of the copyright owner. Unauthorized reproduction of this material is strictly prohibited. For permission to reproduce this material, please contact PMI.

JULY 1999 PM NETWORK

Advertisement

Advertisement

Related Content

  • Project Management Journal

    The Long Road to Benefits Management member content locked

    By Svejvig, Per | Schlichter, Bjarne Rerup This article reports on a four- year action research study, where the aims were to study benefits management at a detailed, practical level and to evaluate the benefits management practices applied.…

  • PM Network

    A Measure of Urgency

    By Grgurich, Hayley Saving the planet is the ultimate goal for climate change initiatives. But before those projects get the green light, their leaders must show how they're moving the needle. Three project leaders…

  • PM Network

    Prefab Push

    To drive down construction costs, shave project schedules and sidestep the U.S. labor shortage hindering its build projects, hotel brand Marriott announced in 2017 that it would be embracing modular…

  • PM Network

    Transfer of Power

    By Parsi, Novid Anxiety and opportunity have gripped Germany's project management. With all eyes focused on post-Brexit repercussions, political and economic uncertainty threaten to slow Germany's full-throttle…

  • PM Network

    Leading Edge

    By Fister Gale, Sarah In the face of unprecedented disruption, organizations must adapt. This year's PMI® Project Excellence Award winners proved that, when teams embrace the need to pivot, they can deliver real value to…

Advertisement