Introduction
It has long been recognized that sound measurement practices are integral to basic management activities such as project planning, monitoring, and control. Metrics are ever present throughout all phases and facets of project management. As more organizations adopt a management-by-projects approach, there is an increasingly heavy reliance on measurement practices that facilitate informed decision making, using measurement and quantified indices that guide product and process improvement. Metrics need to be integrated into project life-cycle processes to support project selection, project execution, workforce improvement, and portfolio management. Metrics can provide indicators of organizational project management maturity, growth, and sophistication. Metrics also can quantify the adequacy of the organization's vision, the quality of each project deliverable, and the progress of projects, processes, and products.
This paper describes the purpose of project management metrics and presents a description of metrics in three categories: ones involving project things, project people, and the enterprise. The paper then focuses on processes and procedures that an organization must embrace as it establishes a culture focused on metrics to support an organization that is friendly toward projects. The paper concludes with a set of guidelines that project professionals can use in their own organizations to set the stage for a culture focused on metrics and one that is friendly toward projects.
What is a Metric?
In the context of project management activities, metrics can offer a glimpse of the full range of expectations of a particular deliverable or the tasks that would produce that particular deliverable. Additionally, metrics can gauge the status of a deliverable relative to expectations for scope, cost, and delivery date. In the latter case, metrics are also aimed at small measurable attributes that have a predictive or comparative capability. A set of properly-designed metrics can provide an early warning of forthcoming problems. The existence of such a warning mechanism enables the organization to avoid and/or, at the very least, to manage emerging problems. However, ultimately metrics do not make decisions, people do; metrics simply provide the foundation and rationale for the decision. The return on investment of a metrics program is the value of the action that the project professionals might take to manage the issue at hand with the assistance of those metrics.
Purpose of Metrics
The main purpose of using metrics must be decided. Often, those project professionals at the mid levels, who have achieved project management certification, and who are users of specific project management tools and techniques, recognize the value of project management, but this value may not be fully embraced by those at the top. A metrics program is one way to communicate effectively the benefits of project management to senior executives. Furthermore, with projects increasingly a way of life, it also is important to demonstrate the added value to the organization through project management methodologies, tools, and techniques. A metrics program then can be viewed as a necessary and basic business practice. Metrics can help identify important events and trends that can separate problems and opportunities and help guide organizations to informed decisions. Additionally, with organizations becoming more knowledge intensive, the importance of knowledge and skills accordingly is emphasized. Continuous improvement is a prerequisite for success. This means that organizations, in order to sustain competitive advantage, must be better informed; information is required on both the current state as well as the future direction. How do you know that your work in project management truly is making a difference and contributing to organizational success?
Metrics Categories
Metrics can be grouped into three categories: things, people, and enterprise. The metrics in each of these categories are a blend of individual indices and models.
Things Metrics
Things metrics are the most familiar metrics available for characterizing the success of the project. They describe the deliverable of the project and the efficiency with which it is being crafted. As a result, things attributes address the performance of the project team in tangible terms such as efficiency, productivity, and deliverables. The metrics for means and modes of the delivery include those dealing with managing elemental cost, project cost, and the project schedule. Therefore, things attributes include topics such as progress monitoring, procedure enhancement, historical data collection, and development of best practices. They are designed to quantify and measure cost, schedule, scope, quality, and the overall project success in meeting the client's needs. These metrics characterize tools that assess the progress and success of projects almost as if these processes run by themselves without any intervention by people. They measure the symptoms of the health of the project. Things metrics, thus, are the tangible signs of the implementation, and eventual success, of the project.
People Metrics
The success, however, of the things issues is a manifestation of the success of the project team members. Therefore, people attributes address team members' inter-relationships with one another, with a focus on teamwork issues such as trust, collaboration, competency, communication, and conflict management. People resources of the organization are intellectual resources. Yet, in many organizations, the corporate objectives for project metrics do not include people factors even though people make the project happen. A motivated team is able to successfully discharge continuous improvement duties in areas such as client satisfaction, flexibility, and productivity. Therefore, the purpose of people metrics is to identify opportunities for motivating team member behavior. They attempt to characterize and quantify the behavioral attributes of people, which by their very nature do not lend themselves to easy quantification. They include indicators of the existence of procedures for conflict management, communications, collaboration, teamwork, and technical competency. They also deal with the features of the enterprise environment that promote leadership, integrity, and professional responsibility. Primarily, project people metrics are intended to assess, directly or indirectly, whether team members are executing their tasks well. These metrics can be viewed as ones that measure the friendliness of the organization toward the project team and the team toward itself. By definition, organizations with higher levels of maturity have metrics that are designed to address people-related issues at both the project and organizational levels.
Enterprise Metrics
Enterprise metrics focus on the environment in which the project team must operate. The attributes in this category describe the organizational friendliness toward projects, involvement of project teams in organizational strategies, and recognition of the project management concept by the organization. These metrics show whether the organization is gaining a competitive advantage through the achievement of its strategic plan. It is an important point that, on the one hand, the organization provides the environment for the project, and that on the other hand, the organization is the direct beneficiary of the results of the success of the project.
Metrics and Project Management Maturity
The most effective program is one in which the metrics are tailored to the organization's important issues and strategic objectives. Metrics should link operations and projects to strategic goals so that everyone knows how they are contributing separately and together in meeting their strategic mission. Probably the most significant advantage of metrics is that they make explicit what is usually implicit in the decision-making process. They also help formalize the rationale for making decisions in order that the experience, knowledge, and wisdom of the experienced project managers would become readily available to all project management practitioners.
Metrics benefit organizations independent of where the organization ranks on a maturity scale, although in less mature organizations some metrics might seem inappropriate or superfluous. Organizations that have not made the transition to formalized project management, and thus can be classified as immature, are those that view technical expertise in the subject matter of the project a sufficient skill for becoming a project manager. These organizations have very few, if any, procedures and metrics that deal with project management. If metrics are available, they deal with project things such as cost, schedule, and scope.
Moving to slightly more sophisticated organizations, the organization might have the full suite of procedures and metrics available as they relate to project things. To put matters in perspective, in these organizations, it is generally accepted that if the things issues are handled and measured, reasonable success will be achieved. Then, moving to maturing organizations, the organization will have metrics and procedures that deal with project people as well as project things. The final stage of sophistication is achieved when the organization deals with issues that define the environment in which the project must operate or the enterprise issues. The premise is that projects will achieve their highest level of success if the organization is friendly toward, and supportive of, projects. Conversely, projects do not thrive, and possibly do not survive, in organizations that are hostile to projects. The former group of organizations, or those that are considered friendly, will have metrics and procedures that define all the three different sets of issues that relate to projects. It is an important point that, on the first glance, all organizations measure the success of projects using metrics that deal with things of the project such as cost, schedule, and scope. The subtle difference is to what extent the things issues are supported, enhanced, and elevated, by the strength in the areas of people issues and enterprise issues.
Implementing a Metrics Program
The goals of a metrics program can vary in urgency as determined by the strategic objectives of the organization. The sophistication of the metrics program, and its funding, will vary based on whether overall goals are to improve project-by-project performance, divisional/sector performance, or organizational performance. The objective is to institutionalize the concept that metrics are good and are not something to be avoided so that a culture is established throughout the organization where project professionals rely on and support the metrics that are being collected and even create and start using their own metrics. The metrics then become a part of each project professional's way of life. Project professionals view metrics in terms of how they can best contribute to the goals of the immediate project, to project management processes, and finally to organizational success.
Potentially, there are so many different things to measure about the project that one could become overwhelmed by opportunities. It is important to plan a metrics program by identifying the most important process and project management issues, selecting and defining the corresponding metrics, and integrating them incrementally into existing processes.
In order to facilitate the metric program's successful implementation, open communication about the metrics initiatives must be direct, consistent, and widespread. Whomever is responsible for the program should have strong communications skills and must balance the metrics program's needs with the participant's readiness to accept and embrace change. Each person's involvement and expectations in the program should be known and communicated. Every effort should be made in order that the implementation of the program does not become an organizational secret, because that will only breed notions of conspiracy, especially if the organization is undergoing any kind of downsizing, outsourcing, or reorganization. A culture change may be required to overcome inherent resistance to monitoring project performance and the wide reporting of results. In such cases the initial purpose of a metrics program might be to provide tools to project professionals to communicate the benefits of project management to senior executives and to operational personnel. Time must be set aside for meetings with stakeholders of the metrics system to discuss the benefits the results will bring to the organization. A number of briefings, and even workshops, may be required so that people throughout the organization recognize the purpose of the metrics system.
Implementation of a formalized metrics program must be treated like any other project, perhaps even more so to highlight the advantages of effective planning and monitoring. A detailed plan for the implementation of procedures and policies in using the results of the metrics program should be prepared. Measurement data must be incorporated into existing policies and procedures. The direct, or indirect, linkage between the metric, the project goals, and the organizational goals must be highlighted. The thresholds of the metric that define normalcy, that is the range of data that includes normal behavior, should be clearly defined. In addition to the list of values that describe normal performance, exceptional performance, and unacceptable performance, the definition of each index must include a corresponding plan of action for cases in which the threshold of tolerance for that index has been breached. Metrics should be assigned a priority ranking, and all metrics should not be treated equally. Some attributes that are easy to measure might not be particularly important. Other attributes that are more difficult to measure might be the most important. Project professionals should know which index has the most impact on their project's performance monitoring. Finally, the data collection process and frequency, results dissemination process, and feedback policies should be spelled out whenever a new metric is introduced and/or implemented.
As part of implementing the metrics program, one should identify the success criteria for the program, and for project management, in the organization. In tangible terms, the impact of the metrics system on the following must be identified: individual projects, programs, project portfolios, morale improvement, and operational benefits. Designers of the metrics program should be mindful that many items of data that project professionals collect will reflect some degree of subjective judgment. Furthermore, one would need to determine in realistic terms the status of the organizational environment. To characterize the organizational attitude toward change, one would need to evaluate whether the culture is project oriented or specialty driven. Current corporate practices must be detailed in order to determine to what extent there is tolerance for inaccurate project data and for project overruns. Executives must be interviewed in order to highlight the current corporate and business functions of the business units and the duties of the personnel within these units. Beyond that, knowledge areas that are most important should be identified, primarily because operation of the metrics program, and its constituent indices, should match the organization's strategic project management needs. Active involvement of the Project Management Office in the process is essential.
The metrics program should be designed specifically with an eye toward the collection of data for project and organizational success factors. The project metrics program might choose to begin its implementation with a focus on project things of only one project. Then, building on the success of this mini-implementation metrics system, data collection and reporting would then extend to project people and to more than one project. The metrics program next expands to include the attributes of the enterprise that encompasses all projects. Further, by conducting periodic assessments, the metrics program can benchmark the organization's current maturity level to the target maturity level and to industry standards of best practices. These interim assessments will chart a course for the full sophistication by identifying areas that are in need of improvement and by demonstrating effectiveness of the metrics system in achieving its goals.
Goldratt (1991) notes, “Tell me how you measure me, and I will tell you how I behave. If you measure me in an illogical way, do not complain about illogical behavior.” This implies that the measurements you take may cause people in the organization to behave in a given manner. Goldratt (1991) further warns, “Change my measurements to new ones that I don't fully comprehend, and nobody knows how I will behave, not even me.” Therefore, as organizational processes for a new suite of metrics are developed, earlier data collection processes must be reviewed in light of their verified utility to projects. It would be useful to identify those key pieces of information that are not included in the system, in order to devise means of incorporating them into future versions of the system. It also would be helpful if the implementation team validates and improves the planned processes, rather than starting to collect the metrics data immediately. Even then, it may take up to two years until the capture and analysis of data yield useful results.
During the early stages of the metrics program implementation, one must determine specific expectations for the organization concerning metrics and how they will be used. The plan should specify incremental progress milestones, such as first visible impact, intermediate significant achievement, and completion targets. A staged implementation plan, therefore, is recommended, with the overall process involving: first an establishment of the vision and strategy; second, the preparation of an execution plan, including a transition plan for the metrics program; and finally implementation of the program. The implementation team must conduct reviews of the program's effectiveness in light of continuous improvement initiatives, and the team must communicate the results and progress to all stakeholders.
Summary
Metrics bring consistency and formality to the project management practice. Through metrics, project decisions can be made on an informed basis. In essence, metrics bring objectivity to the tools by which the progress of projects, and the advancement of the organization, are monitored with consistency and repeatable accuracy.
Much of this paper is an adaptation from a book by Parviz F. Rad and Ginger Levin, titled Assuring Success with Metrics-Based Management that is due for publication by JRoss Publishers in 2004
Augustine, Thomas and Schroeder, Charles. An Effective Metrics Process Model. Crosstalk, June 1999
Goldratt, E. The Haystack Syndrome. North River Press, Croton-on-Hudson, NY, 1991.
