Performance measurement for projects and project management
J.N. Salapatas, P.E.
Assessing project management (or mismanagement) has become an important part of our professional challenge in business and industry today. The unexpected cost overruns and implementation delays of computer systems projects during the early 1970s and the disastrous cost overruns and schedule delays of the heavy construction projects in the late 1970s have given new meaning to the word performance for project management.
Management audits, construction audits and other forms of reviews and assessments of projects have become commonplace in the lives of professional project managers and consultants. But what has really changed? Is there anything innovative in the evaluation of project management performance?
This paper presents a brief overview of a very complex subject: performance measurement for projects and project management and will highlight answers to the following questions:
• How do you measure the performance of a project?
• How do you measure the performance of project management?
• Are they measured the same way?
• What is new in performance measurement for projects and project management?
But first …
Let us dispense with a mythical notion about the relations between performance and productivity.
• Do you believe that performance is equal to productivity, or that
• performance is not equal to productivity?
The productivity formula is output over input. Productivity involves the efficient and effective use of resources. Performance includes the following:
• Quality of work
• Quality of working life
Therefore, performance does not equal productivity because productivity is only a part of the performance picture.
How do you measure the performance of a project? All projects are expected to have specific objectives; that is, an end result, which costs so much and should be completed within a certain time-frame.
Therefore, projects which achieve cost, schedule and quality objectives are successful. Those that do not are failures. Success or failure is a simple measure of performance. For small simple projects, like building a bookcase or writing a budget program on your personal computer, this might work; but, the success-failure measurement is not very practical for most projects.
Experience has shown that project performance can be measured in terms as esoteric as the value of the project to the company president or as quantitative as earned value systems used on large government and utility projects. In every case someone identified elements or facts, smaller than the project, which indicated progress and then set out to watch them during the life of the project. We labeled the watching process as a monitoring system, and the elements or facts as key indicators.
Therefore, in basic terms, project performance can be measured with the help of a monitoring system and key indicators. Like all systems, a project monitoring system must begin with management commitment. The other components are the following:
• Management organization
• Responsibility definition
• Personnel assignments
• Clear objectives
• Measurable indicators
• Authentication of data
• Timeliness of reports
Key indicators are items which if monitored properly provide measurable assessment of project performance.
Experience has shown that most projects can be monitored effectively with a dozen key indicators. The first six are called project facts. Project management professionals know them by heart. They are as follows:
• Original cost
• Current vs. planned costs
• Final costs
• Current vs. planned percent complete
• Current vs. planned schedule
• Schedule completion date
Project status indicators are used for monitoring changes from one reporting period to another. What is significant is the trend of the deviation. Is it getting better or worse?
Physical percent complete, cash flow, and weeks ahead or behind the critical path describe the key indicators: completion, cost and schedule. However, the quality indicator uses several elements to highlight status. They are as follows:
• Inspection performed
• Rejected inspections
• Design changes
Project administration represents the management and control considerations of a project. This includes the following elements:
• Key controls
• New issues
• Resolved issues
• Internal audits
• Independent audits
Key controls are specific activities or procedures during which management performs checks, approves or makes decisions affecting project results. Establishment of key controls and periodic reviews for compliance provides a reasonable assessment as to whether or not the project procedures are being followed. An issue represents anything that can affect project results. It could be a specific problem about a schedule delay or a new design technique that will reduce costs. Keeping track of new and resolved issues demonstrates how well unplanned situations are being handled. Audits are important because they provide a third-party objective review of a project’s administrative and control process. Potential problem areas may be identified by audits before they become serious.
For an example let us look at the ZETA Project and the real world of projects. The ZETA Project is a multimillion dollar design and construct project with a 36-month schedule. The Project Monitoring Report (Exhibit 1) picks up with the 16th month. A quick review of this single page report follows: The Project Facts show that the ZETA Project has had a schedule and budget change and that it continues to fall behind both in dollars spent and percent complete.
Project Status (the key indicators) shows that:
Total Project (↓) monthly progress continues to be unfavorable despite a good last month. This was caused by unfavorable Construction (↓) progress. Quantity and Labor Cost Reports from the site confirmed that this resulted from shifting craft personnel to performing re-work instead of new construction. The re-work was necessary to overcome problems with concrete and welding Rejected Inspections (↓ - April through August).
Cash Flow (↑) shows favorable because less money was spent than planned. While this corresponds to the amount of progress behind plan, the budget(↓) shows unfavorable because, the funds will still be spent but in higher future dollars and with increased interest costs.
Key Controls (X) were not reported this month. Several New Issues (↓) concerning labor disputes have occurred this month. Resolved Issues(↓) regarding shortage of materials and structural steel misalignments are still pending. The recent Internal Audit (↓) revealed several overcharges by the mechanical subcontractor that were inadvertantly paid and a number of discrepancies in material receiving reports at the warehouse. No independent audit was scheduled this month.
How often should the project facts and project’s status be reviewed? It depends primarily on the size and duration of the project (i.e., for small projects, weekly; for larger projects, monthly).
Project Management Performance
How do you measure the performance of project management? Why not in the same way as the project? If the project is meeting expected performance objectives, can we not assume that the performance of project management is equally effective? When we examine the basic performance measurement needs of project management, we arrive at the same conclusion. Project management performance needs a monitoring system and key indicators. However, it is there that the similarity stops. Figuratively speaking, a project has a static outcome while project management is a dynamic process. The elements and facts identified within project management do not indicate progress because management functions are continuous and repetitive.
However, effectiveness can be measured; but, there are so many variables in the process of project management that it is very easy to miss a few. How should things be done? How are decisions being made?
For example, midway into a construction job, a concrete pour which met cost, schedule and quality objectives, but whose planning effort took too much time, adversely impacted the decision for purchasing the embedments and rebar. The planning delay resulted in a poor decision. How did I learn about the incident? I was at the construction site when it happened.
Can we afford to have someone at the site full-time to monitor performance or must we wait until the end of a project to discover about the performance of project management? Certainly we cannot wait until the end for large projects or those with long duration; but, even small projects can have costly problems if they are not monitored.
If we conclude that a monitoring system is needed, what do you monitor and how can decisions be evaluated? Typical process monitoring uses a model and control checks. Since project management is a process and decisions affect outcome, it seemed logical to apply similar analytical techniques. Thus, we found that project management could be monitored if a model existed and if there was a uniform system to analyze decisions.
There is an established field of knowledge concerned with decision analysis, so it was unnecessary to invent anything new. However, we clarified the definition of a decision as follows. A decision is making a choice in the absence of adequate information which would have made the choice obvious.
Since the beginning of project management, much has been written and discussed about what project management is and how it works. Presented below is a model which is the result of an extensive research study in the field of project management. More than 1,000 pieces of subject material, books, documents, articles and conference proceedings were reviewed for its development. It is disarmingly simple yet exhaustively comprehensive. It begins with the classical definition of management.
Plan, Execute and Control are three functions identified as fundamental to the classical management process. Plan includes determining how activities and events should be tied together to reach goals and objectives. It is the process of anticipating, scheduling and prioritizing necessary activities, and organizing and staffing with management and technical skills to achieve the overall objectives. Execute includes the specific actions which lead to achieving the goals and objectives. It involves both management personnel and technical skills alike, as both are required for successful execution. Control involves the establishment and use of mechanisms to monitor status; management reviews status and determines progress. Control is direction to people regarding the execution of specific actions.
Project management logically evolves from classical management, through the application of the concepts of classical management to a specific project. Project management parallels classical management by adhering to the fundamental Plan, Execute and Control functions, yet differs because of the necessary additions of Begin and End functions.
These additions are due to a project’s time-limited nature. Begin consists of determining the project goals and objectives and authorizing and contracting the project work. End includes the steps taken to complete the project and turn over the completed work to the owner. This includes final tests and inspections, final payment and project delivery.
To further describe the project management model, eighteen components were selected which encompass all aspects associated with project management systems. The significance placed on individual components may vary from project to project, yet it is essential that each component be presented to some degree in projects. These components are as follows:
• Goals and objectives
• Project authorization and contracting
• Overall planning
• Project organization and structure
• Systems and procedures
• Detailed project plan
• Estimating and budgeting
• Project-specific tasks
• Technology and productivity
• Data collection and reporting
• Coordination and communications
• Evaluation and reporting
(Cost control/performance measurement)
• Quality control
The project management model also includes three components which are considered ongoing throughout the life of the project. They are as follows:
• Quality assurance
• Records management
Project management is essential in integrating and coordinating the efforts of a project team, including all of the interdisciplinary skills and departments involved, so that the results are a cohesive project effort. The concept of project management is universally applicable to all projects, regardless of size or scope; however, the specific characteristics of any given project will determine which project functions must be emphasized in the project management model.
The project management model represents what project management is and how it is intended to work. This is especially useful in projects where considerable coordination is essential (i.e., those projects which cross many different interdisciplinary functional lines). An effective project management system will, among other things, continually monitor project status and provide an early warning system to detect out-of-bounds conditions, enabling changes and revisions to occur in a timely manner.
How often should the actual conduct of project management be reviewed against the model? How often should decisions be tested for effectiveness? Once again, this depends on the size of the project and its duration.
Perhaps a more important question should be — How do you use the project management model? Models are fine for students, professors and staff people. But, how practical is the project management model? How does a big, ugly project manager apply this paper pundit to the real world? To see how it works, let’s consider the ZETA project as an example.
In fact, let us focus on what appears to be one of the serious problems reported by project monitoring systems, the project schedule.
The project management model contains a scheduling component. It is a part of the Plan function. Scheduling is defined as the sequencing of activities which have start dates, completion dates and expected durations. The purpose of scheduling, claimed by the model, is to be able to create project schedules which show progessive sequence and relationships of work activities and events.
Of course, project professionals know all this, so why bother with the model? Because the model illustrates what should be in place for effective scheduling results. It helps the project manager or anyone else who may be evaluating the scheduling process by giving them a formal basis for comparision, that is, what should be, to what actually is happening. The evaluation is accomplished by looking at the model and asking questions such as:
Q – Does the ZETA project have a schedule?
A – Yes, of course!
Q – Does the ZETA project schedule contain activities and events which have start dates, completion dates and expected durations where appropriate?
A – Yes, most of them do.
Q – Are the ZETA activities and events displayed in such a manner which show progessive sequence and valid relationships?
A – Some do, some don’t.
Q – Does the ZETA project have a procedure for timely updating of the schedule?
A – Yes, we update monthly.
Q – Is the ZETA project schedule procedure being followed?
A – Pretty much so, but had some lapses about six months ago when we switched computer systems and had some problems with the software.
Q – Are procedures in place which ensure reasonable relationships between estimating and scheduling?
A – No, it’s not formal, but we do it anyway.
Q – Are ZETA project activities scheduled at an appropriate level of detail?
A – Sure, but it depends on the work. Some activites are a couple of days, maybe weeks or even a month or two.
Q – Does the ZETA project schedule correlate with the WBS?
A – What’s a WBS?
The questions and answers are real, only the names have been eliminated to protect the guilty. As you can see, the questions can help even the most expert of project professionals to focus quickly on the trouble spots. The answers to most questions can be obtained by interviews. But, on-site observations and verification of the facts are a must for reaching sound conclusions.
Having done all this, we concluded that the ZETA project, among other ailments, did not have an accurate project schedule. They were lax in timely production and updating of their project schedule. Their schedule activities were not consistent, but resource loaded, and suspect with regard to estimating accuracy.
Scheduling is only one component of the project management model, other components may be evaluated in the same manner using questions from the model to discover the existence and effectiveness of every important aspect of project management.
We have just reviewed the essential factors for measuring the performance of a project and of project management. It was suggested that the success-failure method is not practical for most projects because of the complex nature of project work and the almost infinite number of variables involved.
Monitoring systems and key indicators were described as necessary factors for measuring both projects and project management; but, we determined that it is incorrect to measure them in the same way because projects are inanimate things and project management is a people process.
Projects are measured by monitoring facts and status. Project management is measured by monitoring actual conduct vs. a model and quality of decision making.
Finally, we concluded that when stripped of all the “projectese” jargon, the real innovation is that we confirmed the basic principles of sound management practices for measuring performance. We simply put them in correct order for project management.
Project Monitoring Report Summary
Earning a scheduling certification helped three project professionals deliver more precise estimates — and become better leaders.