Abstract
Within the construction industry, a great deal of effort is normally spent on measuring the traditional performance indices like cost and schedule where as the evaluation of the overall project performance is carried out in a less structured or subjective manner. An integrated framework for project performance measurement is required to formalize the way contractors evaluate performance of construction projects. This paper proposes a methodology that quantifies separately the performance of the major objectives of a project in order to measure the overall performance. The overall index is based on the measurement of eight project objectives; namely, cost, schedule, billing or cash flow, profitability, safety, quality, project team satisfaction, and client satisfaction. Implementing this set of performance objectives throughout the project construction phase will provide consistent information that will enable project managers to measure all aspects of performance against a quantitative and explicit set of targets. The proposed model integrates the eight dimensions of project performance into one overall index equation by assigning a priority or weight to each dimension. The Analytical Hierarchy Process (AHP) methodology is proposed to facilitate quantification of the weights in order to derive the overall project performance index. The proposed model will be able to draw the attention of management to poor performance in every dimension and will also lead to a more reliable and comprehensive performance benchmarking of projects.
Introduction
The objective of construction planning and controls, a basic project management function, is to ensure a well-coordinated and successful project. A basic element of planning is the set-up of objectives. The objectives will guide the many decisions made during the project's life. These decisions involve trade-offs between schedule, cost, quality, and other performance attributes. Effective monitoring of the progress of construction projects requires the integration and quantification of the various aspects of performance. The traditional performance indicators in the construction industry are completion time, cost, and quality. Most current project control systems measure quantitatively cost and schedule status and forget other major aspects of project performance like cash flow, profitability, quality, safety, project team satisfaction, and client satisfaction which are in some cases as important as cost and schedule. Very few project management systems quantify the later project attributes and they do so independently without proper integration to the overall project performance. The perception of failure and success of projects is usually based on personal indices and the experience of the project manager and it is not uncommon that two project managers would assess the performance of the same project using the same data differently (Rad, 2003). The disparity of judgment is mainly due to the lack of a clear and consistent evaluation procedures and methodology. There are many occasions where the project is under budget and progressing as scheduled. Yet it is considered a failure by upper management because of the low quality and safety performance records. Conversely, a project can be behind schedule and over budget and still be considered a successful one because it was completed with high quality, excellent safety record, and to the satisfaction of the client.
A great deal of effort is normally spent on accurately measuring some performance indices like cost and schedule where as the evaluation of the overall performance is carried out in a less structured or subjective manner. The objective of this paper is to present an evaluation model/methodology where construction practitioners can use to assess project performance during the construction phase. This research proposes a framework to integrate the project performance and formalize the evaluation process by introducing eight performance indices. These indices cover: cost, schedule, billing, profitability, quality, safety, project team satisfaction, and client satisfaction. Implementing this set of performance factors throughout the project construction phase will provide consistent information that will enable project managers to measure all aspects of performance against a quantitative and explicit set of targets.
The paper is divided into three sections. Section 1 presents the proposed project performance hierarchy and explains the significance of setting up and communicating project objectives. Quantification of the performance indices is presented under section 2. Section 3 outlines the advantages of the proposed integrated project performance evaluation model and finally a conclusion. The research methodology used in this study includes four steps: (1) identification of the project objectives and performance hierarchy; (2) quantification of the performance indices; (3) normalization of the indices; and (4) integration of the various performance indices to develop an overall project performance function.
Project Performance Hierarchy
The Construction Industry Institute (CII) Project Organization Task Force considers the objective-setting process as a critical element to the success of projects (Rowings, Nelson, & Perry, 1987). The same study indicated that on projects experiencing difficulties, the objectives lacked definition, clarity, and consistency. Identification, evaluation, and selection of the project objectives are the first and most important step in planning (Pinnell, 1980).
Objectives or Goals
Objectives are essential to the concept of project management (Pinnell, 1980). Objectives or goals provide the project management team a sense of direction by focusing attention on priorities. A structured goal hierarchy for a project:
- Provides an analytical platform for decisions and corrective action plans.
- Provides a clear and direct method of communicating objectives.
- Serves as a basis for project performance evaluation.
- Provides a rationale for the quantification of the overall project performance.
Without objectives it is difficult to measure results and performance against prior expectations and the project leader may not have any idea of whether the project is on the right track or not.
Because project objectives must be consistent with the policies and procedures of the organization, the objective setting process for construction projects is an extensive exercise that involves many functional departments within the contractor's organization. Some of the areas that are usually part of the objective setting process are: operations, quality, safety, cost/schedule control, human resources, and finance. Once the project objectives are set, sub-objectives are defined in order to track the variance in each main objective. This will enable management to monitor progress for any specific project objective during the project's construction.
In addition, executive management needs to support the project objectives and needs to motivate those who will achieve them. This is best accomplished by developing the project objectives at upper management level with input from the various functional areas of the company. This will ensure that the project objectives are in line with the overall company goals. During the execution phase, the project management team should review the performance indicators periodically, analyze any overruns, propose, and implement corrective actions. It is the ultimate responsibility of the project manager to make sure the project objectives are communicated and accomplished.
Communication of Objectives
Setting up a hierarchy of objectives and priorities for a construction project is necessary but not sufficient. The project objectives need to be communicated to all participants through a set of mechanisms. Rowings et al. (1987) identified two categories of mechanisms: primary and reinforcing. Primary mechanisms are used to directly communicate objectives to project participants and can include items such as:
- Scope of work
- Contract clauses
- Policies and procedures
- Written objectives and priorities.
Primary mechanisms are vital to project success, but alone, would not guarantee the success of a project. Reinforcing mechanisms will maintain focus and will support the communication of objectives and priorities in an indirect manner. These mechanisms give project leaders the opportunity to clarify the objectives. The following is some of the reinforcing mechanisms identified by Rowings et al. (1987):
- Weekly progress meetings
- Progress reports
- Safety reports
- Project instructions
- Cost and schedule reports
- Toolbox safety talks
- Upper management reviews.
The objectives of the project must be made known to all project personnel and team leaders at every level of the organization (Kerzner, 1989). If the project goals are not timely and accurately communicated, then it is entirely possible that functional managers and project leaders may all have a different understanding of the ultimate project objective, a situation that generates conflict among competing objectives.
Identification of Construction Performance Objectives
Most construction organizations look only at the time and cost parameters. If a schedule slippage or cost overrun occurs, then project managers will identify the cause of the variance. Looking only at time and cost performance might identify immediate contributions to profit, but will not tell whether or not the project itself was managed properly. Construction project success is often measured by the evaluation of three parties: the project team, the construction organization, and the client's organization. The assumption here is that a construction project cannot be considered successful unless it is recognized so by the three groups. This study presents a hierarchy of construction performance objectives that takes into account all success factors as viewed by the major players. The proposed goal hierarchy is systematic, and flexible enough to handle specific project requirements. The reader should realize that although project procedures can vary from project to project, project policies are usually similar in nature and do not differ between projects. Exhibit 1 depicts a project performance hierarchy that forms the structural foundation for a formal construction performance evaluation system.
Exhibit 1 – Hierarchy Design for the Project Performance Model
Quantification and Normalization of the Project Performance Indices
Before a construction company sets up the performance indices hierarchy, it is necessary to develop an understanding of the multi-dimensional nature of performance. Indicators of construction success must be identified, understood and agreed upon by the project management team. Each performance index needs to be: (1) quantified, (2) normalized or measured to a standard scale, and (3) prioritized. Quantification and normalization will be explained in the next sections, whereas prioritization, using the AHP, will be briefly discussed in this study.
Measurement of Project Success: A Challenge
Measurement of project success is a real challenge and quite a complex task. Performance measurement is also a must for all organizations executing any type of projects because if success cannot be measured, it cannot be improved upon. Some researchers have indicated that the task of measuring project success in solely objective terms is impossible (de Wit, 1986; Morris, 1986). The complexity of measurement of performance is due to the following facts:
- Project objectives are dynamic in nature and change over time.
- Many project participants representing various interests are involved in defining and prioritizing the project objectives.
- Some of the desirable objectives are subjective in nature.
Project Performance Indices
Traditionally, cost, schedule, quality, and safety are the objectives considered as the most critical to the success of construction projects. The proposed research identifies eight performance indices and presents a methodology to measure the overall performance index. The development of the eight indices to measure the success of projects evolved from the authors’ experience and from literature review. The performance indicators represent efficiency in terms of cost, time, billing or cash flow, profitability, safety, quality, team satisfaction, and client satisfaction. Each of these eight indices is quantitatively determined and transformed into a standard scale as will be shown as follows:
Cost Performance Index (CPI)
The Cost Performance Index (CPI) is a measure of the cost efficiency of the project. The CPI is determined by dividing the earned value by the actual costs incurred. Any value of CPI < 1 indicates that costs are overrun. For example, a CPI of 0.85 indicates that for every dollar spent; only 85 cents of value is earned and consequently 15 cents are lost. The CPI is given by:
Where,
BCWP = Budgeted Cost of Work Performed. It is the budgeted amount of cost for work-completed to-date or the cost allowed (based on budget) to be spent for the actual work done.
ACWP = Actual Cost of Work Performed. It is the cost incurred to complete the accomplished work to-date.
The values for the BCWP and ACWP used to calculate the CPI in the above equation are cumulative and include all project work up to the current data date.
The cost variance Vc, is the difference between what was earned (BCWP) and what was incurred (ACWP). For example, 50% of the project budget may have been expended to accomplish only 25% of the budgeted work. In this case, the project is over budget. VC is represented as:
A positive VC (CPI >1.0) is desired because it means that the actual cost of work performed is less than the budgeted cost of the same work and therefore the project is under budget. Critical variances are reported to management for further analysis and corrective action.
Because construction projects are unique in nature, performance-rating tables are unique to every project and must reflect the specific conditions and the cost control philosophy of the project. The cost rating table as shown in Exhibit 2 is proposed for illustration purposes only.
Exhibit 2 – Cost Performance Rating Table
Schedule Performance Index (SPI)
The Schedule Performance Index (SPI) is a measure of the schedule efficiency of the project; the SPI is determined by dividing the earned value by the scheduled value. Any value of SPI < 1 indicates that we are running behind schedule. The SPI is given by Equation 3:
Where,
BCWP = Budgeted Cost of Work Performed. It is the budgeted amount of cost for work completed to date.
BCWS = Budgeted Cost of Work Scheduled. It is the budgeted amount of cost for work scheduled to date.
The schedule variance VS, is the difference between what was done (BCWP) and what was planned (BCWS) and is represented by Equation 4:
A positive VS (SPI >1.0) is desired because it means that the actual amount of work performed is greater than the amount of work scheduled and the project is therefore ahead of schedule. The Schedule Rating Table is shown in Exhibit 3 to demonstrate the proposed methodology and it is up to each company to specify its own index ranges.
Exhibit 3 – Schedule Performance Rating Table
Billing Performance Index (BPI)
A critical factor for construction organizations to run a profitable business is their ability to carry out construction operations with minimal financing costs. The establishment of bank drafts is a common method of financing construction projects (Ahuja, 1976). At any period during construction, contractors may not be able to execute any work if cash is not available despite the obligation to abide by the schedule. Many project managers recognize the need to control the cost and schedule, but fail to monitor the cash flow status and how it can impact the overall project success. Project managers must understand the impact of correct and timely invoicing on cash flow and ultimately on project profitability. Many of the existing project management tools monitor cost, time, safety, and quality without considering the impact of cash flow on the ultimate project success. Some other tools monitor cash flow at the beginning of the project for financing purposes and at the end for auditing purposes. This research suggests that cash flow management be used as part of the integrated control mechanism and be monitored by project management, among other performance attributes, on an on-going basis.
In lump sum projects, contractors are typically paid based on their demonstrated percentage complete, together with the approved revenue (as stipulated in the contract) for the completed work. This is equivalent to the Earned Revenue of Work Performed (ERWP). In this context, the Billing Performance Index (BPI) measures the efficiency of invoicing the Client for the earned work. The BPI is determined by dividing the Billed Revenue by the Earned Revenue for the Work Performed. Submitting complete and timely invoices to the client enhances the project cash flow and minimizes the cost of borrowed money. The assumption here is that the project is a lump sum contract and that billing is based on the physical progress earned. The BPI is given by following expression.
Where,
BRWP = Billed Revenue of Work Performed, or the cumulative amount of invoices.
ERWP = Earned Revenue of Work Performed, or the revenue earned for the actual work accomplished to date.
A BPI value of 1.0 is desired because it means that the amount billed by the contractor covers all the work earned and the project is therefore efficient in billing the client. Because the contractor cannot bill more than what is earned, the maximum value of BPI is 1.0. Based on Equation 5, the Billing Variance (VB) is the difference between BRWP and ERWP. The Billing Performance Rating is shown in Exhibit 4.
Exhibit 4 – Billing Performance Rating and Normalization Table
Profitability Performance Index (PPI)
The Profitability Performance Index (PPI) is a measure of how profitable the project is to date. The PPI is determined by dividing the Earned Revenue of the Work Performed (ERWP) by the Actual Cost of the Work Performed (ACWP). The actual cost should be inclusive of all direct, in-direct and overhead costs incurred to date. At the end of the project, the PPI is indicative of the overall project profit and the ERWP will be equal to the total Contract Amount. The PPI is given by the following equation:
Where,
ERWP = Earned Revenue of Work Performed, or the revenue earned for the actual work accomplished.
ACWP = Actual Cost of Work Performed. It is the cost incurred to complete the accomplished work.
A PPI value greater than 1.0 is desired because it means that the revenue earned for the amount of work achieved to date is greater than the cost incurred for that same work and the project is therefore profitable. The PPI rating table is shown in Exhibit 5.
Exhibit 5 – Profitability Performance Rating and Normalization Table
To reiterate again, the objective of this study is to explain the methodology and every company must develop its own profitability targets and charts.
Safety Performance Index (SFI)
The Safety Performance Index (SFI), as proposed in this model, is a measure of how safe the site activities are carried out without lost time incidents. Maintaining an excellent safety record is vital to the project success and is considered to be one of the most important project performance indices. In almost all projects, the contractor and owner's business objectives place a strong emphasis on construction safety. In order to maintain a good reputation within the construction industry and to properly care for the safety and well being of the project staff and labor force, it is obvious that safety be a top business objective in any company.
In this research, the calculation used to determine the safety performance of projects is based on an industry-wide formula. Accordingly, the non-normalized SFI is the Lost Time Incident (LTI) Frequency Rate given by:
Where,
LTI = Number of Lost Time Incidents to date
M = Total man-hours expended to date; and
C = is a constant (200,000) which represents 100 employees working for a full year (100 x 2,000).
SFI is calculated for the project as a cumulative value to reflect to date safety status. Although every company should work toward the ultimate goal of “Zero Harm” and the elimination at source of any risks, a project safety rating scale is proposed in Exhibit 6 for illustration only.
Exhibit 6 – Safety Performance Rating and Normalization Table
Quality Performance Index (QPI)
The demand for high quality projects is on the rise throughout the construction market. Quality is a major project performance attribute that requires measurement and continuous improvement. A strong quality performance can have the following benefits:
- Enhances an organization's ability to market its services.
- Increases the client satisfaction and consequently the chances for repeat business.
- Reduces the amount of rework, and improves the effectiveness and efficiency of construction operations.
The Quality Performance Index (QPI) is a measure of consistency in the application of the Project Standards and Procedures as well as the compliance of the delivered product with the project specifications. Non-consistency in the application of project processes will lead to rework, poor quality audits and high number of Non Conformance Reports (NCRs). From the contractor's perspective, the QPI is best measured by the Construction Field Rework Index (CFRI), as defined in the pilot study for “Measuring and Classifying Construction Field Rework.” The study was carried out by the University of Alberta and presented to the “Construction Owners Association of Alberta (COAA) Field Rework Committee” (Fayek, Dissanayake, & Campero, 2003). The study defined field rework as:
“Activities in the field that have to be done more than once in the field, or activities which remove work previously installed as part of the project regardless of source, where no change order has been issued and no change of scope has been identified by the owner.”
The non-normalized QPI is given by Equation (8):
QPI = CFRI = Construction Field Rework Index, where:
QPI reflects the cumulative quality status. The project quality ratings table is proposed under Exhibit 7.
Exhibit 7 – Quality Performance Rating and Normalization Table
Team Satisfaction Index (TSI)
Human factors have a major impact on project quality and the successful completion of projects. The Project Team Satisfaction Index (TSI) is a measure of how satisfied the project team is. Building and sustaining high performing teams in today's competitive construction environment is a challenging task. Team members should support each other and communicate openly and clearly. Research conducted by the Construction Industry Institute Planning Research Team (CII, 1995) has established a clear link between teamwork and positive project performance. Many studies indicate that project team motivation is one of the top factors contributing to project success. Mohsini and Davidson (1992) maintained that inter-organizational conflicts in a construction project would negatively impact its performance. Developing a team atmosphere on a project is necessary for the project to be successful because the team members will work together towards the objectives (Rowings et al., 1987). Parker and Skitmore (2005) found that project management turnover occurs predominantly during the execution phase of the project and is mainly due to career and personal development and dissatisfaction with the organizational culture. The same study confirmed that turnover negatively impacts the performance of the project team, and consequently the project. Based on above, it is of paramount importance to regularly monitor and evaluate the performance of the project team and deal with team functioning problems as it is directly related to project performance.
The TSI is determined by calculating the earned rating for every area of concern to the team member based on his or her evaluation and the priority assigned to every area of concern. The priority weights can be either assumed by the project management team through consensus or measured, using some quantitative techniques, like the AHP methodology. The non-normalized TSI is given by:
Where,
W's = Relative weights for the various areas of concern.
R's = Ratings for the areas of concern on a scale from 1 to 10, 10 being the highest.
Based on discussions carried out by the author with team members in various construction projects, 12 areas of concern were identified and are listed in Exhibit 8.
Exhibit 8 – Project Team Members Satisfaction Rating Table
To normalize the calculated index, a project team satisfaction-rating scale is proposed in Exhibit 9.
Exhibit 9 – Team Satisfaction Performance Rating and Normalization Table
Client Satisfaction Index (CSI)
Meeting the expectations of the project owner (client) is the only way to ensure that a contracting company will continue to have repeat business. A formal survey or asking very basic questions could help us know better our clients. Because what gets measured gets done, it is important to measure the clients’ expectations against an established baseline. Sims and Anderson (2003) suggested eight steps, including quantification of expectations, which a contracting organization can use to maintain an on-going and working relationship with its clients.
In this research, the Client Satisfaction Index (CSI) evaluates the satisfaction of the Client's needs in a global sense. The CSI is determined by calculating the earned rating for every Client's area of concern based on the evaluation and the priority assigned by the Client to each area of concern. The areas of concern and their significance should be evaluated taking into consideration the client's specific objectives. The priority weights can be measured using the AHP process or using the subjective assessment of the project management team. This performance measurement will help the Project Leader to get client feedback in a structured manner and address any area of concern the customer might have. TSI and CSI are interdependent in the sense that ignoring the needs of the project team members makes it very difficult to create a desire within the team to care for the needs of the external customer. The non-normalized CSI is given by:
Where,
W's = Relative weights for the twelve areas of concern.
R's = Ratings for the areas of concern on a scale from 1 to 10, 10 being the highest.
Based on discussions carried out by the author with many client organizations and construction project owners, 12 areas of concern were identified and are listed in Exhibit 10.
Exhibit 10 – Client Satisfaction Rating Table
Once a formal Client Satisfaction Survey is completed, the Contractor should use it to propose mitigation actions if required. This feedback will help the construction company to continuously improve its work processes and services to its customers thus enabling the company to gain competitive edge over other contractors. Most often, informal “face-to-face” surveys of client satisfaction conducted by the Contractor's representative would not disclose the real situation and the client's answers tend to be diplomatic.
To normalize the obtained index, a client satisfaction rating scale is proposed in Exhibit 11. The proposed scale is for illustration only and needs to be modified to reflect the project specific conditions.
Exhibit 11 – Client Satisfaction Rating and Normalization Table
Project Performance Index (PI)
Controlling all of the above performance attributes defines the need for a multi-dimensional Integrated Project Performance Management system. To develop a useful index of project performance from the above results, a common measurement platform was established to normalize all the indices. Moreover, the classification of the performance variables into a common value scale made it possible to combine all eight indices into a performance index (PI) equation. Combining the variables identified with the corresponding weights yields a weighted equation for the total project performance. PI can be expressed in a linear additive form as follows:
Where and
CPI, SPI, BPI, PPI, SFI, QPI, TSI, and CSI are the normalized performance indices and can be calculated as defined earlier. W1 W8 are the respective priority weights or relative importance of each index with respect to the overall project PI.
Measurement of the project performance indices should take place at regular intervals, certainly monthly, but recommended to be weekly especially for short term or fast track projects. This is true for all indices except for the TSI and the CSI where measurement is not practical every week and can be assessed on a quarterly basis or whenever the project management team feels the necessity. These eight indices provide a wealth of data reflecting the true health of projects and assist the project management team to monitor, analyze, and initiate preventive measures if required.
Quantification of the Priority Weights
The application of the AHP Methodology, developed by Saaty (1982), is proposed to derive the priority weights (Ws) or relative importance of the indices. These weights will indicate the sensitivity of the outcome, or the overall PI, to the individual performance indices. At least three reasons support the use of AHP in this study: First, the ability of AHP to incorporate the qualitative and quantitative factors involved in project evaluation. Second, the structure of the project performance hierarchy is identical to the hierarchical design of AHP. Third, the ability of AHP to incorporate the experience and the knowledge of project managers to define weights. The reader is referred to Saaty (1982) for a detailed description of the weights quantification process using AHP.
Advantages of the Integrated Project Performance Evaluation Model (IPPM)
The proposed methodology provides a systematic and structured process to evaluate the performance of construction projects. The concept of project success is defined with respect to clear set of goals and objectives. Despite the complexities of measuring project performance, the concept was quantified into a meaningful index that is based on measured and objective data. The advantages of this research can be summarized as follows:
- Introduction of a tool that allows managers of construction projects to evaluate the performance of projects in a formal and systematic way. The model is very flexible and can be adapted to meet the specific requirements of any construction project.
- The consistent historical performance data can be utilized in future projects to improve planning and effective implementation of project execution processes.
- A composite performance index can provide an overall project performance status. In addition, by determining the individual performance indices, the project management team can effectively communicate to their staff the project status and future priorities in each area.
- The project performance equation will allow contractors to compare the success of two or more projects within their organization for benchmarking purposes. Also, they can compare the success in similar individual areas of performance.
Conclusion
Under the current levels of competition, projects are being implemented in complex, dynamic, and uncertain environments. As project management is getting more integrated, performance measurement of projects is expanding to include more aspects of performance. As a result, contractors who are project-oriented organizations need to use a unified performance measurement system that integrates all project objectives.
In this research, a framework for project performance measurement was developed to formalize the way contractors evaluate projects and assist project managers in controlling projects during the execution phase. The new methodology measures separately the performance of all the critical objectives of a project as well as the overall performance. The system will be able to draw the attention of management to poor performance in every dimension, and the project manager will be able to realize the extent of its impact on achieving the project objectives. The Integrated Project Performance Model combines all dimensions of project performance in one overall index equation by assigning a priority or weight to each dimension. The overall index is based on eight objective measurements of project performance: cost, schedule, billing, profitability, safety, quality, team satisfaction, and client satisfaction. These indices were considered of major significance and necessitated measurement and close monitoring by the project management team. The performance index proposed in this study can be implemented by any construction organization that needs to measure the success of its projects. The objective is neither to standardize the performance scales nor their priorities, but rather to establish a framework for a systematic and quantitative performance evaluation process for construction projects.