Validation of the project definition rating index (PDRI) tool for strategic oil storage tanks of Iran
Construction Risk Department, National Iranian Oil Co.
Preproject planning is the project phase encompassing all the tasks between project initiation to detailed design. Preproject planning was defined as a phase in the project when owners and contractors realized that ability to influence overall project cost is greatest at the beginning of the project, when expenditures are relatively low, and decreases as the project progresses and expenditures become more significant.
The preproject planning phase is paramount in the project life cycle. Preproject planning is defined as the “process for developing sufficient strategic information with which owners can address risk and decide to commit resources to maximize the chance for a successful project” (CII, 1995). It begins when a validated project concept is developed during the business planning phase and ends with the decision to proceed with detailed design and construction. This decision is generally referred to as final authorization, at which time the appropriate funding is granted for execution of the project.
Over the past years, the Construction Industry Institute (CII) has funded several research projects focused on preproject planning. Findings from these investigations have dramatically changed the awareness of project management professionals within CII toward the importance of the process and the benefits of early project planning. Research results have shown that greater preproject planning efforts lead to improved performance on industrial projects in the areas of cost, schedule, and operational characteristics (Gibson & Hamilton, 1994; CII, 1995; Griffith & Gibson, 1995; Griffith et al., 1998).
Research has shown the importance of preproject planning on capital projects and its influence on project success. Findings of a recent study have proven that higher levels of preproject planning effort can result in significant cost and schedule savings.
Project Scope Definition
One of the major subprocesses of the preproject planning process is the development of the project scope definition package. Project scope definition is the process by which projects are defined and prepared for execution. The information identified during this process is usually presented in the form of a project definition package. A project definition package is a detailed formulation of a continuous and systematic strategy to be used during the execution phase of a project to accomplish the project objectives. This package should include sufficient supplemental information to permit effective and efficient detailed engineering to proceed (Gibson et al.,1993).
Success during the detailed design, construction, and start-up phases of a project is highly dependent upon the level of effort expended during the scope definition phase. Some construction industry officials consider lack of scope definition to be the most serious problem on construction projects (Smith & Tucker, 1983). A poorly defined project can experience considerable changes that may result in cost overruns and a greater potential for disputes. Inadequate scope definition can lead to changes that may delay the project schedule, cause rework, disrupt project rhythm, and lower the productivity and morale of the work force.
In a study of megaprojects (projects whose capital cost for completed construction exceeds $1 billion), which often experience significant cost growth and schedule slippage due to their sheer magnitude and complexity, poor definition was found to be the most important source of faulty estimates (Merrow, 1988). A primary recommendation resulting from this study was that the definition phase for megaprojects should be broadened substantially both in scope and detail to reduce project risk.
In addition to these studies, previous CII research has shown that scope definition is the highest ranking design input for construction projects on an industry-wide basis (Broaddus, 1995).
The Construction Industry Institute
CII is an organization that involves members that are either contractors or owners in the construction business, and has partnerships with several American universities. The main goal of CII is to develop best practices for the construction industry to improve the business effectiveness. By having involved representatives of the different sides in the industry, they ensure participation.
The PDRI for industrial projects is one of the different research topics that were promoted by CII in order to improve the scope definition level of industrial projects (Dumont, Gibson, & Fish, 1997).
Development of the PDRI
The CII research team sought to create three versions of the PDRI: one for industrial, one for building, and one for infrastructure projects. The research team decided to develop the industrial projects version of the PDRI first, as it best aligned with the majority of the members' expertise. The building projects and infrastructure projects versions were scheduled for future development.
To accomplish its objective of producing a simple and easy-to-use tool for measuring project scope development on industrial projects, the research team sought to identify and categorize the critical elements that should be included in an industrial project's scope definition package. To develop a detailed list of these required elements, the research team utilized four primary resources:
- A literature review of previous scope definition research
- Documentation from a variety of owner and contractor companies
- The expertise of the research team members
- A separate workshop of project managers and estimators.
A preliminary list of items to be included in the PDRI was obtained using documentation from both previous work and current industry practices. This list, which began with approximately 150 elements, was refined to a list of 70 using the research team's internal expertise. During the team's monthly meetings, brainstorming, nominal group techniques and affinity diagramming were all used to categorize and refine the list of elements (Table 1).
Table 1. PDRI Elements
|I. BASIS OF PROJECT DECISION|
|A. Manufacturing Objectives criteria|
|A1. Reliability Philosophy|
|A2. Maintenance Philosophy|
|A3. Operation Philosophy|
|B. Business Objectives|
|B2. Market Strategy|
|B3. Project Strategy|
|B6. Future Expansion Considerations|
|B7. Expected Project Life Cycle|
|B8. Social Issues|
|C. Basic Data Research & Development|
|D. Project Scope|
|D1. Project Objectives Statement|
|D2. Project Design Criteria|
|D3. Site Chars. Available vs. Required|
|D4. Dismantling and Demolition|
|D5. Lead / Discipline scope of work|
|D6. Project Schedule|
|E. Value Engineering|
|E1. Process Simplification|
|E2. Design & Material Alternatives|
|E3. Design for Constructability Analysis|
|II. FRONT END DEFINITION|
|F. Site Information|
|F1. Site Location|
|F2. Survey & Soil Tests|
|F3. Environmental Assessment|
|F4. Permit Requirement|
|F5. Utility Sources|
|F6. Fire Protection|
|G. Process / Mechanical|
|G1. Process Flow Sheet|
|G2. Heat & Material Balances|
|G3. Piping & Instrument Diag. (P&ID)|
|G4. Process Safety Mangement (PSM)|
|G5. Utility Flow Diagram|
|G7. Piping system Requirements|
|G8. Plot Plan|
|G9. Mechanical Equipment List|
|G10. Line List|
|G11. Tie-in List|
|G12. Piping Specialty item List|
|G13. Instrument Index|
|H. Equipment Scope|
|H1. Equipment Status|
|H2. Equipment Location Drawing|
|H3. Equipment Utility Requirements|
|I. Civil Structural & Architectural|
|I1. Civil / Structural Requirements|
|I2. Architectural requirements|
|J1. Water Treatment Requirements|
|J2. Loading / Storage Facilities Req.|
|J3. Transportation Requirements|
|K. Instrument & Electrical|
|K1. Control Philosophy|
|K2. Logic Diagrams|
|K3. Electrical Area Classifications|
|K4. Substation Requirements|
|K5. Electrical Single Line Diagram|
|K6. Instrument & Electrical Specs.|
|III. EXECUTION APPROACH|
|L. Procurement Strategy|
|L1. Identify Long Lead / Critical Eq.|
|L2. Procurement Procedures & Plans|
|L3. Procurement Resp. Matrix|
|M1. CADD / Model Requirements|
|M2. Deliverables defined|
|M3. Distribution Matrix|
|N. Project Control|
|N1. Project Control Requirements|
|N2. Project Accounting Requirements|
|N3. Risk Analysis|
|P. Project Execution Plan|
|P1. Own Approval Requirements|
|P2. Engr. / Constr. Plan & Approach|
|P3. Shut Down / Turn Around Requirements|
|P4. Pre-Commissioning Turn over|
|P5. Start up Requirements|
|P6. Training Requirements|
The writers knew that the 70 elements within the PDRI were not equally important with respect to their potential impact on overall project success. Therefore, it was decided that the elements needed to be weighted relative to each other to enhance their usefulness as a risk analysis tool. The method chosen to quickly develop reasonable and credible weights for the PDRI elements was to rely on the expertise of a broad range of construction industry practitioners marshaled together in workshops. The weighting development was therefore an inductive process in nature that incorporated expert input into developing final weights.
Assuming that scope development for the project had been completed, the workshop participants were instructed to apply what they felt to be an appropriate cost contingency to each element, given two circumstances—the element was undefined (level of definition 5), or it was completely defined (level of definition 1). The weighting was based on their opinions as to the relative impact that each element has on the overall accuracy of the project's total installed cost (TIC) estimate. All 70 elements were reviewed in this manner.
Oil Storage Tanks
Iran is one of the main oil producing countries in the world with producing of four million barrels per day. Oil storage tanks play a strategic role to have a continuous production rate because there are many unavoidable events that may threaten oil exporting.
Based on the these reasons, the planning department of National Iranian Oil Company (NIOC) decided to construct oil storage tanks with capacity of 10 million barrels:
- Three million barrels steel oil storage tanks
- Seven million barrels concrete oil storage tanks.
The author in this paper assessed and measured the level of scope development for concrete storage tanks. Concrete oil storage tanks have been located in two locations in south of Iran.
Three Million Barrels of Concrete Oil Storage Tanks
Four oil storage tanks have been located in “OMIDIEH” in south Iran that contains two 1 Million barrels and two 0.5 Million barrels. NIOC signed a contract with “Boland Payeh Company”. This company is a high reputable contractor in concrete structures in Iran. These oil storage tanks are round conical shaped and are buried fully in soil for protection against attack.
Four Million Barrels of Concrete Oil Storage Tanks
Six oil storage tanks have been located in “GOREH” besides of Khark Island Pump Station in south Iran that contains two 1 Million barrels and four 0.5 Million barrels. NIOC awarded this contract to a contractor called “Sabir” company. These storage tanks are cubic shaped.
The methodology the author uses for assessing the scope of work and validating the PDRI for concrete oil storage tanks shares many aspects of the previous validation efforts made by Dumont and Gibson (1996). During those efforts, the process consisted of collecting data from various projects using an extensive questionnaire that was addressed to the people in the managing team of the projects. Similarly to what previous researchers had done, the author looked for concrete oil storage tank projects within the organization and that were being completed within a few years that would ensure the possibility to collect the necessary objective and subjective data required in the questionnaires.
The author used the same questionnaire that was used in previous research (Gibson & Dumont, 1996). The reason for that decision was that the questionnaire that was developed for CII was very generic, thus applicable to a wide range of projects so it collects all the necessary data to perform the validation analysis of the PDRI.
The questionnaire is divided into five different sections:
- Project background information
- General project information
- Schedule information
- Cost information
- Change information
Sections 1 and 2 gather the necessary data to understand the nature of the project, its purpose, and its organizational structure. Sections 3, 4, and 5 ask for objective measures of success with several closed-ended questions and an open-ended question in sections 3 and 5.
Out of all the questions asked in the questionnaire only a handful proved relevant for performing the statistical analysis. The reasons for discarding most of the questions for statistical analysis are that the questions were not strictly applicable to the data available in the way they were formulated. Let's take the questions regarding cost information, for example. Not all organizations use the same criteria to allocate the cost packets into the different groups. This translates into the difficulties that represent tracking every cost category and reorganizing them into the groups suggested by the questionnaire.
Some of the questions included in the questionnaire are more geared toward providing the researchers a more accurate idea on the project characteristics and increasing their understanding of the project rather than providing data to be processed statistically. The main reason is that some questions are not accurate enough for that purpose.
Gibson and Dumont (1996) found in a bibliographical review that the most cited measures of project success are technical performance efficiency of project execution (the ability to meet cost and schedule targets). Nonetheless, they proposed measuring success based on financial performance. In any case this research does not intend to assert which is the most accurate or valid measure of project success, but rather to find out which one is most useful to understand PDRI scores.
In conclusion, the questions that were finally used for further statistical study were as follows:
- Planned construction duration
- Actual construction duration
- Planned total project costs
- Actual total project costs
- Number of change orders.
The data collected from the four first questions had to be mathematically treated in order to remove the influence that the size of the projects would exercise over the sample. The operation performed was to subtract the planned value from the actual value and divide the result by the planned value. See expression in Equation 1.
Equation 1. Non Dimensional Equation
This operation yields a non dimensional value that can be either positive, negative, or zero. When the result is zero then there is no variation between the plans and the actual implementation of the project. When the result is positive, it means that there has been an increase in cost or in schedule, and the absolute value represents the percentage of the increase relative to the planned value. When the result is negative, it means that there has been a reduction in cost or in schedule, and the absolute value represents the percentage of the reduction relative to the planned value.
After performing this operation, the author obtained three different variables to measure project performance and found a relationship between PDRI scores and project success. The variables are listed in Table 2.
Table 2. Variables Used for Measuring Project Performance
The author worked closely with the Assistant Director of Petroleum Engineering & Development Company (PEDEC), one of subsidiaries of NIOC, as the storage tanks client to identify potential projects to be used for the study and to contact the project leaders that would be interviewed to collect the necessary data for the study. As soon as the author was given the contact information for each of the project directors, an e-mail was sent to request an appointment for an interview.
The goal of the interviews was to present the objectives of the research to the project directors that were to provide the necessary data, and to introduce the questionnaire. In this way, the author was able to quickly dispel doubts on several questions and at the same time engaged the interviewees to help in the development of the research. In some interviews the project director showed satisfaction and immediately after browsing the questions said that he would gladly complete the questionnaire and send it back to the author. In other cases the project director would try to satisfy the data required on the spot as much as possible. However, in all cases some of the questions required checking the archives, which always implied that the interview had to finish without thorough completion of the questionnaire.
In certain cases the questionnaire was returned to the author with some fields incomplete. If that happened the project director was immediately contacted by email and asked to review the missing data. If he declared himself unable to provide the required information he would suggest another member of the managing team that would likely be able to provide the required information. In those particular situations, the data collection process slowed down considerably. Another factor that slowed down data collection was the time constraints of the project directors. Because of the huge effort being made by the Department of Facilities, the time availability of its project directors is very little, so scheduling interviews was difficult.
The scatter plot is a statistic tool used to represent data graphically and to identify, in an easy way, relationships that numbers could otherwise hinder. It is normally used to represent two different sets of data and to show relationships between them.
The regression analysis is yet another statistical tool used when correlation between two or more sets of data exists. Its most common use is to infer the equation of the line that approximates the relationship between data. However, it can also be used to infer equations that are not necessarily linear. All the statistical operations and tools previously described are done for this thesis using the software Maple. The data analyzed include the following:
- The PDRI scores for every project
- The performance variable related to cost
- The performance variable related to schedule
- The performance variable related to change orders.
Table 3. Values of the Performance Variables Used and PDRI Scores for Every Project
|OMIDIE||Two 1 Million Bar.||42.34||38.89||107||537|
|Two 0.5 Million Bar.||34.62||34.62||83||471|
|GOREH||Four 0.5 Million Bar.||14.40||30||35||377|
|Two 1 Million Bar.||18.33||36.11||62||438|
Scatter Plots and Correlation Factors
This section provides another type of graphical representation of the data. Now the purpose is to observe how each of the performance variables confronts to the PDRI ratings. To accomplish this author employs bivariate scatter plots with each of the pair of data using the PDRI ratings as the independent variable. Also, the author calculates the correlation factor existent to give a numerical value to any possible correlation.
Figure 1. Scatter Plots Showing Cost Performance Variable and the PDRI Scores
Figure 2. Scatter Plots Showing Schedule Performance Variable and the PDRI Scores
Figure 3. Scatter Plots Showing Number of Change Orders and the PDRI Scores
The plots in Figures 1, 2, and 3 represent the scatter plots of PDRI scores versus the cost, schedule, and change orders variables. In these plots we can identify clearly a linear tendency of the four points.
This section presents the regression analysis that the author performed to correlate the PDRI ratings with the projects' cost and schedule performances. The previous section showed that a strong linear relationship exists between the two variables. Therefore, it is interesting to write an equation for that relationship and study its characteristics.
Equation 2. Linear Regression of the Cost-PDRI Correlation
This equation is represented in Figure 4 with the points of the data:
Figure 4. Linear Regression of the Relationship Between Cost and PDRI
One particular interesting point to study in this case is the 0 cutting point, because that point defines the definition level at which the project will presumably be completed on budget. For the linear model that point is approximately PDRI = 250.
The following equation presents schedule variation related to PDRI scores.
Equation 3. Linear Regression of the Scheduled Correlation
This equation is represented in Figure 5 with the points of the data:
Figure 5. Linear Regression of the Relationship Between Schedule and PDRI
Another important point to study in this case is the 0 cutting point, because that point defines the definition level at which the project will presumably be completed on time. For the linear model that point is approximately PDRI = 220.
Critical Analysis of Results and Comparison
The results presented in this paper differ substantially from those presented by previous researchers on this topic (Gibson & Dumont, 1996). The author mentions that the criteria used to generate the weights of the different elements in the PDRI score list are essentially economic. The author also recalls that the relationship established by the model between the different levels of element definition and the corresponding weights is forced to be linear by definition. From these two basic model characteristics, one would expect in the validation process to find a linear relationship between total project scores and success based on economic data. This can be explained because of the way the total score is computed. It is done by the simple addition of the scores provided by each single element of the list. Given the property of additively consisting of the fact that linear functions added together result in another linear function, it is possible to infer that the scores that a project is given by the PDRI are linearly related to the necessary contingency to cover the risk of in definition in the project.
On the self-critique exercise the author must say that in order to accomplish a good linear relationship representation had to discard one observation from the sample whose contribution was doubtful to the goodness of the dataset. However, enough arguments were presented supporting that decision.
The other observation to make is that the PDRI does not solve all the problems that may arise during the development of a project. Out of the models proposed in this paper, the author prefers the linear model because of the qualitative value it brings to demonstrate the principles of the development of the PDRI.
However, the quadratic model also adds an important feature as it limits the possible cost reduction to a credible value. In addition, this quadratic model can be eventually approached by a linear one on the higher spectrum of the PDRI range of values.
The work presented in this paper has made interesting findings related to the managerial tool for industrial projects named PDRI. The three main objectives have been accomplished satisfactorily.
Review of Objectives
Thanks to the critical analysis previously made, it is now easier to understand the working principle of the PDRI. The PDRI scores of the projects relate to the cost performance of the project. This relationship appears to be linear. The reason for this relationship is entirely based on the definition of the tool itself. First, because of the criteria used by the researchers to weight all the 70 different elements that compose the PDRI list.
In this research the author presents evidences that this relationship exists, and there doesn't appear to be any other reason to explain it. Previous researchers made the hypothesis that PDRI scores could be related to other variables measuring project success.
Another accomplishment of the thesis is to enlarge the pool of projects used to validate the PDRI for industrial projects. Before the present work, and based on the literature, the PDRI had been validated using 23 different projects with a total cost of approximately $1404 million. This project contributes with four new projects that represent a total cost of more than $46.36 million. This increases of 18% in the number of projects and 4% in the cost of the projects surveyed.
Finally, the author concludes with a few guidelines for future use in oil and gas projects in Iran. Based on the results presented, the author estimates that the pre-planning effort of oil and gas projects should lead to definition levels scoring 220 or lower in the PDRI to be on schedule and 250 or lower to be on budget for industrial projects. This is considered a good target for this reason that the two regressions performed in the analysis yield equations that cross the zero variation line in schedule at point near the 220 and in cost at point near 250 values.
The author emphasizes that the findings based in the regressions do not have to be valid for every organization. The analysis was performed using exclusively projects developed under the ownership of NIOC, and therefore results can only be expected valid for NIOC projects. However, the methodology presented should be applicable to any organization willing to use the PDRI to assess the definition level of their projects. This methodology also takes advantage of the little variation in the perturbations that may affect the sample used (i.e., location, market conditions, owner experience, etc.), given that all the projects were developed by the same owner, in a short period of time and in nearby locations.
The author strongly recommends the use of the PDRI to assess the managing team of a particular project. This recommendation becomes even stronger if the team belongs to an organization developing several projects, thus having the opportunity to validate the PDRI using the methodology presented in this thesis. In this sense, it is crucial to maintain the linear characteristics of the tool by not rounding the interpolated values of the weights used. It is also very important not to group data values into discrete groups, thus reducing the richness of the data collected. There are no proofs of these actions being pernicious, but by avoiding them the author has achieved very good results.
The author in this paper used the original weighted PDRI score sheet by Gibson and Dumont (1995) but it is recommended to form some workshops and invite owners and contractors in each organization in oil and gas industry of Iran to define new elements and reweight the original elements. If this happens, organizations can trust to the results more than before.
Another recommendation the author makes is to be very critical whenever using the PDRI to score a project. Because the people evaluating the project with the PDRI are the members of the managing team, it becomes a self-evaluation process, in which honesty is critical to yield good results.
Finally, the author wants to express his interest that a similar PDRI tool be developed for infrastructure projects. Industrial projects and building projects already enjoy the existence of this tool, and by creating the PDRI for infrastructure projects, all the different fields of the construction industry could benefit from this useful tool.
Dumont, Peter R., Gibson, G. Edward & Fish, John R. (1997). Scope management using project definition rating index. Journal of Management in Engineering 13(5), 54–60.
Freeman, Mark, & Beale, Peter. (1992). Measuring project success. Project Management Journal, 23(1),, 8–16.
Gibson, G. Edward, Cho, Chung–Sunk, & Wang, Y. (2006, Jan.). What is preproject planning, anyway? Journal of Management in Engineering, 35–42.
Gibson, G. Edward, & Dumont, Peter R.(1995, December). Project definition rating index (PDRI) for industrial projects. Implementation Resource 155 – 2 Construction Industry Institute, University of Texas at Austin,
Gibson, G. Edward, Dumont, Peter R., & Griffith, A. (1995). Preproject planning tools. Construction Industry Institute, Annual Conf., Austin, Texas.
Gibson, G. E., & Hamilton, M. R. (1994). Analysis of pre-project planning effort and success variables for capital facility projects. Report Prepared for Construction Industry Institute, University of Texas at Austin, Austin, Texas.
Haynes, M. E. (1996). Project management. Menlo Park, CA: Crisp Publications, Inc.
Lewis, J. P. (1995). Project planning, scheduling & control. Chicago: IRWIN Professional Publishing.
McNeil, H. J., & Hartley, K. O. (1986, March). Project planning and performance. Project Management Journal, 17(1), 36-43.
Project Management Institute (1996). A guide to the project management body of knowledge (PMBOK® guide). Newtown Square, PA: Project Management Institute.
Signore, A. (1985, Sept.). Conceptual project planning from an owner's perspective. Project Management Journal, 16(4), 52-58.
Wang, Y. (2006, Jan.). Applying PDRI in project risk management. PhD thesis, University of Texas, Austin, Tex, Journal of Management in Engineering.
Woolshlager, L.C. (1986, Aug.). Scope management. Project Management Journal, 37-42.
© 2008 Project Management Institute