Project Management Institute

Validation of the project definition rating index (PDRI) tool for strategic oil storage tanks of Iran

Construction Risk Department, National Iranian Oil Co.

Introduction

Preproject planning is the project phase encompassing all the tasks between project initiation to detailed design. Preproject planning was defined as a phase in the project when owners and contractors realized that ability to influence overall project cost is greatest at the beginning of the project, when expenditures are relatively low, and decreases as the project progresses and expenditures become more significant.

The preproject planning phase is paramount in the project life cycle. Preproject planning is defined as the “process for developing sufficient strategic information with which owners can address risk and decide to commit resources to maximize the chance for a successful project” (CII, 1995). It begins when a validated project concept is developed during the business planning phase and ends with the decision to proceed with detailed design and construction. This decision is generally referred to as final authorization, at which time the appropriate funding is granted for execution of the project.

Over the past years, the Construction Industry Institute (CII) has funded several research projects focused on preproject planning. Findings from these investigations have dramatically changed the awareness of project management professionals within CII toward the importance of the process and the benefits of early project planning. Research results have shown that greater preproject planning efforts lead to improved performance on industrial projects in the areas of cost, schedule, and operational characteristics (Gibson & Hamilton, 1994; CII, 1995; Griffith & Gibson, 1995; Griffith et al., 1998).

Research has shown the importance of preproject planning on capital projects and its influence on project success. Findings of a recent study have proven that higher levels of preproject planning effort can result in significant cost and schedule savings.

Project Scope Definition

One of the major subprocesses of the preproject planning process is the development of the project scope definition package. Project scope definition is the process by which projects are defined and prepared for execution. The information identified during this process is usually presented in the form of a project definition package. A project definition package is a detailed formulation of a continuous and systematic strategy to be used during the execution phase of a project to accomplish the project objectives. This package should include sufficient supplemental information to permit effective and efficient detailed engineering to proceed (Gibson et al.,1993).

Success during the detailed design, construction, and start-up phases of a project is highly dependent upon the level of effort expended during the scope definition phase. Some construction industry officials consider lack of scope definition to be the most serious problem on construction projects (Smith & Tucker, 1983). A poorly defined project can experience considerable changes that may result in cost overruns and a greater potential for disputes. Inadequate scope definition can lead to changes that may delay the project schedule, cause rework, disrupt project rhythm, and lower the productivity and morale of the work force.

In a study of megaprojects (projects whose capital cost for completed construction exceeds $1 billion), which often experience significant cost growth and schedule slippage due to their sheer magnitude and complexity, poor definition was found to be the most important source of faulty estimates (Merrow, 1988). A primary recommendation resulting from this study was that the definition phase for megaprojects should be broadened substantially both in scope and detail to reduce project risk.

In addition to these studies, previous CII research has shown that scope definition is the highest ranking design input for construction projects on an industry-wide basis (Broaddus, 1995).

The Construction Industry Institute

CII is an organization that involves members that are either contractors or owners in the construction business, and has partnerships with several American universities. The main goal of CII is to develop best practices for the construction industry to improve the business effectiveness. By having involved representatives of the different sides in the industry, they ensure participation.

The PDRI for industrial projects is one of the different research topics that were promoted by CII in order to improve the scope definition level of industrial projects (Dumont, Gibson, & Fish, 1997).

Development of the PDRI

The CII research team sought to create three versions of the PDRI: one for industrial, one for building, and one for infrastructure projects. The research team decided to develop the industrial projects version of the PDRI first, as it best aligned with the majority of the members' expertise. The building projects and infrastructure projects versions were scheduled for future development.

Definition Process

To accomplish its objective of producing a simple and easy-to-use tool for measuring project scope development on industrial projects, the research team sought to identify and categorize the critical elements that should be included in an industrial project's scope definition package. To develop a detailed list of these required elements, the research team utilized four primary resources:

  • A literature review of previous scope definition research
  • Documentation from a variety of owner and contractor companies
  • The expertise of the research team members
  • A separate workshop of project managers and estimators.

A preliminary list of items to be included in the PDRI was obtained using documentation from both previous work and current industry practices. This list, which began with approximately 150 elements, was refined to a list of 70 using the research team's internal expertise. During the team's monthly meetings, brainstorming, nominal group techniques and affinity diagramming were all used to categorize and refine the list of elements (Table 1).

Table 1. PDRI Elements

I.     BASIS OF PROJECT DECISION
A.    Manufacturing Objectives criteria
A1. Reliability Philosophy
A2. Maintenance Philosophy
A3. Operation Philosophy
B.    Business Objectives
B1. Products
B2. Market Strategy
B3. Project Strategy
B4. Affordability/Feasibility
B5. Capacities
B6. Future Expansion Considerations
B7. Expected Project Life Cycle
B8. Social Issues
C.    Basic Data Research & Development
C1. Technology
C2. Processes
D.    Project Scope
D1. Project Objectives Statement
D2. Project Design Criteria
D3. Site Chars. Available vs. Required
D4.     Dismantling     and     Demolition
Requirements
D5. Lead / Discipline scope of work
D6. Project Schedule
E.    Value Engineering
E1. Process Simplification
E2. Design & Material Alternatives
E3. Design for Constructability Analysis
II. FRONT END DEFINITION
F.    Site Information
F1. Site Location
F2. Survey & Soil Tests
F3. Environmental Assessment
F4. Permit Requirement
F5. Utility Sources
F6. Fire Protection
G.    Process / Mechanical
G1. Process Flow Sheet
G2. Heat & Material Balances
G3. Piping & Instrument Diag. (P&ID)
G4. Process Safety Mangement (PSM)
G5. Utility Flow Diagram
G6. Specification
G7. Piping system Requirements
G8. Plot Plan
G9. Mechanical Equipment List
G10. Line List
G11. Tie-in List
G12. Piping Specialty item List
G13. Instrument Index
H.    Equipment Scope
H1. Equipment Status
H2. Equipment Location Drawing
H3. Equipment Utility Requirements
I.    Civil Structural & Architectural
I1. Civil / Structural Requirements
I2. Architectural requirements
J.    Infrastructure
J1. Water Treatment Requirements
J2. Loading / Storage Facilities Req.
J3. Transportation Requirements
K.    Instrument & Electrical
K1. Control Philosophy
K2. Logic Diagrams
K3. Electrical Area Classifications
K4. Substation Requirements
K5. Electrical Single Line Diagram
K6. Instrument & Electrical Specs.
III. EXECUTION APPROACH
L.    Procurement Strategy
L1. Identify Long Lead / Critical Eq.
L2. Procurement Procedures & Plans
L3. Procurement Resp. Matrix
M.   Deliverables
M1. CADD / Model Requirements
M2. Deliverables defined
M3. Distribution Matrix
N.    Project Control
N1. Project Control Requirements
N2. Project Accounting Requirements
N3. Risk Analysis
P. Project Execution Plan
P1. Own Approval Requirements
P2. Engr. / Constr. Plan & Approach
P3. Shut Down / Turn Around Requirements
P4. Pre-Commissioning Turn over
P5. Start up Requirements
P6. Training Requirements

Weighting Process

The writers knew that the 70 elements within the PDRI were not equally important with respect to their potential impact on overall project success. Therefore, it was decided that the elements needed to be weighted relative to each other to enhance their usefulness as a risk analysis tool. The method chosen to quickly develop reasonable and credible weights for the PDRI elements was to rely on the expertise of a broad range of construction industry practitioners marshaled together in workshops. The weighting development was therefore an inductive process in nature that incorporated expert input into developing final weights.

Assuming that scope development for the project had been completed, the workshop participants were instructed to apply what they felt to be an appropriate cost contingency to each element, given two circumstances—the element was undefined (level of definition 5), or it was completely defined (level of definition 1). The weighting was based on their opinions as to the relative impact that each element has on the overall accuracy of the project's total installed cost (TIC) estimate. All 70 elements were reviewed in this manner.

Oil Storage Tanks

Iran is one of the main oil producing countries in the world with producing of four million barrels per day. Oil storage tanks play a strategic role to have a continuous production rate because there are many unavoidable events that may threaten oil exporting.

Based on the these reasons, the planning department of National Iranian Oil Company (NIOC) decided to construct oil storage tanks with capacity of 10 million barrels:

  • Three million barrels steel oil storage tanks
  • Seven million barrels concrete oil storage tanks.

The author in this paper assessed and measured the level of scope development for concrete storage tanks. Concrete oil storage tanks have been located in two locations in south of Iran.

Three Million Barrels of Concrete Oil Storage Tanks

Four oil storage tanks have been located in “OMIDIEH” in south Iran that contains two 1 Million barrels and two 0.5 Million barrels. NIOC signed a contract with “Boland Payeh Company”. This company is a high reputable contractor in concrete structures in Iran. These oil storage tanks are round conical shaped and are buried fully in soil for protection against attack.

Four Million Barrels of Concrete Oil Storage Tanks

Six oil storage tanks have been located in “GOREH” besides of Khark Island Pump Station in south Iran that contains two 1 Million barrels and four 0.5 Million barrels. NIOC awarded this contract to a contractor called “Sabir” company. These storage tanks are cubic shaped.

Methodology

The methodology the author uses for assessing the scope of work and validating the PDRI for concrete oil storage tanks shares many aspects of the previous validation efforts made by Dumont and Gibson (1996). During those efforts, the process consisted of collecting data from various projects using an extensive questionnaire that was addressed to the people in the managing team of the projects. Similarly to what previous researchers had done, the author looked for concrete oil storage tank projects within the organization and that were being completed within a few years that would ensure the possibility to collect the necessary objective and subjective data required in the questionnaires.

The Questionnaire

The author used the same questionnaire that was used in previous research (Gibson & Dumont, 1996). The reason for that decision was that the questionnaire that was developed for CII was very generic, thus applicable to a wide range of projects so it collects all the necessary data to perform the validation analysis of the PDRI.

The questionnaire is divided into five different sections:

  1. Project background information
  2. General project information
  3. Schedule information
  4. Cost information
  5. Change information

Sections 1 and 2 gather the necessary data to understand the nature of the project, its purpose, and its organizational structure. Sections 3, 4, and 5 ask for objective measures of success with several closed-ended questions and an open-ended question in sections 3 and 5.

Variables Used

Out of all the questions asked in the questionnaire only a handful proved relevant for performing the statistical analysis. The reasons for discarding most of the questions for statistical analysis are that the questions were not strictly applicable to the data available in the way they were formulated. Let's take the questions regarding cost information, for example. Not all organizations use the same criteria to allocate the cost packets into the different groups. This translates into the difficulties that represent tracking every cost category and reorganizing them into the groups suggested by the questionnaire.

Some of the questions included in the questionnaire are more geared toward providing the researchers a more accurate idea on the project characteristics and increasing their understanding of the project rather than providing data to be processed statistically. The main reason is that some questions are not accurate enough for that purpose.

Gibson and Dumont (1996) found in a bibliographical review that the most cited measures of project success are technical performance efficiency of project execution (the ability to meet cost and schedule targets). Nonetheless, they proposed measuring success based on financial performance. In any case this research does not intend to assert which is the most accurate or valid measure of project success, but rather to find out which one is most useful to understand PDRI scores.

In conclusion, the questions that were finally used for further statistical study were as follows:

  • Planned construction duration
  • Actual construction duration
  • Planned total project costs
  • Actual total project costs
  • Number of change orders.

The data collected from the four first questions had to be mathematically treated in order to remove the influence that the size of the projects would exercise over the sample. The operation performed was to subtract the planned value from the actual value and divide the result by the planned value. See expression in Equation 1.

Non Dimensional Equation

Equation 1. Non Dimensional Equation

This operation yields a non dimensional value that can be either positive, negative, or zero. When the result is zero then there is no variation between the plans and the actual implementation of the project. When the result is positive, it means that there has been an increase in cost or in schedule, and the absolute value represents the percentage of the increase relative to the planned value. When the result is negative, it means that there has been a reduction in cost or in schedule, and the absolute value represents the percentage of the reduction relative to the planned value.

After performing this operation, the author obtained three different variables to measure project performance and found a relationship between PDRI scores and project success. The variables are listed in Table 2.

Table 2. Variables Used for Measuring Project Performance

img

Interviews

The author worked closely with the Assistant Director of Petroleum Engineering & Development Company (PEDEC), one of subsidiaries of NIOC, as the storage tanks client to identify potential projects to be used for the study and to contact the project leaders that would be interviewed to collect the necessary data for the study. As soon as the author was given the contact information for each of the project directors, an e-mail was sent to request an appointment for an interview.

The goal of the interviews was to present the objectives of the research to the project directors that were to provide the necessary data, and to introduce the questionnaire. In this way, the author was able to quickly dispel doubts on several questions and at the same time engaged the interviewees to help in the development of the research. In some interviews the project director showed satisfaction and immediately after browsing the questions said that he would gladly complete the questionnaire and send it back to the author. In other cases the project director would try to satisfy the data required on the spot as much as possible. However, in all cases some of the questions required checking the archives, which always implied that the interview had to finish without thorough completion of the questionnaire.

In certain cases the questionnaire was returned to the author with some fields incomplete. If that happened the project director was immediately contacted by email and asked to review the missing data. If he declared himself unable to provide the required information he would suggest another member of the managing team that would likely be able to provide the required information. In those particular situations, the data collection process slowed down considerably. Another factor that slowed down data collection was the time constraints of the project directors. Because of the huge effort being made by the Department of Facilities, the time availability of its project directors is very little, so scheduling interviews was difficult.

Statistical Analysis

The scatter plot is a statistic tool used to represent data graphically and to identify, in an easy way, relationships that numbers could otherwise hinder. It is normally used to represent two different sets of data and to show relationships between them.

The regression analysis is yet another statistical tool used when correlation between two or more sets of data exists. Its most common use is to infer the equation of the line that approximates the relationship between data. However, it can also be used to infer equations that are not necessarily linear. All the statistical operations and tools previously described are done for this thesis using the software Maple. The data analyzed include the following:

  • The PDRI scores for every project
  • The performance variable related to cost
  • The performance variable related to schedule
  • The performance variable related to change orders.

Table 3. Values of the Performance Variables Used and PDRI Scores for Every Project

  Projects Cost
Overrun (%)
Schedule
Overrun (%)
Change
Orders
PDRI Score
OMIDIE Two 1 Million Bar. 42.34 38.89 107 537
Two 0.5 Million Bar. 34.62 34.62 83 471
GOREH Four 0.5 Million Bar. 14.40 30 35 377
Two 1 Million Bar. 18.33 36.11 62 438

Scatter Plots and Correlation Factors

This section provides another type of graphical representation of the data. Now the purpose is to observe how each of the performance variables confronts to the PDRI ratings. To accomplish this author employs bivariate scatter plots with each of the pair of data using the PDRI ratings as the independent variable. Also, the author calculates the correlation factor existent to give a numerical value to any possible correlation.

Scatter Plots Showing Cost Performance Variable and the PDRI Scores

Figure 1. Scatter Plots Showing Cost Performance Variable and the PDRI Scores

Scatter Plots Showing Schedule Performance Variable and the PDRI Scores

Figure 2. Scatter Plots Showing Schedule Performance Variable and the PDRI Scores

Scatter Plots Showing Number of Change Orders and the PDRI Scores

Figure 3. Scatter Plots Showing Number of Change Orders and the PDRI Scores

The plots in Figures 1, 2, and 3 represent the scatter plots of PDRI scores versus the cost, schedule, and change orders variables. In these plots we can identify clearly a linear tendency of the four points.

Regression Analysis

This section presents the regression analysis that the author performed to correlate the PDRI ratings with the projects' cost and schedule performances. The previous section showed that a strong linear relationship exists between the two variables. Therefore, it is interesting to write an equation for that relationship and study its characteristics.

Linear Regression of the Cost-PDRI Correlation

Equation 2. Linear Regression of the Cost-PDRI Correlation

This equation is represented in Figure 4 with the points of the data:

Linear Regression of the Relationship Between Cost and PDRI

Figure 4. Linear Regression of the Relationship Between Cost and PDRI

One particular interesting point to study in this case is the 0 cutting point, because that point defines the definition level at which the project will presumably be completed on budget. For the linear model that point is approximately PDRI = 250.

The following equation presents schedule variation related to PDRI scores.

Linear Regression of the Scheduled Correlation

Equation 3. Linear Regression of the Scheduled Correlation

This equation is represented in Figure 5 with the points of the data:

Linear Regression of the Relationship Between Schedule and PDRI

Figure 5. Linear Regression of the Relationship Between Schedule and PDRI

Another important point to study in this case is the 0 cutting point, because that point defines the definition level at which the project will presumably be completed on time. For the linear model that point is approximately PDRI = 220.

Critical Analysis of Results and Comparison

The results presented in this paper differ substantially from those presented by previous researchers on this topic (Gibson & Dumont, 1996). The author mentions that the criteria used to generate the weights of the different elements in the PDRI score list are essentially economic. The author also recalls that the relationship established by the model between the different levels of element definition and the corresponding weights is forced to be linear by definition. From these two basic model characteristics, one would expect in the validation process to find a linear relationship between total project scores and success based on economic data. This can be explained because of the way the total score is computed. It is done by the simple addition of the scores provided by each single element of the list. Given the property of additively consisting of the fact that linear functions added together result in another linear function, it is possible to infer that the scores that a project is given by the PDRI are linearly related to the necessary contingency to cover the risk of in definition in the project.

On the self-critique exercise the author must say that in order to accomplish a good linear relationship representation had to discard one observation from the sample whose contribution was doubtful to the goodness of the dataset. However, enough arguments were presented supporting that decision.

The other observation to make is that the PDRI does not solve all the problems that may arise during the development of a project. Out of the models proposed in this paper, the author prefers the linear model because of the qualitative value it brings to demonstrate the principles of the development of the PDRI.

However, the quadratic model also adds an important feature as it limits the possible cost reduction to a credible value. In addition, this quadratic model can be eventually approached by a linear one on the higher spectrum of the PDRI range of values.

Conclusion

The work presented in this paper has made interesting findings related to the managerial tool for industrial projects named PDRI. The three main objectives have been accomplished satisfactorily.

Review of Objectives

Thanks to the critical analysis previously made, it is now easier to understand the working principle of the PDRI. The PDRI scores of the projects relate to the cost performance of the project. This relationship appears to be linear. The reason for this relationship is entirely based on the definition of the tool itself. First, because of the criteria used by the researchers to weight all the 70 different elements that compose the PDRI list.

In this research the author presents evidences that this relationship exists, and there doesn't appear to be any other reason to explain it. Previous researchers made the hypothesis that PDRI scores could be related to other variables measuring project success.

Another accomplishment of the thesis is to enlarge the pool of projects used to validate the PDRI for industrial projects. Before the present work, and based on the literature, the PDRI had been validated using 23 different projects with a total cost of approximately $1404 million. This project contributes with four new projects that represent a total cost of more than $46.36 million. This increases of 18% in the number of projects and 4% in the cost of the projects surveyed.

Finally, the author concludes with a few guidelines for future use in oil and gas projects in Iran. Based on the results presented, the author estimates that the pre-planning effort of oil and gas projects should lead to definition levels scoring 220 or lower in the PDRI to be on schedule and 250 or lower to be on budget for industrial projects. This is considered a good target for this reason that the two regressions performed in the analysis yield equations that cross the zero variation line in schedule at point near the 220 and in cost at point near 250 values.

The author emphasizes that the findings based in the regressions do not have to be valid for every organization. The analysis was performed using exclusively projects developed under the ownership of NIOC, and therefore results can only be expected valid for NIOC projects. However, the methodology presented should be applicable to any organization willing to use the PDRI to assess the definition level of their projects. This methodology also takes advantage of the little variation in the perturbations that may affect the sample used (i.e., location, market conditions, owner experience, etc.), given that all the projects were developed by the same owner, in a short period of time and in nearby locations.

Recommendations

The author strongly recommends the use of the PDRI to assess the managing team of a particular project. This recommendation becomes even stronger if the team belongs to an organization developing several projects, thus having the opportunity to validate the PDRI using the methodology presented in this thesis. In this sense, it is crucial to maintain the linear characteristics of the tool by not rounding the interpolated values of the weights used. It is also very important not to group data values into discrete groups, thus reducing the richness of the data collected. There are no proofs of these actions being pernicious, but by avoiding them the author has achieved very good results.

The author in this paper used the original weighted PDRI score sheet by Gibson and Dumont (1995) but it is recommended to form some workshops and invite owners and contractors in each organization in oil and gas industry of Iran to define new elements and reweight the original elements. If this happens, organizations can trust to the results more than before.

Another recommendation the author makes is to be very critical whenever using the PDRI to score a project. Because the people evaluating the project with the PDRI are the members of the managing team, it becomes a self-evaluation process, in which honesty is critical to yield good results.

Finally, the author wants to express his interest that a similar PDRI tool be developed for infrastructure projects. Industrial projects and building projects already enjoy the existence of this tool, and by creating the PDRI for infrastructure projects, all the different fields of the construction industry could benefit from this useful tool.

References

Dumont, Peter R., Gibson, G. Edward & Fish, John R. (1997). Scope management using project definition rating index. Journal of Management in Engineering 13(5), 54–60.

Freeman, Mark, & Beale, Peter. (1992). Measuring project success. Project Management Journal, 23(1),, 8–16.

Gibson, G. Edward, Cho, Chung–Sunk, & Wang, Y. (2006, Jan.). What is preproject planning, anyway? Journal of Management in Engineering, 35–42.

Gibson, G. Edward, & Dumont, Peter R.(1995, December). Project definition rating index (PDRI) for industrial projects. Implementation Resource 155 – 2 Construction Industry Institute, University of Texas at Austin,

Gibson, G. Edward, Dumont, Peter R., & Griffith, A. (1995). Preproject planning tools. Construction Industry Institute, Annual Conf., Austin, Texas.

Gibson, G. E., & Hamilton, M. R. (1994). Analysis of pre-project planning effort and success variables for capital facility projects. Report Prepared for Construction Industry Institute, University of Texas at Austin, Austin, Texas.

Haynes, M. E. (1996). Project management. Menlo Park, CA: Crisp Publications, Inc.

Lewis, J. P. (1995). Project planning, scheduling & control. Chicago: IRWIN Professional Publishing.

McNeil, H. J., & Hartley, K. O. (1986, March). Project planning and performance. Project Management Journal, 17(1), 36-43.

Project Management Institute (1996). A guide to the project management body of knowledge (PMBOK® guide). Newtown Square, PA: Project Management Institute.

Signore, A. (1985, Sept.). Conceptual project planning from an owner's perspective. Project Management Journal, 16(4), 52-58.

Wang, Y. (2006, Jan.). Applying PDRI in project risk management. PhD thesis, University of Texas, Austin, Tex, Journal of Management in Engineering.

Woolshlager, L.C. (1986, Aug.). Scope management. Project Management Journal, 37-42.

This material has been reproduced with the permission of the copyright owner. Unauthorized reproduction of this material is strictly prohibited. For permission to reproduce this material, please contact PMI or any listed author.

© 2008 Project Management Institute

Advertisement

Advertisement

Related Content

  • ¿Por qué las grandes ideas fallan y cómo evitar que esto suceda?

    By Vargas, Ricardo Viana | Conforto, Edivandro Carlos | Oumarou, Tahirou Assane Para reducir las tasas de fracaso e implementar estrategias con éxito a partir del 2020, las organizaciones deben superar las fuerzas disruptivas y convertir los desafíos en oportunidades y ventajas…

  • Por que grandes ideias falham e como garantir de que isso não ocorra

    By Vargas, Ricardo Viana | Conforto, Edivandro Carlos | Oumarou, Tahirou Assane Para reduzir as taxas de falhas e ser bem-sucedido ao oferecer estratégias em 2020 e além, as organizações devem superar as forças disruptivas e transformar desafios em oportunidades e vantagens.…

  • Sponsored Research

    Dynamics of New Business Creation

    By Kock, Alexander | Gemünden, Hans Georg | Kaufmann, Carsten The aim of this research project was to explore how innovative firms create new growth paths, and how project management practices can contribute. In addressing this question, we wanted to use a new…

  • Why Great Ideas Fail and How to Make Sure They Don't

    By Vargas, Ricardo Viana | Conforto, Edivandro Carlos | Oumarou, Tahirou Assane To reduce failure rates and successfully deliver strategies in 2020 and beyond, organizations must overcome disruptive forces and flip posing challenges to opportunities and advantages. It all…

  • Pulse of the Profession

    Tomorrow's Teams Today

    By PMI PMI research reveals that it’s the organizations with creative, collaborative and agile teams that will thrive in The Project Economy.

Advertisement