Evaluation of projects in the oil sector
analysis and results of a probabilistic approach
The aim of this study is to develop and analyse a project's economic assessment model, which outruns the deterministic method using “point estimate” values and which exploits a more accurate, probabilistic, and simulation-based approach capable of evaluating the uncertainties of the context in which the project is developed by maintaining the variability of the input parameters.
Starting from identifying and quantifying the uncertainty of each input parameter, in particular the oil production profile, it is possible to determine the variability of the project NPV and establish which parameters have the greatest impact on this value.
Crude oil production has begun to decline in recent years and despite the development of new, advanced technologies, projects to exploit existing reserves are becoming increasingly complex. This is particularly true as a result of the growing uncertainty regarding critical factors in the project, such as the amount of oil that can be extracted from the field, the time needed to implement the project, the capital invested, the selling price of the oil, and the operational costs of extraction and production.
In the oil business that has emerged over the last decades, based on ever-more-complex projects lasting decades and, above all, based on increasingly demanding financial commitments, a project management approach that is not well integrated can lead to insufficient overall performance in the face of market demands. The greater competition in a globalised market must be addressed through an integrated vision of the project, overlaying processes both in terms of time, varying the duration of activities in function of project progress (Pillai, Joshi, & Srinivasa Rao, 2002), and of the different disciplines. The goal is transfer and update the information acquired during development and including in the analysis the interaction of the project with the environment in which it is developed.
The company analysed is Eni Exploration and Production Division, one of the world's leading oil companies operating in the oil and natural gas, electrical energy generation, petro-chemicals, and engineering and contracting sectors.
To facilitate the project management process, Eni E&P has sought to structure and thus to subdivide the project life-cycle of a finite number of processes and phases, each with a specific aim and objective. In this way, it is possible to manage in an integrated way processes lasting tens of years within a life-cycle, which can be as long as 30 years.
The management of the investment project is divided into three main processes:
- Exploration & Appraisal: the choice of the area for future exploration is assessed in function of the possibility and probability of a find based on a certain number of fundamental geological elements (studies and research, knowledge of the area, assessment of the drilling risk), as well as economic considerations.
- Development: on the basis of the opportunities identified during the exploration process, different project plans are defined, together with the necessary production plants and support structures.
- Production: once development has been completed, production begins as the hydrocarbons are extracted from the field, treated in the plants, and sent directly to the market or to further transformation processes (such as a refinery) (Eni S.p.A. Exploration & Production Division, 2003).
Each process interacts with the preceding and subsequent activities and is founded on a structured set of roles, techniques, and procedures that must be respected. Each of these elements is subdivided into phases interlinked by gates that ensure the continuation of only projects considered realistic and where available resources are optimised. If projects are not economically viable, they can be abandoned at each process gate (Eni S.p.A. Exploration & Production Division, 2003).
The first step in integrating the phases of the investment project life-cycle is enlarging the project management focus in order to include among the processes listed above the opportunity analysis and research phase prior to the actual development of the project.
The literature (Fangel, 1993) also highlights the need to move from a view based on the management of quality, time, and resources within a given period of time to one that considers the entire life-cycle of the investment project, includes an analysis of the interaction of the project with the context in which it is developed, and looks at overall performance.
To be in a position to manage and interface these multiple aspects, project management must forecast the interaction between the various specialist functions that, in various roles, are involved in the development and realisation of the project (geologists, designers, analysts, financers, project managers and risk managers, among others).
By means of the integrated management of the project phases, it is possible to transfer information and knowledge from one phase to the next, thus progressively improving the accuracy of the estimates for project performance throughout the project life-cycle. At the same time, the uncertainty associated to the project in the early phases must also be taken in consideration when planning the subsequent phases.
Management of Uncertainty in the Decision-Making Process
In a situation such as the oil industry, which involves many elements of uncertainty, the economic assessment of an investment project plays a fundamental role in deciding whether to continue to the next phases.
This has led to introducing risk analysis into the project planning and control process, providing more realistic indications regarding project performance and then defining the actions necessary to realise the desired objectives.
As the uncertainty of future scenarios grows, the decision-makers become increasingly aware that during the project life-cycle the actual value of each critical parameter may differ significantly from the forecasted value. Consequently, the economic calculation becomes more and more unpredictable, both as a result of the instability in the dynamics within the sector in which the contractor operates, and because of the evolution of the external project environment, which makes profitability ever more uncertain.
The factors to pay particular attention to during development, assessment, and, ultimately, realisation of the project are as follows:
- The production profile of the oil-field, as it is not possible to forecast exactly the production resulting from the reserves
- The duration of the project, which is influenced by changes in project assumptions, the feasibility of defined plans, the joint co-ordination of various contractors, and the possibility of unexpected events
- The discount rate, which must include all possible risks within the project that could occur as a result of events that cannot be forecasted accurately
- The selling price of oil, the volatility of which is determined mainly by conditions in the refinery market, that is, supply and demand, as well as by the world economic and political situation
- The capital invested, which is influenced by the choice between different contractors, the specific features of the extraction and production plant, the cost of raw materials, the characteristics of the field and where the project is implemented, the environmental impact, and the authorisations required to carry out the work
- The operational costs of production, which can vary in function of consumption and costs of fuel for vehicles and the plant, personnel costs, and expenditure for control and supervisory activities.
As far as reserves are concerned, the risk analysis method used during the first phase of project development—the Field Potential Analysis—considers all parameters of uncertainty as casual variables whose interactions and interdependences are assessed by means of numerous simulations.
The risk associated to a reserve is then represented as a series of production profiles that reflect the variability of the amount of hydrocarbons in the field (Exhibit 1). The analysis of this risk is an essential input to the first phases in the decision-making process, because definitive results for a concrete assessment plan are still not available, so the aim is to establish whether the reserves discovered are sufficient to justify the further development of the project.
Exhibit 1 – Profiles Representing Cumulative Oil Production Over Time
These profiles are equally probable. However, the different values of cumulative oil production are not evenly distributed. This means that a number of profiles will generate a given total production figure, while some quantities will not be covered by any of the profiles.
In contrast to the Field Potential Analysis, the economic calculations during the development phase are deterministic. On the basis of three scenarios (the “best case,” which takes a production profile able to produce a total quantity of oil equal to P10 of the distribution, the “base case,” which uses a profile corresponding to P50 of the distribution, and a “worst case,” where the profile corresponds to P90 of the distribution), different budgets are calculated, each of which may involve additional activities and costs and involve different resources and deadlines.
The only parameter that is described as a variable is the cost of the investment. Consequently, the uncertainty in the estimate of the final project costs is represented by a range of values that becomes narrower from one phase to the next, as it is based on the more accurate and detailed information available in the phase in which the assessment is undertaken (Eni S.p.A. Exploration & Production Division, 2003).
Generally, the most significant and most common criterion to assess major investments in the oil and gas sector is the net present value (NPV). The value is calculated as the difference between revenues derived from the investment (the sale of oil) and the costs sustained to realise these revenues, that is, both the investment costs to realise the extraction plant (Capex) and the operational costs of extraction and management (Opex).
The project NPV is therefore calculated as:
Revenues, revenue from the sale of oil produced = Amount of oil extracted in month t* Price of oil in month t
Capex: expenditures associated with project realisation
Opex: expenditures associated with oil production
T: number of months in which there are cash outflows derived from the realisation of production plant
t = 0: starting point when the decision to invest is taken
r: discount factor
In the initial phase, the company sustains a series of capital exploration and development costs (investment costs) that generate outflows. When the field comes online, operational costs and revenues are generated resulting in cash outflows and inflows.
Through a combined analysis of these aspects, the model can output a value indicating the project's economic value (Exhibit 2).
Exhibit 2 – The Assessment Model Based on Net Present Value
An application of the model based on a deterministic approach would be much more restrictive, as it reduces the risk assessment to an analysis that only considers a limited number of scenarios or looks at factors one at a time (Ward & Deren, 1991).
The other limitations in the economic assessment of the development process mainly regard the fact that when making the decision, it is not possible to effectively use all the information generated in the Field Potential Analysis. Using a deterministic approach, it is difficult to incorporate all the available probabilistic information and process the data collected. Consequently, the influence of all the identified critical aspects is not interpreted or assessed in depth (Hertz, 1994).
The company in this case therefore felt the need to try different methods to improve the decision-making process and make use of all the information collected in the previous phases. One possible solution to the problem envisages the use of models exploiting a simulation-based probabilistic approach (Badiri, Mortagy, Davis, & Davis, 1997).
The first step in improving the reliability of the risk analysis involves including the uncertainty linked to all the parameters affecting the project result in the economic assessment carried out during the decision-making process.
The risk analysis process thus undergoes a significant modification: from an assessment based on “point estimate” values for the variables, to the consideration of uncertainty through the use of a defined “range” of values for given variables, and finally to a simulation approach based on a “probability distribution” description of the variables, evaluating simultaneously the uncertainty of all project parameters and representing every possible scenario during project development (Kleijnen & Van Groenendaal, 1992).
In this context, the simulation process can be described as a series of what-if analyses, in which each iteration represents a possible combination of the values for the various input variables.
As a first step, the uncertainty for each individual input variable must be identified and quantified, in order to assess the risk involved in making the wrong decision (Milke, 1998).
It is in these cases that the simulation-based approach appears to represent a valid alternative.
Our objective was to develop and test a financial assessment model that takes into account the uncertainties of the project environment by maintaining the variability of the input parameters.
The in-company testing allowed us to address the problem of risk analysis in a practical way, by using a simulation-based probabilistic model with a project developed by Eni S.p.A. Exploration and Production for the realisation and exploitation of an off-shore oil extraction plant located 55 km off the coast of Nigeria at a depth ranging from 502 m to 770 m.
The project, which was realised in 2003 and is currently in production, involves the detailed planning, supply, fabrication, transport, and installation of the plant, the pre-commissioning, the commissioning, and the start-up of services and structures needed for production. It also covers all the activities involved in the drilling management and modelling of the field.
The method chosen to exploit the field is based on a sub-sea production system fixed to floating collection apparatus and flexible pipes to connect and transport the oil produced.
There are three production wells, two water injection and one gas injection.
The underwater structures are connected to the other equipment by vertical collectors and a set of pipes and electrical cables.
By testing the model in the assessment of a real project, we were able to verify its effective utility. The risk analysis had five principal phases:
- Development of the model by defining the problem and the situation to be simulated
- Identification of the parameters that influence project development
- Determination of the uncertainties, defining the variability of the parameters involved in the model
- Analysis of the model and simulation to determine the variability and the probability for each result
- Decision made on the basis of the results of the analysis and expert opinion.
To carry out effective assessments, the distributions associated to the individual parameters need to be defined carefully, seeking to identify those that have the greatest impact on the variability of the output value.
We therefore aimed to reconstruct the trend of each typical parameter, in order to represent as realistically as possible the actual conditions of the project:
- With regard to the production profile of the field, we already had as inputs the data on the 100 alternative solutions representing the variability; the choice of profile to determine the project value was therefore equally probable.
- For the distributions regarding the investment costs, we took the bid values used by different contractors. For each of them we defined a probability distribution considering the proposed value as the minimum value accepted by the contractors to carry out the work requested. It normally happens that after the definition of the facilities specification and depending on market conditions, the defined price can have an increase that can vary on the basis of additional equipments or equipments underestimated during the evaluation phase. We considered each value as a normal distribution, with the bid value as the 5% probability, while the 95% probability considered as the same value with a 20% increase. The average of these distributions describes the uncertainty for each cost item considered in the model.
- For the other parameters, such as the project realisation time and the operational costs, we used an initial estimate determined during the design phase and past experience in similar projects. The uncertainty in the project realisation time, which includes the time required to set up the plant, infrastructure, and equipment needed to start production, was described by identifying the main factors that could influence project duration and could lead to a variation in the time required for completion. On the other hand, the operational costs—the costs sustained for maintenance, production and management after the field goes online—are divided into three principal components: a fixed amount for interventions on the wells, a variable amount defined as a function of the amount of oil produced, and an operating fee referring to the cost of personnel on land and at sea to support the production and transport of the oil.
- The discount factor is defined by company policy.
- However, given the continual instability of the cost of oil, which made it impossible to use past data, we undertook specific analyses for this particular parameter that looked at the current market situation, historical series, future forecasts made by external bodies, and the conservative policy adopted by the company.
The simulation was carried out by considering a different production profile at each iteration. Indeed, each of the 100 profiles can be seen as a deterministic sequence of values representing a possible life-cycle of the field.
Taking the individual profiles and the values for the remaining variables that have an impact on project economics, we obtain the values for the output variable, the NPV.
Results of the Simulation Process
By summing the results of each iteration, we define the distribution probability of the project NPV. The result is not a deterministic value, rather each individual value is associated to a probability of occurrence (Exhibit 3).
Exhibit 3 – Representative Cumulative Probability Curve of Project NPV
From this curve, we can define and analyse the probability of realising a given result or obtaining a value above a given threshold. Inversely, we can obtain the value corresponding to a given probability of success.
The values for the NPV cover a rather large range, from a minimum of -$102,230,280 to a maximum of $1,042,000,000, a spread of $1,143,230,280.
The probability distribution gives a measure of the project risk: the greater the possible offsets from the average value, the more uncertain the NPV.
The expected value given by the distribution is $482,390,135, that is, that value with a 50% probability of realisation. There is only a 10% probability of obtaining a project value greater than $824,983,616. However, the risk of a negative result is also very limited. After having defined the probability distribution linked to the project NPV, a more detailed analysis can be carried out by verifying which particular combinations could lead to failure. The result reveals that there is a 2.06% probability that the project does not produce an economic return. Looking back over the simulation, we can see that, irrespective of the values assumed by the other variables, two of the one hundred profiles result in a negative performance.
The analysis of the oilfields that defined the distribution associated to the production profile highlights a factor that influences the values associated to the two profiles. Both profiles refer to a scenario in which there is a barrier in one lens of the field which obstructs the flow of oil towards the extraction well. It would be useful to verify the economic value of a further analysis to establish more precisely whether this barrier exists, and so reduce the variability associated to the production profile and, above all, provide more accurate data for the investment decision. The analysis of the results should therefore aim to cover the identified risk factors. In function of the results obtained, the most critical parameters can be identified and modified by varying the form of their distribution or reducing the associated uncertainty. In this way we can:
- Reduce the variability of the input parameter, so as to have more reliable final results
- Improve the expected value of the output variable
- Increase the volatility of the output variable in the direction of an improvement in results
- Reduce the volatility of the output variable in the direction of a deterioration in results.
The analysis undertaken also leads to the definition of correlation coefficients between the input parameters and the given output value. The parameter with the greatest impact on the variability of the final result is the production profile. This means that in the specific case in question, an increase in profile variability implies a significant rise in the variability of the project performance. This is also demonstrated by the similarity between the distribution curves for the project NPV and for the total amount of oil extracted from the field (Exhibit 4).
Exhibit 4 – Comparison Between the Probability Distribution Associated to the Project Result and the Distribution for the Total Amount of Oil Extracted
A comparison of the two probability distributions reveals that the bi-modal form of the output variable probability distribution is determined by the variability of the total amount of oil extracted. This peculiarity also results from the hypothesis formulated during the Field Potential Analysis regarding the presence of a barrier within the larger lens of the field (the corresponding probability was set at 60%), which could have a significant impact on the amount of oil extracted.
The comparison between the results obtained with a deterministic method and those from a simulation-based approach reveal a significant improvement in the accuracy of project assessment.
Together with a simple deterministic assessment, a traditional sensitivity analysis is also carried out. The result is compared with the values obtained by varying the investment costs by ±15% or using the P10 and P90 in the production profile distribution (Exhibit 5).
Exhibit 5 – Comparison Between the Distribution Obtained with the Probabilistic Assessment and the Values Derived from the Traditional Analysis
This deterministic approach does not consider the values in the tail of the distribution, which, on the other hand, do emerge in the probabilistic assessment. In particular, it is not possible to obtain through a deterministic approach a negative NPV, while this is possible in the distribution resulting from the simulation.
In conclusion, the simulation technique allows the company to analyse accurately and quickly various scenarios that consider changes in the project's economic and geographic conditions. The model therefore is valuable support for the project planning and assessment process, representing simultaneously the financial, technical, and institutional risks involved.
Badiri, M., Mortagy, A., Davis, D., & Davis, D. R. (1997). Effective analysis and planning R&D stages: A simulation approach. International Journal of Project Management, 15(6), 351–358.
Cox, M., Harris, P., & Siebert, B. R. L. (2003). Evaluation of measurement uncertainty based on the propagation of distributions using Monte Carlo Simulation. Measurement Techniques, 46(9), 824–833.
Eni S.p.A. Exploration & Production Division (2003). OPDS opportunity and project development system handbook.
Fangel, M. (1993). The broadening of project management. International Journal of Project Management, 11(9), 72.
Hertz, D. B. (1994, Sept/Oct). Risk analysis in capital investment. Harvard Business Review.57(5) 169–181
Kleijnen, J. P. C., & Van Groenendaal, W. J. H. (1992). Simulation: A statistical perspective. Chichester, England: John Wiley and Sons.
Mazumdar, M. & Chrzan, L. (1995). Monte Carlo stochastic simulation of electric power generation system production costs under time-dependent constraints. Electric Power System Res., 35(2), 101–108.
Milke, M. W. (1998). Computer simulation to evaluate the economics of landfill gas recovery Water Science and Technology, 38(2), 201–208.
Sivathanu Pillai, A., Joshi, A., & Srinivasa Rao, K. (2002). Performance measurement of R&D projects in a multi-project, concurrent engineering environment. International Journal of Project Management, 20, 165–177.
Ward, W. A., Deren, B. J. (1991). The economics of project analysis: A practitioner's guide. Washington: The Word Bank.
Williams, T. M. (1990). Risk analysis using the embedded CPA package. International Journal of Project Management, 8(2), 84–88.
© 2008, Laura Orlando, Paolo Rossi, Franco Caron
Originally published as a part of 2008 PMI Global Congress Proceedings – Marrakech, Morocco