Project assessment

a guide for implementation

Quaker Oats Company

Darryl J. Ellis

Doric Corporation

Companies have procedures and guidelines for all activities. However, no system of management information can be so all-inclusive that it provides automatic solutions to every business problem. One group of decisions which continually challenges managers is that of alternative capital investments. Investment selection alternatives center around make or buy decisions, replacement or repair of equipment, addition or divestiture of business units (or product lines), and alternative co-mixture of products.

Evaluation of capital investments has as its foundation, the principle that capital productivity can be measured by the return on investment which arise in future periods. As is commonly accepted, a dollar received today is worth more than the same dollar received at a future date. Also, as inflation continues to remain a significant force in the world’s economics, the cost of capital expenditures in the future remains uncertain, except for the likelihood that costs will rise. Thus, the manager is faced with two alternatives: 1) he can spend now and receive something back immediately, or 2) he can spend now but not receive anything until later. Therefore, a problem arises centering around measurement and comparison of costs spread over time. This is complicated by the fact that alternative investments are not identical in size, nature, scope, length of time or benefits.

Too many firms use imprecise methods to estimate if capital expenditures are necessary and if the benefits are realistically attainable. Therefore, the purpose of this paper is three-fold. First, to briefly describe the Delphi technique which attempts to effectively use informed intuitive reasoning in forecasting returns on capital projects; second, to review a capital expenditure simulation model which •uses the Delphi results as inputs; and last, to demonstrate and discuss problems which arise during the implementation of such tools of this nature.


During the decade of the sixties, a new attitude toward forecasting the future became more apparent in public and private planning as well as in the research community. This new pragmatic attitude arose due to the fact that not only was technology changing, but the environment in which technology operated altered the way business had been transacted. In response to the changing environment, a new method was developed. That method — the Delphi technique — can be of assistance when assessing the future. It derives its importance from the realization that projections of the future upon which decisions must rely, are largely based upon individual perception and/or expectation rather than discreet data. In forecasting the future, and in the absence of a proper theoretical foundation, to some extent, one must always rely on intuitive expertise.

It must be remembered that when one seeks data from which expert judgments have to be relied upon, the following primary considerations must be considered:

• The experts must be selected wisely.

• Proper control conditions must be created.

• There must be a high degree of commitment by committee members.

• There must be careful movement towards the same consensus by panel members.

The traditional, and in many ways, the simplest method of achieving a consensus of experts, is to sit them in one room and have a roundtable discussion followed by a position paper. However, this modus operandi does not always present a clear view of all the opinions. For example, the outcome is apt to be a compromise, thereby moderating all sets of opinion. Secondly, one must not forget psychological factors such as gentle persuasion by persons of eminent authority or merely the loudest voice; and finally, the more persuasive pressure of the majority opinion. Addressing itself to these issues, is perhaps the reason why the Delphi technique is the most effective tool in overcoming forecasting difficulties. In its simplest form, it eliminates committee pressure on any expert and replaces it with a carefully designed program of segmental, individual interrogation usually conducted through questionnaires, interspersed with opinion feedback and additional information. We have found the Delphi technique* useful in attempting to determine what return on capital projects one might expect, employing it in order to place typologies on our independent variables, e.g., resource demands, scheduling, variables cost, promotional expenses and costs of new government regulations.

One must not forget that Delphi results alone are not enough when estimating the returns on projects. One must also have some method of simulating this data into meaningful information.

The following case history illustrates the point. The executives of a chemical company faced the decision of entering a new marketplace which necessitated the construction of a new plant. The major factors which management felt were the determining variables were: capital costs, total size of the market, share of market, raw material availability and cost, availability of a technically competent labor force, operating costs, growth of the market, promotional expenses, and costs of potential government regulations. Initially, the project appeared to be a tremendous success. When managers were asked for a most probable estimate (i.e., 70%) for each variable, the calculated return rate for the project was about 25%. However, it is necessary to understand that this 25% DCF is based upon a 70% chance of correctness on each of the nine determining variables happening for each year of the project. Using the laws of probability, the DCF of 25% has only a 4% chance that all nine will be correct for one year. (Arrived at by calculating (.7)9). Thus, the “most probable” return was dependent upon the unlikely incident that all determining variables would perform as expected.

This 1 out of 25 probable occurrence happens on just a simple problem. In reality, it has been our experience that major capital projects are comprised of a multitude of determining variables.

Thus, the expected rate of return represents only one point on a continuous curve of potential outcomes. It is as though one were at a roulette table betting on “5 black”. Predicting a single most probable rate of return in discrete numbers does not illuminate the entire picture, and, in fact, can distort the true most probable occurrence. Hence, the Delphi techniques should be applied to each determining variable in order to provide a typology for the independent variables. After all variables have been analyzed, the following simulation model can be employed.

Simulation Model

A simulation of the way variables may combine in the future is the key to generating accurate forecasts. The model which we employ provides a simple algebraic language and a group of specialized building blocks for normally used probability functions and statistical routines. These routines can be linked in any desired fashion so that all necessary requirements can be simulated. (For example, financial, market, production and research.) Input variables are represented as either fixed quantities or Delphi typologies. Output is available in many forms such as probability density functions describing the risk spread either in terms of R.O.I, or Net Present Value, cumulative probability functions and data tables. In addition, standard statistical computations for means, standard deviations, confidence intervals, sensitivity analysis, chi-squared, correlation co-efficients, etc., are provided.

The model assumes a large number of the simulation chores and technical tasks, thus forcing the risk manager to direct his concern to formulating his problem and interpreting results.

For proper analysis of projects, several steps must be taken. Initially, a range of values for each determining factor must be established. These should be obtainable from the previous Delphi analysis. Within this range, the likelihood of occurrence for each value must also be determined.

Next, through random selection, a value for each particular factor is assigned from the distribution of values. Once these individual values are selected, a DCF is computed for that particular combination. For example, one case may result in the combination of the highest growth range with the lowest price range.

Lastly, it is necessary to repeat the random combination of factors to enable definition and evaluation of the odds of occurrence for each possible rate of return. As there are literally countless potential combinations of values, it is necessary to test the likelihood of occurrence of various specific investment returns. This results in a cataloging of achievable return rates with the estimates that have been made. For each of these rates the probability of occurrence is determined. Then, the average of the values of all outcomes is weighed by their probability of occurrence. This step is important because management prefers lower variability for the same return if all other factors are equal; and the model reveals availability of returns for different contributions and events.

When the expected return and variability of each of a series of investments has been determined, the identical techniques may be used to examine the effectiveness of various combinations in meeting management’s objectives.

Implementation of I. A.M.

As simple as the concept might appear and as easy as it is to utilize, Delphi/Simulation is often difficult to implement. Among the reasons causing this difficulty are:

1) The reluctance of senior management to accept techniques with which they are not familiar. This reluctance stems in part from an information gap resulting from a rapid change in technology which has occurred during the last decade.

2) Communication difficulty between corporate disciplines. The recent move towards specialization of disciplines has created some inter-corporate misinterpretation and rivalry.

3) The lack of a competent technical staff. Most companies have guidelines for investment in hard assets, however, little has been done in the area of people. Not only is it important to have the most technically advanced hardware, more importantly, corporations need the people talent to direct the operation of this equipment. Traditionally, data processing supports the Accounting Department and only later does it encompass other functions.

4) Inadequate systems particularly in the software area. Despite the significant move towards data processing, many companies lack advanced software capability to get the most productivity from their hardware. This has resulted in an underutilization of assets with the necessity of employing outside expertise. Three problems arise from using outside services — confidentiality, rapid turnaround, and management reluctance to continue support of non-affiliated businesses.

As is often useful, an actual example demonstrates more clearly the theoretical principles discussed. The following guide demonstrates that through proper coordination, gentle persuasion and perseverence, a planning function can successfully implement sophisticated new tools such as an Investment Assessment Model.

Introduction & Assessment Phase

Step I — Construct A Plan

Although planners often plan a company’s future endeavors, sometimes they neglect their own areas. Consequently, it is necessary to follow a formal program for completing a task such as the implementation of Delphi/Simulation. This program can take advantage of the ensuing step by step guide.

Step II — Determination of Management’s Receptivity

If management has requested sensitivity analysis, determination of most and least likely cases, and asks for the probability of success; unconsciously, your management is searching for a better method of capital investment assessment. At the same time, if you have performed on a manual basis numerous iterations attempting to answer these questions on specific projects, you, too, are ready for a better approach.

Step III — Technological Familiarization

Prior to any commitment of your resources, take the time to investigate and comprehend the latest analytical techniques. You must become familiar with the algorithms, the inputs, the outputs, and the information developmental process. As you become familiar with the available programs, the realization that modifications must be made to fit your specific needs will become apparent. Rushing in head first will only cause problems. If controversies arise, management is likely to become less receptive to change.

Step IV — Analyze Internal Capability

An important factor in the success of any major new undertaking is the preparedness of the staff. To meet the challenge of using a sophisticated analytical tool, it is necessary to have a staff that can easily be trained in the use and methodology of the procedure. Of course, it will be vital that the staff has access to the latest equipment. A manager, therefore, is faced with a simple choice. If your capability is non-existent, then it has to be developed either in-house or outside. Once this hurdle has been overcome, the staff must become familiar with the new tools so that a concensus of opinion for the use of one tool may be obtained.

It has been our experience that these initial steps which comprise the assessment of tool selection and development (including modifications) takes at least six months, but often longer.

Action Phase

Step V — Selection Of An Appropriate Project

Once the capability has been developed, it is important to be patient. Introduction of the new tools should be carefully planned. Not all projects require the degree of sophisticated analysis which you now hold. It is important to select your initial project judiciously.

We are familiar with this selection process. For example, despite the fact that numerous projects surfaced during our action phase, one stood out from the others due to the vital role it would play in the corporate future. Among the outstanding features were:

1) The project was a joint venture with three participants.

2) The project was international in scope.

3) The project would severely strain the corporation’s capital structure.

4) The project’s life span was over 40 years and would greatly influence the future of the organization.

5) Early recognition of the mammoth analytical task arising from the huge number of independent variables caused serious management concern with the current analytical techniques.

Despite the gravity of the situation, management was uneasy with applying new techniques. However, because of the Planning Department’s perseverence in discussing the serious weaknesses of the current techniques; displaying confidence in the ability to utilize the new tools; and the careful process with which planning familiarized senior management with the benefits of these new tools; management gave its reluctant go-ahead.

Step VI — Appropriate Data Collection

One of the most difficult tasks in collecting information revolves around sources of data. From whom do you collect data? It is not an uncommon practice for a Chief Executive Officer to call in his team and solicit individual opinions. However, it has been our experience that this approach normally results in the generation of biased opinions. For example, if the Chief Executive Officer has a forceful personality, tending towards his being an opinion leader, the results of the team tend towards confirmation of his position. Consequently, in such situations, we feel the Delphi technique to be superior to roundtable discussions.

First, it is important to isolate the independent variables. Next, determine which variables need simulation and which of those can be simulated. For example, cost of capital may be an independent variable, however, its probability distribution is normally tight and straightforward. On the other hand, sales volume is subject to numerous opinions.

After these variables have been identified, those requiring management knowledge and expertise must be investigated. To accomplish this, teams of experts (normally in-house, but if necessary, outside), are formed for the purpose of examining independent variables. To encourage independent thinking, the identity of team members remains concealed except at the final presentation of results. At no time are the individual data questionnaires revealed, thus guaranteeing anonymity to respondents.

Upon return of all questionnaires, the Monte Carlo technique is applied to each year in order to derive probability distributions. This creates an easily viewed probability for individual variables by year. The results are then reviewed by all questionnaire participants and senior management to arrive at agreement on the basic inputs.

Questionnaires should be drawn for all independent variables covering the entire life of the project.

Step VII — Simulate The Project

Up to now, work on the project represents a collection of disjointed data. What remains is to link all of the data into a representative, functioning model. Using a simulation model similar to that discussed, this linkage occurs. Output should take many forms. We suggest that a concentration should be placed upon the following statistical tests for validity of input and output variables.

Among the statistical tests that we found to be most useful in verifying our project assumptions were: standard computations for means, standard deviations and confidence intervals; chi-square tests; correlation coefficients; F-tests, regression analysis; min and max points, and the coefficient of Kurtosis (which indicates degree of skewedness). It is important to make certain that both the inputs and outputs accurately describe the project.

At this point, the benefits of having developed a strong analytical team are reaped. If the team is ill-prepared, confirmation of results will be lacking and open to senior management skepticism.

If everything has been done correctly, there probably will be 500-1,000 simulated project runs. From this an expected rate of return will be generated along with the maximum rate of return as well as the minimum rate of return. Additionally, sensitivity analysis will have been performed on each independent variable which identifies those variables that are the driving forces in the project. These are the ones upon which concentration must be placed.

You are now about ready to meet with management.

Step VIII — Simplify The Presentation

Because of the process just completed, a multitude of data has been compiled, simulated and analyzed. The temptation is to impress senior management with your efforts by presenting an in-depth review of the output. It is advisable to resist this temptation for two reasons. The first being that management will never reach your point on the learning curve in one afternoon. The second reason is that within a short period, management will recognize the extent of your accomplishments.

All presentations must remain simple. For example, use bar graphs which might not depict the distributions perfectly, but with which management is familiar. As more projects are presented, you can slowly familiarize management with actual output.

Also, do not speak in statistical terminology. It has probably been many years since management has been exposed to these techniques, particularly at this level of sophistication. For example, the following statement is easily understood. “Gentlemen, there is a 50% probability that the rate of return on our investment will be between 18% and 22%. Analysis reveals the maximum return on this project is about 26% and the minimum return is about 5%. However, if sales volume increases by 10% in each year, we can expect our rates of return to approach 26% to 30%.”

At this stage, management might request additional analysis based upon revised project estimates and it is likely that you will be involved in solving future problems.


The questions facing management in determining which capital project should go first are: What precise results can be expected? What information must be estimated to obtain results? Is there a way to get basic agreement on key factors such as — demand, prices, costs, insurance coverage and so on? And how is return on invested capital measured?

Current conventional methods are one dimensional. The reason is that estimates made to depict future occurrences are just that, estimates. Because uncertainty encompasses these estimates, all calculations prove to be self-defeating. Even estimates derived independently from individual specialists in the corporate structure are subject to question. Information gathered from numerous sources is meaningless if not logically descriptive of future results. For these reasons, the described Delphi simulation approach has the inherent advantage of simplicity in depicting reality. However, it requires management support in wanting a portrait of the risks and rewards; as well as expert follow-through on the part of the planners. The technology to simulate has already been developed and is easy to use; all that is necessary is management’s need and planning’s ability to analyze uncertainty

Before closing, we would like to comment on our experience in using this approach to simulation and modeling. Some of our capital projects have contained over 100 variables. Needless to say for such an undertaking, good cooperation must exist between all the major disciplines (Planning, Finance, Marketing, Production and R & D) as well as management’s support. Only with such cooperation does an accurate picture of the risks emerge. We have found that the process is not easy. Much preparation on our part which is not readily visible is necessary to continually review the system with management. However, the rewards make the behind-the-scenes preparation worthwhile.

* For a more indepth discussion of Delphi, see Technological Forecasting for Industry and Government. James P. Bright — Prentice-Hall.



Related Content

  • Project Management Journal

    Narratives of Project Risk Management member content locked

    By Green, Stuart D. | Dikmen, Irem The dominant narrative of project risk management pays homage to scientific rationality while conceptualizing risk as objective fact.

  • Project Management Journal

    Identifying Subjective Perspectives on Managing Underground Risks at Schiphol Airport member content locked

    By Biersteker, Erwin | van Marrewijk, Alfons | Koppenjan, Joop Drawing on Renn’s model and following a Q methodology, we identify four risk management approaches among asset managers and project managers working at the Dutch Schiphol Airport.

  • PMI Sponsored Research

    Digitalization as a Game Changer in Project Stakeholder Management member content open

    By Kier, Christof | Huemann, Martina This research work aims to discuss contemporary project stakeholder engagement and examines how digitalization shapes and affects the field.

  • Project Management Journal

    Top Ten Behavioral Biases in Project Management member content locked

    By Flyvbjerg, Bent This article identifies the 10 most important behavioral biases for project management.

  • Project Management Journal

    Collective Mindfulness member content locked

    By Wang, Linzhuo | Müller, Ralf | Zhu, Fangwei | Yang, Xiaotian We investigated the mechanisms of collective mindfulness for megaproject organizational resilience prior to, during, and after recovery from crises.