Abstract
Because projects are often perceived as processes aimed at generating a unique output, they frequently end before organizational benefits have been secured. This paper develops a model that expands the traditional approach by introducing a new framework of accountabilities for key players and by offering some novel tools in order to secure a future flow of outcomes from a project. The model was validated using a triangular research design research. First, a quantitative study involving 102 managers showed that a funder's focus on achieving target outcomes contributes to project success. Outcome and output definition and project plan development and control are found to mediate this relationship. Second, a case study demonstrated the benefits to the organization from using this approach. Third, a literature analysis indicated that the proposed approach is in alignment with recent trends in research. Finally, we discuss five practical implications of the proposed model that suggest significant changes to the traditional project management body of knowledge and practice.
Keywords: Benefit realization, benefit generation, project, success measurement
Introduction
The importance of projects for the achievement of desired outcomes for organizations continues to gain wide acknowledgement (Lewis, Welsh, Dehler, Green, 2002). However, despite their importance, most projects fail to achieve their stated goals (Shenhar & Dvir, 2007; Love, Edwards, & Irani, 2008; Zwikael & Globerson, 2004). In response, scholars have explored ways to increase project success rates by analyzing the causes of failure (Shenhar, 2001), identifying critical success factors (Pinto & Slevin, 1987; Cooke-Davies, 2002), establishing project success criteria (Muller & Turner, 2007), repositioning the role of project managers (Webber & Torti, 2004), and developing project management methodologies and frameworks (Kerzner, 2009; Project Management Institute [PMI], 2008; Office of Government Commerce [OGC], 2007; International Project Management Association [IPMA], 2006). In general, these endeavors have had little effect on project success rates (Lewis et al., 2002; Zwikael & Sadeh, 2007; Scott-Young & Samson, 2008). Project failures continue to impose a heavy financial burden on organizations and economies.
One explanation for low success rates may lie with the observation that common definitions of projects (e.g., PMI, 2004; IPMA, 2006) and supporting practices focus on output delivery. This view has two consequences: most projects end when the primary output has been delivered, and accordingly, success measurement is limited to time, cost, and output quality. For some time, concerns have been expressed that this view might be flawed (Shenhar & Dvir, 1996; Meredith, 2002; Packendorff, 1995). Turman (1986), for example, predicted, “the days when we could define success in terms of cost, schedule and technical objectives are gone.” Similarly, Turner and Müller (2003) criticized that “classical definitions of projects are not wrong, just incomplete.” Others (e.g., Shenhar & Dvir, 1996; Turner, 2006) claimed that most research literature on the management of projects suffers from a scanty theoretical basis and a lack of concepts.
As a result, the project management discipline has become preoccupied with deliverables, while neglecting benefits (Carden & Egan, 2008; Remenyi, White, & Sherwood-Smith, 1997; Ashurst, Doherty, & Peppard, 2008). Because the only purpose of a project is to generate outcomes that are sought by the funding organization (Turner, 2006; Shenhar & Dvir, 2007; OGC, 2007; Saarinen, 1990; Shenhar & Dvir, 2007; Turner & Muller, 2007), we offer a different view. In contrast to accepted output-based practice, this paper describes an alternative outcome focused approach to the way organizations perceive and manage projects. It then goes on to validate this approach using a triangulated research design based on a quantitative field study, a qualitative case study, and a comparison with the recent management literature.
The Purpose of Projects
Projects are a core area of work for most organizations and of intense interest to many internal and external stakeholders. This is because projects are central to the generation of organizational benefits such as reduced operating costs, increased revenue, increased market share, and enhanced product/service quality. A key feature of a project is that it has specific target benefits to achieve (Dvir & Lechler, 2004; Kerzner, 2009). A project can also be viewed as a form of investment in which outlays are made today with the intention of realizing a flow of benefits over some future timeframe.
Common Perceptions of Projects
In practice, instead of being benefit-focused, project management is often perceived as an outputdelivery exercise that ends with the implementation of a “unique product or service” (PMI, 2004). A new bridge, an upgraded software system, or a suggested new procurement process are all examples of outputs. Table 1 summarizes some traditional definitions of “project” in the recent literature.
Table 1. Project Definitions Focused on Output Delivery
| Source | Project definition | Output-Oriented Terms Included in the Definition |
| Project Management Institute, 2004 | Temporary endeavor undertaken to create a unique product or service | Product or service |
| Meredith and Mantel, 2006 | A specific, finite task to be accomplished | Task |
| Lewis, 2000 | A project is a one-time, multitask job that has clearly defined starting and ending dates, a specific scope of work to be performed, a budget, and a specific level of performance to be achieved | A job with scope of work to be performed |
| International Project Management Association's Competence Baseline, 2006 | Time and cost constrained operation to realize a set of defined deliverables up to quality standards and requirements | Deliverables |
Most of these definitions accept output delivery as the main purpose for project execution—reflecting operational view—whereby the work of a project consumes resources, (generically identified as inputs) to execute processes resulting in an output. When linked in a diagram these same three elements provide us with a conceptual view of a project, as is shown in Figure 1 (Smyrk, 1995). This representation is known in the operations management discipline as the input-process-output (IPO) model (Chase, Jacobs, & Aquilano, 2006; Krajewski & Ritzman, 2005; Russell & Taylor, 2005). The link between outputs and the process is an example of strong causality (Hicks, 1979). This means that the outputs will exist if, and only if, the process is executed. In business, we treat processes as controllable and so are willing to guarantee outputs. The IPO model is also often applied to projects, as being a special type of process (e.g., PMI, 2008). Such a view is consistent with the proposition that operations management is an allied discipline to project management (Kwak & Anbari, 2008).
Figure 1. The Input-Process-Output (IPO) Model of a Project
The focus on outputs is consistent with Turner's claim that “scope management is the raison d'être of Project Management” (Turner, 1993). As a result, most project management methodologies focus on the delivery of project outputs within constraints of quality, time, and budget (Scott-Young & Samson, 2008; PMI, 2008; Dvir & Lechler, 2004).
Weaknesses in Current Perceptions of Projects
Many scholars criticize current project management methodologies and the lack of a robust theory. For example, Shenhar and Dvir (1996) argued, “most research on the management of projects is relatively young and still suffers from a scanty theoretical basis and a lack of concepts.” Meredith (2002) claimed that the project management literature is often characterized by frameworks unrelated to previous literature and non-rigorous research methods. Reasons for the relative immaturity of much of the material appearing in the project management literature include claims that it is practitioner-driven (Jugdev, 2004) and reliant on “war stories” (Meredith, 2002). Other criticisms relate to the extensive use of normative (rather than positive) approaches (Lipsey & Lancaster, 1956; Budzinski, 2008) and appeals to lists of factors derived from surveys of project practitioner opinions rather than empirical research grounded in theory (Packendorff, 1995). Bygstad and Lanestedt (2009) support this view by showing that project management methodologies are rarely used “as is” in Germany and Switzerland, but usually modified or adapted before application. Increasingly, there have been calls for improved theory generation through the adoption of research designs that build on existing literature to develop models for rigorous, evidencebased testing in industry (Meredith, 2002; Winter, Smith, Morris, & Cicmil, 2006).
More specifically, because most current project management approaches are preoccupied with output delivery, some researchers acknowledge a major fault—that they ignore the generation of benefits (Carden & Egan, 2008; Remenyi et al., 1997; Ashurst et al., 2008). Moreover, because there is widespread confusion between outcomes and outputs in the literature (Nogeste & Walker, 2005), a number of approaches that claim to be outcomes-oriented may simply be variants of an outputs based approach. While the PRINCE2 methodology (OGC, 2007) leans towards a benefit-based view of project management, it still lacks practical tools to manage project outcomes.
Support for the importance of outcome generation is also found in the organizational modeling literature. For example, the Logic Model (Savaya & Waysman, 2005), which integrates planning, evaluation, and action in organizations, suggests a sequence of steps to achieve a desirable impact. This sequence recognizes inputs, activities, outputs, and outcomes, leading to some form of beneficial effect. The logic model associates outcomes with specific changes in areas such as program participants’ behavior, knowledge, skills, status, and level of functioning. However, it does not explain the mechanism by which outputs give rise to outcomes.
A consequence of these weaknesses is that there is little encouragement and support in the literature for organizations to seek, measure, and manage specific benefits from projects. Instead, accepted wisdom appears to promote the view that senior managers should be satisfied with outputs delivered according to plan. Correspondingly, there tends to be little by way of a complete project benefit theory in the literature.
An Alternative Project Approach
The drawbacks identified in the literature suggest that an alternative approach is required: one that focuses on outcome realization, rather than output delivery. The need for a benefits-based framework is also recognized by leading scholars in recent studies (e.g., Turner, 2006; Malach-Pines, Dvir, & Sadeh, 2009).. This view accepts that generating outcomes is the real reason for every project.
Project Outcomes
If benefits are to be realized from projects, one has to accept that target outcomes provide the rationale for producing a project's outputs. Because outputs take the form of artifacts (“things”), they are properly described as tangible. Outcomes, on the other hand, take the form of measurable effects and so are classified as intangible. Outcomes in general, and target outcomes in particular, can always be expressed as a change in the value of a variable associated with an end-effect. Because an outcome represents a change for the organization, its labeling should recognize this expected change. As a result, the wording used in the titles of outcomes will frequently begin with a participial adjective (loosely an “–ed” word such as “increased” or “decreased”), for example: “reduced waiting times,” “increased market share,” and “compliance with legislation.” Table 2 summarizes the major differences between outputs and target outcomes.
Table 2. Outputs Versus Outcomes
| Characteristic | Output | Outcome |
| Intention | What is to be delivered? | What is the objective? |
| Form | Artifact | Measurable end effect |
| Specification | Set values for all critical fitness-for-purpose features | Set seven attributes (characteristics) |
| Appearance | Impossible without execution of process | In certain cases possible - even if process is not executed |
| Lead time | Available immediately after process is executed | Delayed until after execution of the process |
By way of illustration, consider a project that is being proposed to reduce traffic congestion in a city's central business district. Outputs from the exercise include a cross-city tunnel, changes in the configuration of existing city streets, a tolling system, a suite of management/maintenance processes, and a new business unit to operate the facility. Candidate target outcomes could include “reduced accident rates„ and “reduced travel times.„
The Definition of a Project
Support for an outcome-based project approach can also be found in the increasing prominence of benefits and outcomes in new definitions of “project„ in the recent literature, as is presented in Table 3. While traditional definitions focused on output delivery (Table 1), an analysis of these newer definitions show that, in general, they accept that projects are a means to creating change and achieving agreed goals. Interestingly, an outcome-based definition suggested as far as half a century ago, has, unfortunately, been ignored until recently: “A project is an organization unit dedicated to the attainment of a goal„ (Gaddis, 1959).
Table 3. Recent Project Definitions Focused on Outcome Realization
| Source | Project Definition | Outcome-Oriented Terms Included in the Definition |
| Turner, 2006 | Temporary organization to which resources are assigned to do work to bring about beneficial change | Change |
| Gray and Larson, 2006 | A complex, non-routine, one-time effort limited by time, budget, resources, and performance specifications designed to meet customer needs | Needs |
| Shenhar and Dvir, 2007 | A temporary organization and process set up to achieve a specified goal under the constraints of time, budget, and other resources | Goal |
| Pinto, 2007 | A unique venture with a beginning and end, conducted by people to meet established goals within parameters of cost, schedule and quality | Goal |
| Kerzner, 2009 | Any series of activities and tasks that have a specific objective to be completed within certain specifications | Objectives |
| PMI, 2008 | Temporary endeavor undertaken to create a unique product, service or a result | Result |
Although the acknowledgement of the importance of outcomes in the recent literature (as is suggested in Table 3), no models appear to have been developed to explain the causal effect between outputs and outcomes.
The Project Model
This section supports an outcome-oriented project approach with a new definition of a project and an accompanying conceptual model. Following the significant shift in emphasis from outputs to outcomes, the proposed definition for a project in this paper is: “A unique process intended to achieve target outcomes.” This definition suggests that the IPO model presented in Figure 1 be modified by expanding the three components that are already there (inputs, process, and outputs). The new structure represents the ITO model of a process, as shown in Figure 2. The ITO model is so named because it seeks to explain how inputs on the left are transformed into outcomes on the right as in the following diagram (Smyrk, 1995).
Figure 2. The ITO (Inputs Transformed into Outcomes) Model of a Project
The left hand half of the ITO model is simply the IPO model—to which has been appended a utilization mechanism and a flow of target outcomes. The “left-to-right” chronology implied by the IPO model in Figure 1 can now be extended in the following way: the project's outputs are eventually delivered to someone who then utilizes them in a way that subsequently generates target outcomes. In the ITO model we propose a mechanism by which outputs give rise to outcomes—namely utilization. The entities who utilize a project's outputs in such a way as to generate target outcomes are called the project's customers. All projects should have target outcomes (explicit or implicit), because they are funded to effect some intended change.
The chronology that underpins the ITO model can be made a little more obvious by showing a horizontal timeline. Under this view, the “work” part of the model (represented by the process ellipse) would have defined start and finish dates—obtained notionally by dropping perpendiculars from the left and right hand extremities of the ellipse onto the X-axis. The process is the duration of the work required to produce the project outputs. While this is the traditional view of a project's duration, the ITO model suggests that duration is, in fact, somewhat longer, because a project can only be regarded as finished when a flow of target outcomes has been achieved, or at least secured. Such an event occurs only after utilization has begun (following delivery of outputs).
An Example of the New Project Model
The following example demonstrates the use of the ITO model. A project is executed by a state health department to improve the overall performance of its hospitals. This has been triggered by a combination of (conflicting) budgetary and community pressures. As illustrated in Figure 3, the five ITO components for this project would typically be:
- Inputs: Funds measured in dollars (arising from purchased-in products and services) and labor (representing the involvement of departmental staff assigned to the exercise), measured in personhours.
- Process: Reengineer critical hospital-related procedures (such as admit patients, perform surgery and discharge patients).
- Outputs: A typical initiative of this kind would involve deliverables such as: new business and medical processes, new enabling technologies and new organizational arrangements.
- Utilization: To achieve the objectives set for the exercise, all the outputs must be utilized by a number of stakeholders (known as the project's customers)—the most prominent of which would be administrative and medical staff. Patients would also be involved in the utilization of certain outputs as well). Clearly, if the project's customers do not utilize the outputs to a desired level, then the target outcomes (described next) cannot be generated.
- Target outcomes: In this case, examples of appropriate target outcomes would include: reduced waiting time for elective surgery (measured for certain benchmark procedures), increased efficiency of hospital administration, reduced recovery times for patients and reduced rates of readmission. (It should be noted that if all these were adopted, some trade-offs may be necessary when setting targets for each).
Figure 3. The ITO Model of an Illustrative Hospital Project
Two of the ITO components (outputs and outcomes) when augmented with a succinct objective, make up the project's statement of scope. In this illustration, an objective might appear as “This project seeks to enhance the performance of the state health system.”
Model Validation
In this section, we employ a concurrent mixed method design (Jick, 1979; Sackett & Larson, 1990; Scandura & Williams, 2000; Yin, 1984), which uses a triangular approach to validate various aspects of the project model: an empirical study, qualitative case study, and literature analysis.
Model Validation—a Quantitative Pilot Field Study
The objective of the empirical study was to validate the importance and core role of outcomes in the project process. This section raises research hypotheses, followed by data collection and statistical analyzes.
Research Hypotheses
The ITO model and the supporting literature trigger our list of research hypotheses described in this section. The first hypothesis states that outcomes drive projects (Carden & Egan, 2008; Remenyi et al., 1997) and hence their achievement is critical to a project funder—thus representing the trigger for the whole project process.
H1: A funder's objective is to realize target outcomes from a project.
Based on the previous hypotheses, the ITO model (presented earlier) and the conventional project management methodology, a research model has been developed. As can be seen in Figure 4, the model assumes that the project process is triggered by a desire to realize certain outcomes and is then driven by their importance. This, in turn leads through outcome and output definition, project planning, and control to project success.
Figure 4. The Research Model
The research model assumes causal effects that start with the desire for target outcomes, and their definition. Then, the phases described in project management methodologies follow: scoping the project by defining its outputs, planning the project, executing and controlling it, and delivering outputs. According to the ITO model, these outputs support the achievement of target outcomes, and as a result, contribute to project success. The following research hypotheses state the relationship among these variables as a series of mediating effects.
The first two of these processes require that the project's outcomes and outputs be clearly defined— drawing directly from the ITO model and the statement of scope. Hence, we propose the following hypotheses:
H2a: The relationship between the desire for target outcomes and output definition is mediated by outcome definition.
H2b: The relationship between outcome definition and project planning is mediated by output definition.
The management literature suggests strongly that, work in organizations has to first be planned, then controlled (Chase et al., 2006). The project management literature views planning, execution and control as core processes required towards the achievement of project outputs (Aladwani, 2002; Kerzner, 2009; PMI, 2008). In addition, project management studies confirm that high quality planning, accompanied by appropriate control of project execution contributes to successful delivery of project outputs (e.g. Pinto & Slevin, 1988; Zwikael & Globerson, 2004; Dvir & Lechler, 2004). Hence, we assume that planning and later, control of execution, are essential for successful output execution.
H2c: The relationship between output definition and project control is mediated by project planning.
H2d: The relationship between project planning and output development is mediated by project control.
H2e: The relationship between project control and achieving target outcomes is mediated by output development.
Finally, the ITO model suggests outcomes generation is required for project success, hence:
H2f: The relationship between output development and project success is mediated by the achievement of target outcomes.
Research Methodology
Based on the hypotheses, a questionnaire was developed and distributed to 102 project managers, program managers, project officers, and project funders in Asia Pacific countries. All research participants were members of a Project Management Institute (PMI) chapter in the Asia Pacific region. Twenty-nine responses came from India, 16 from New Zealand, 15 from Australia, and the rest from other Asia Pacific countries. There were 20.2% responses from software organizations, 19.0% from services, 11.9% from engineering, 13.1% from government, and 8.3% from production organizations. Project duration ranged between 2 and 60 months with an average of 15.6 months. Of these projects, 55.4% were executed for an internal funder within the same organization, while 44.6% of the projects were undertaken for an external organization. Eighty-nine percent of responses were male and 11% female.
Data collected from these questionnaires included information to address all research hypotheses: (1) the relative importance of various project management factors (described in the research model in Figure 4) to the project funder using a five-point Likert scale (see the questionnaire structure in Table 4); (2) level of effort invested in the same factors using a similar five point Likert scale; (3) organizational and project characteristics; (4) project success, measured as the level of funder satisfaction (Dvir & Lechler, 2004; Hackman, 1987; Baker, 1988; Scott-Young & Samson, 2008; Zwikael & Sadeh, 2007) with the project on a scale of 1 to 5 and; (5) the person accountable for outcome realization in the last completed project. The unit of measurement was a project.
Table 4. Example of the Project Questionnaire: Importance of Project Management to the Project's Funder
| Project Management Factor | Not important | Extremely important | |||
| 1. Developing a list of agreed outputs (deliverables) | 1 | 2 | 3 | 4 | 5 |
| 2. Developing a list of agreed target outcomes (benefits) | 1 | 2 | 3 | 4 | 5 |
Results—The Importance of Target Outcomes in a Project Context
The first research hypothesis argues that a funder's objective is to realize target outcomes and hence outcome realization is of greatest importance to them. To test this, we analyzed the relative importance of various factors to project funders. Table 5 ranks the factors, using their means—on a scale of 1 (low) to 5 (high).
Table 5. The Ranking of Project Management Factors by Importance to Funders
| Project Management Factors - A Ranked List | Importance to Funder (1-5 scale) |
| 1. Achieving target outcomes (benefits) | 3.98 |
| 2. Approving a business case | 3.92 |
| 3. Developing a business case | 3.91 |
| 4. Developing a list of agreed outputs (deliverables) | 3.91 |
| 5. Producing outputs (deliverables) | 3.91 |
| 6. Developing a list of agreed target outcomes (benefits) | 3.84 |
| 7. Effective communications with stakeholders | 3.70 |
| 8. Controlling the project | 3.68 |
| 9. Developing a project plan | 3.61 |
| 10. Managing project risks | 3.53 |
| 11. Assigning a person accountable for target outcomes (benefits) achievement | 3.46 |
| 12. Support provided by senior managers | 3.40 |
| 13. Assembling a suitable project team | 3.19 |
| 14. Updating the project plan | 3.17 |
| 15. Managing the project team | 3.09 |
| 16. Developing the project team | 2.70 |
This analysis shows that achieving target outcomes is the single most important factor for project funders with a mean of 3.98. An additional statistical analysis shows that the 95% confidence interval is (3.78, 4.18), which means that the importance of achieving target outcomes to a funder is significantly higher than most other project management factors. While some other factors also have very high scores, they too relate (albeit indirectly) to outcomes—in particular, development/approval of the business case (where outcomes are defined) and development/production of outputs (that directly support outcome generation). The importance of outcomes and their related processes to the projects’ funders allows us to accept the first research hypothesis, and claim that the funder's objective is to realize target outcomes from a project.
Results: Validating the Research Model
A second set of research hypotheses, which includes the factors linking the desire for target outcomes to project success, has been used to validate the research model. In order to test these hypotheses, we first analyzed the correlations among all project management factors included in the research model. For each pair of factors, Table 6 presents the Pearson correlation and significance level (in brackets). A one-tailed test was used, as the hypotheses assume positive correlations among all variables.
Table 6. Correlations Among Research Variables
| F4-Importance of developing target outcomes | P4-Developing a list of agreed target outcomes | P3-Developing a list of agreed outputs | P5-Developing a project plan | P6-Controlling a project | P14-Producing outputs | P15-achieving target outcomes | Project success | |
| F4-Importance of developing target outcomes | 1.000 | 0.414** (0.000) | 0.392** (0.000) | 0.125 (0.119) | 0.137 (0.198) | 0.242* (0.011) | 0.443** (0.000) | 0.234* (0.016) |
| P4-Developing a list of agreed target outcomes | 1.000 | 0.609** (0.000) | 0.294** (0.002) | 0.399** (0.000) | 0.183* (0.042) | 0.533** (0.000) | -0.083 (0.228) | |
| P3-Developing a list of agreed outputs | 1.000 | 0.490** (0.000) | 0.600** (0.000) | 0.519** (0.000) | 0.443** (0.000) | 0.269** (0.007) | ||
| P5-Developing a project plan | 1.000 | 0.674** (0.000) | 0.387** (0.000) | 0.227* (0.016) | 0.225* (0.020) | |||
| P6-Controlling a project | 1.000 | 0.534** (0.000) | 0.396** (0.000) | 0.110 (0.160) | ||||
| P14-Producing outputs | 1.000 | 0.576** (0.000) | 0.400** (0.000) | |||||
| P15-achieving target outcomes | 1.000 | 0.317* (0.002) | ||||||
| Project success | 1.000 |
* p≤0.05; **p≤0.01
These results show positive and significant correlations among all seven pairs of factors included in the hypotheses (bolded values in Table 6). However, as the research model and hypotheses include several mediating effects, we also conducted a test for mediation (Baron & Kenny, 1986). Six of the eight variables in our model are mediators, and so a series of three tests were conducted to validate each mediating effect, as summarized in the six rows in Table 7. The first tests the significance of the correlation between the independent and the mediating variables—for which the results appear in the column entitled “X1→X2.” The second tests the significance of the correlation between the independent and the dependent variables—for which the results appear in the column entitled “X1?Y”). The third involves a regression including the independent (X1), mediating (X2), and dependent (Y) variables. Here a mediating effect is confirmed if it is found that the dependent variable (Y) is unrelated to the independent variable (X1), but significantly related to the mediating variable (X2). For each set of tests, results include the identification of the independent, mediating and dependent variables and the significance level of the slopes for each of the three tests.
Table 7. Tests for Mediator Effects in the Model
| Independent variable (X1) | Mediating variable (X2) | Dependent variable (Y) | Significance level X1→X2 | Significance level X1→Y | Significance level (X1, X2)→Y | Mediating effect confirmed? |
| F4 | P4 | P3 | 0.000** | 0.000** | F4: 0.069 ns P4: 0.000** | Yes |
| P4 | P3 | P5 | 0.000** | 0.005** | P4: 0.956 ns P3: 0.000** | Yes |
| P3 | P5 | P6 | 0.000** | 0.000** | P3: 0.000** P5: 0.000** | Partially |
| P5 | P6 | P14 | 0.000** | 0.000** | P5: 0.696 ns P6: 0.000** | Yes |
| P6 | P14 | P15 | 0.000** | 0.000** | P6: 0.211 ns P14: 0.000** | Yes |
| P14 | P15 | Project success | 0.000** | 0.000** | P14: 0.006 P15: 0.152 ns | No |
* p≤0.05; **p≤0.01; ns: not significant
Table 7 allows us to fully accept four of the mediating effects, partially accept another, and reject one. These results appear generally consistent with those predicted by the ITO model. Before claiming support from this analysis for the research model, it is necessary to examine another explanation for the results— whereby significant linkages exist among isolated subsets of moderating variables, but not among all the variables that collectively make up the complete model. The strategy to test for such an effect involves analysis of the impact that independent and control variables have on each variable added to our research model. Thus, we move through the model, one variable at a time to ensure that each additional process makes a significant contribution to the model. The dependent variable from one iteration becomes the independent variable in the next, with the others becoming controlled variables, as summarized in Table 8.
Table 8. Tests for Controlling Effects in the Model
| Independent Variable | Controlled Variable | Dependent Variable | R-squared Value | Significance Value |
| F4 | - | P4 | 0.414 | F4: 0.000** |
| P4 | F4 | P3 | 0.394 | F4: 0.069 P4: 0.000** |
| P3 | F4, P4 | P5 | 0.245 | F4: 0.437** P4: 0.893 P3: 0.000** |
| P5 | F4, P4, P3 | P6 | 0.559 | F4: 0.248 P4: 0.374 P3: 0.001** P5: 0.000** |
| P6 | F4, P4, P3, P5 | P14 | 0.398 | F4: 0.128 P4: 0.015* P3: 0.002** P5: 0.927 P6: 0.003** |
| P14 | F4, P4, P3, P5, P6 | P15 | 0.561 | F4: 0.020* P4: 0.000** P3: 0.093 P5: 0.307 P6: 0.459 P14: 0.000** |
| P15 | F4, P4, P3, P5, P6, P14 | Project success | 0.351 | F4: 0.159 P4: 0.002** P3: 0.028* P5: 0.024* P6: 0.0852 P14: 0.242 P15: 0.012* |
* p≤0.05; **p≤0.01; ns: not significant
In each regression run, the last added independent variable has a significant effect on the dependent variable, given the controlled variables. These results support the suggested model and (with the additional evidence from Tables 6 and 7) allow us to accept all H2 sub-hypotheses.
Model Validation—A Qualitative Case Study
The purpose of the qualitative case study is to illustrate the logic of both the ITO and accompanying research model.
PSMA is an Australian company that supplies value-added spatial information generated from raw data obtained from various agencies across the country. This information is packaged as datasets such as transport and topography (which is concerned with road, rail, air infrastructure, parks, and water bodies). PSMA's clients are firms who develop products and services that present the data in meaningful and useful ways (such as map displays for in-car navigation systems). In 2003, the company saw a gap in the market for an authoritative geocoded address index covering the whole country. A project entitled G-NAF was commissioned to fill this gap.
In terms of the variables included in the research model, G-NAF can be characterized in the following way:
Project objective: PSMA saw that an increase in the levels of commitment amongst its customers to the company's product sets was critically important to achievement of its strategic business goals. Thus, an objective was established “To enhance PSMA's position as a leader in the supply of spatial information.
Outcomes definition: PSMA saw that an increase in the levels of commitment among its customers to the company's product sets was critically important to achievement of its strategic business goals. This objective was then expressed as two defined outcomes—increased numbers of organizations committed to G-NAF as their source of national address data and increased number of projects undertaken by organizations to enhance their products/services/processes by exploiting G-NAF.
Outputs definition: The project was scoped in terms of five major outputs: a geocoded national address file, a set of agreed custodial licensing arrangements (involving data-suppliers), a distribution strategy, a new business unit to manage ongoing G-NAF-related operations, and a product maintenance strategy.
Project planning: After the business case for G-NAF was accepted, a detailed project plan was assembled including a work breakdown structure, schedules of milestones, resource plans, etc.
Project control: A strict regime of regular reporting on the project was instituted at the outset and adhered to throughout the entire exercise. Monthly reports covered: progress against time, budget status, a risk/issues report, and confirmation of recent deliveries.
Output development: The project was executed in accordance with the project plan. PSMA's chief operating officer was appointed as project manager and the CEO as project owner.
Achieving target outcomes: Some time after all outputs were delivered, a study was undertaken to assess the extent to which target outcomes had been achieved. The conclusion drawn from this was that both had been realized.
Assessing project success: When all outputs from the project had been delivered and implemented, a formal closeout workshop was conducted to assess the success of the project under eight performance criteria (such as adherence to budget). The results of that exercise, together with later analysis of client records, indicated that G-NAF had been successful.
Dan Paull, the CEO of PSMA Australia Limited has concluded this project by saying: “there is no doubt that the use of the ITO model played a significant role in success of the project to develop G-NAF. With this framework in place, it [became] easier to monitor the progress of the project, manage contingencies, and respond to opportunities in a very dynamic environment.”
Model Validation—Project Success Measures
As part of the validation exercise, this section analyzes the alignment of project success dimensions appearing in the recent project management literature with the suggested outcome focused approach.
Unlike traditional project management theories where success is related to the delivery of outputs, our model suggests that achievement of outcomes must be taken into account when judging project success. Many of the success dimensions that are now appearing in the literature can be viewed as specific instances of target outcomes. For example, Dvir, Lipovetsky, Shenhar, and Tishler (2003) provided an illustration of a project success criterion in the form of “increased business opportunities for the future”.
Our model assumes that achieving target outcomes will meet a funder's objective for the project. Consistent with this view, Kerzner (2009) required that its results be acceptable to the funder. Dvir and Lechler (2004) sought confirmation that the funder is satisfied and Dvir et al. (2003) want funder benefits. (On a point of terminology, the word “customer,” as used in the original papers referenced here, identifies the stakeholder that we call the “funder.”)
Implications of the ITO Model
As there is little point to developing project outputs that do not eventually generate desirable outcomes, senior managers should reconsider the way they perceive, sponsor, and manage projects. The following paragraphs describe some immediate implications of this view on projects.
The Project Life—A New Project Phase
Conventionally, projects are regarded as complete when outputs have been delivered (PMI, 2008; Kerzner, 2009). Projects are commonly broken into three phases: initiation (when an idea is approved to become a project), planning (when a suitable project environment is established and approval is given to begin substantive work), and execution (when project's outputs are produced, delivered and implemented). Throughout, the project manager is accountable for delivering all project outputs.
If, however, outcomes are central to success, then a project's life should be extended to accommodate outcome realization. Outcome realization is an additional phase of a project whereby, through a program of appropriate intervention, attempts should be made to secure the flow of target outcomes within an acceptable period of time (Figure 6).
Accountabilities under the New Model—The New Role of the Project Owner
According to the ITO model, outcome realization is associated with an increase in the level of funder satisfaction. This suggests that making someone accountable for outcome generation could make a significant contribution to project success. Once this is accepted, the next issue to be resolved concerns which person should fill this role. Participants in the empirical study described earlier were also asked to identify the person who actually filled the role in the last project in which they were involved. Table 9 summarizes their responses.
Table 9. Accountability for Outcome Realization in Practice
| Accountable Role | Number of Occurrences | Percentage |
| Project manager | 47 | 52% |
| The funder (sponsor) | 11 | 12% |
| Program manager | 8 | 9% |
| Senior manager | 8 | 9% |
| No one was accountable | 7 | 8% |
| Line/functional manager | 5 | 6% |
| Team member | 3 | 3% |
| Someone else | 1 | 1% |
Table 9 shows that in the majority of projects, the project manager was not only accountable for output delivery but also for outcome realization. This suggests that in practice, no one is identified by funders as holding specific accountability for outcome realization and hence this role falls (by default) to project managers.
Because the outcome realization process is so different from output delivery, the role of the person who is accountable requires a formal definition (made up of definitions, objectives, processes, and personal attributes). Under the new model, accountability for outputs remains with the project manager, but because “benefits are not delivered or realized by the project manager and project team” (Cooke-Davies, 2002), the question of accountability for target outcomes now needs to be answered. At first look, it would appear that the project funder is self-accountable. While a view could be taken that achievement of target outcomes is the business of the funder and no one else, the issue is complicated by the circumstances under which a funding decision is made. Two situations surrounding the acceptance of a business case are common. The first is where the idea for a project (and an associated business case) has come from someone other than the funder (often called the project “champion”). In that case, the funder could well hold the champion accountable for the outcomes that he or she has claimed as benefits in the business case. The second is where the funder is too busy to play an active day-to-day role in the conduct of the project. In that case, someone would have to be appointed to look after the funder's interests. Whether filled by the project champion or by another player altogether, a new role emerges whenever a business case is accepted and a new project begins. This new role is identified here as that of project owner, who becomes the funder's agent.
The project owner is defined here as the person held accountable by the funder for securing the project's target outcomes. For a variety of reasons, this role should not be filled by the project manager. One of these relates to potential conflicts between the interests of the funder and the project manager in the project. Another relates to the suggestion that project managers usually have a mindset that better suits output delivery, but not the realization of organizational benefits (Kerzner, 2009). A third arises from the fact that a project manager is a transient position—to which the attachment of long-term accountabilities may be most inappropriate (in fact, project managers are frequently appointed on a short-term contract). All these suggest that the role should be filled by someone else in the organization—typically a senior executive and commonly a manager who will be later responsible for the ongoing operation of the project outputs. For example, if a project is to deliver a new incentive scheme so that staff turnover can be reduced, it would be appropriate to appoint the head of human resources as the project owner. In general, the project owner may be a functional manager, a business owner, the project champion or the project funder himself or herself.
This separation of the roles of project manager and project owner can be used as the foundation for a project governance model in which the project owner becomes the project manager's client and the project manager becomes the project owner's supplier. Such a separation also removes some constraints on the appointment of a project manager. Because the project owner acts as the funder's agent, he or she must be appointed from within the funding organization. A project manager, by way of contrast need not be a member of the funding organization's staff. Splitting accountabilities in this way allows external candidates to be considered.
There appear to be two ways of facilitating the proposed outcome realization phase: have the project owner administer it directly or appoint someone as administrator. In the latter case, the appointee could well be the person who had previously been working as project manager.
This discussion indicates not only that a project manager's accountability is limited, but also that a different person may well be accountable for each project phase. Hence, the conventional project model can be extended to identify the entities that are accountable in relevant project phases. The project manager is only one of several entities who are accountable for a project. In contrast to what sometimes happens in organizations, a project is not a “one-person show,” (featuring the project manager), but a collaborative effort. Each person takes a leading role in different phases, but they all should work as a united team to achieve shared and agreed outcomes.
Another implication of an outcomes-oriented approach is found in the formation of a project steering committee—the body that is formally charged with supporting the project owner in discharging his/her accountabilities. The steering committee is made up of a small group of powerful supporters of the project. In order to secure outcomes, the steering committee may continue to exist after project closure as a post project monitoring committee, a stage that may only require a skeleton group. A steering committee, lead by the project owner, is subtly different from most of the “oversighting” bodies recognized in conventional frameworks in that it too is concerned exclusively with the eventual realization of target outcomes.
The Modified Role of the Project Manager
The traditional role of a project manager is concerned with delivering outputs on time, within budget and to specifications (Shenhar et al., 2001; PMI, 2008). This suggests a rather narrow mindset that may prevent project managers from seeing the big picture on which wise decisions that contribute to outcome generation and long term organizational benefits depend. Shenhar et al. (1997) stated that project managers “must understand the business environment and view their project as part of the company's struggle for competitive advantage, revenues and profit.” The ITO model suggested in this paper is aligned with such an approach and requires that project managers be aware not only of a project's required outputs but also its target outcomes. Considering this, it must also be noted that awareness does not translate to accountability as far as outcomes are concerned.
Among the implications of this approach, is a requirement that project managers should not only have technical and tactical skills (required for project planning and execution), but also that they should be capable of taking a strategic view that allows them to better understand the influence of their project on organizational success. This, in turn, affects the organization's recruitment and training processes for project managers. Moreover, organizations would be well advised to foster a strong relationship between project managers and functional managers to make sure strategic and tactical project efforts are well aligned.
Project Success
Because the conventional definition of “project success” is predominantly concerned with outputsbased evaluation, a project would be judged as successful if its outputs have been delivered—even though this may have had no beneficial impact on the organization. For example, a software system that was completed on time, but which did not lead to intended reductions in inventory levels. It is clear from this discussion that we need to distinguish between the concepts of project management success (focused on outputs) and project management success (focused on outcomes).
Project success can be decided by a business case regression test in which certain of the parameters appearing in the original document (such as cost and target outcomes) are replaced with the actual values measured at the conclusion of the project. The original funder can now be asked “Would you have funded the project based this (adjusted) business case?” If the answer is “yes,” the project is judged successful; otherwise, it is judged a failure.
Project success (under the responsibility if the project owner) is dependent on actual values of the following three measures:
- Desirable outcomes,
- Undesirable outcomes, and
- Cost.
The suggested approach, which calls for the definition of target and undesirable outcomes for each project, is aligned with the contingency theory, suggesting that organizational effectiveness is dependent upon the organization's ability to adjust or adapt to the environment (Lawrence & Lorsch, 1967; Drazin & van den Ven, 1985; Pennings, 1992; Sauser, Reilly & Shenhar, 2009). Several studies have supported the relevance of contingency theory in the project environment (e.g. Keller, 1994; Sauser, et al., 2009). In this context, it means that instead of all projects having similar success measures (e.g., completion within budget), they require various outcome measures that are most relevant in the organizational context where the project is taking place.
Although irrelevant to judgments about project success, the traditional test of project performance remains relevant to the evaluation of the work done by the project manager. The accepted test of project management success (commonly called the iron or golden triangle) involves three criteria: outputs delivered on time, within budget, and to specification. The approach discussed in this paper suggests that the iron triangle is incomplete and that a test of project management success requires an additional measure, related to undesirable outcomes that are attributable to the project manager. Take, for example a new bridge delivered to its specifications—on time and within budget—but where the project manager used equipment allocated to another project that subsequently failed as a result of his decisions. The failure of the second project would have to be taken into account when judging his performance on the first.
The Utilization Map—A New Tool for Validating Project Scope
In order to make sure that all the effort that goes into each project (and all the outputs it produces) contributes to the eventual generation of target outcomes, the scope of a proposed project should be validated.
Many projects face the scoping problem at some point in their life cycle. The scoping problem arises in either of two forms: underscoping, whereby the current outputs cannot support target outcomes; and overscoping, whereby the current scope includes redundant outputs (or outputs with superfluous features). While the scoping problem is acknowledged in conventional approaches, accepted practice appears to be confined to “obtaining the stakeholders formal acceptance of ... scope” (PMI, 2008). Few, if any, suggestions are made as to how the “correctness” of the project's proposed scope might be confirmed. A new tool is required.
The utilization map—derived from the ITO model—is a device that is intended to validate project scope by making sure that all outputs and only necessary outputs are identified in the business case. It does this by identifying the particular stakeholders (identified here as “customers”) who generate target outcomes by utilizing specific outputs. The utilization map is a table in which:
- Each column is associated with a target outcome.
- Each row is associated with an output.
- Each cell shows who utilizes the output on the left to generate the outcome at the top.
To illustrate the utilization map, Table 10 refers to the hospital initiative introduced previously, (where a health department is seeking to improve the performance of its hospitals). For ease of illustration, only two of the previously suggested target outcomes are represented by the columns, while the number of outputs shown in the rows goes beyond the lists discussed earlier. (It should be noted that the entries in each cell will be absolutely peculiar to the specific details of the project.)
Table 10. A Representative Portion of a Utilization Map for the Hospital Improvement Project
| Target Committed | Reduced Waiting Times for Elective Surgery | Increased Efficiency of Hospital Administration |
| Outputs Outcomes | ||
| New administrative processes | Administrative staff Medical staff Patients | Administrative staff Medical staff Patients |
| New medical procedures | Medical staff | N/A |
| Enabling information systems and new IT infrastructure | Administrative staff Medical staff Patients | Administrative staff Medical staff Patients |
| New medical infrastructure | Medical staff | N/A |
| Programs of accredited professional development for staff | Administrative staff Medical staff | Administrative staff Medical staff |
| New organizational arrangements | Administrative staff Medical staff | Administrative staff Medical staff |
| New offices for administrative staff | N/A | N/A |
Cells in the utilization map can finish up with any combination of three sorts of entry:
- Blank (the cell is empty). No one utilizes the output at the left to generate the outcome at the top. For example in Table 10, no one utilizes the panel of preferred suppliers to reduce payment times to suppliers.
- A single utilization. One customer utilizes the output to realize an outcome. For example, only procurement staff utilizes the programs of professional development to reduce costs of procurement. (Although there are many staff, collectively they make up a single customer entity.).
- Multiple utilizations. Several customers utilize the output to realize an outcome. For example, reduced payment times to our suppliers require that the new procurement processes be utilized by both the procurement staff and suppliers.
Validating a project's scope is done by examining the utilization map in the following way:
1. For each outcomes column:
- Check to see if the combined contributions of all utilizations listed anywhere under this outcome represent an acceptable level of outcome achievement. The extreme case is where a column of the utilization map is empty (that is no customer is identified as utilizing any output to generate this outcome). This could indicate missing outputs (that is, the project is currently underscoped).
2. For each outputs row:
- Check if the combined contributions of all utilizations listed anywhere against this output represent a significant role for it. The most extreme case is where the row of the utilization map is empty (that is no customer is identified as utilizing this output at all). This could indicate that the output is redundant (that is, the project is currently overscoped).
Conclusions
The new project approach proposed in this paper emphasizes project outcomes that, in turn, support organizational benefits. This approach can be viewed as part of the “success” research school of project management (Bredillet, 2008). It is also aligned with recent benefit realization models (e.g., Baccarini, 1999; Simon, 1976; Ashurst et al., 2008; Ward & Daniel, 2006; Peppard, Ward, & Daniel, 2007; OGC, 2007) and project-strategic fit models (Benko & McFarlan, 2003). This model indicates that although senior managers do not normally influence project results directly, they have an indirect effect on the eventual levels of success by clearly defining, analyzing, and validating the relationship between project outcomes and outputs.
This paper has also suggested an additional project phase: a new project accountability (for outcomes), a new role (that of project owner), a modified role for project managers, new project success measures, and a tool to define and validate outcomes and outputs (the utilization map). This approach has been validated using triangulated design (Jick, 1979; Sackett & Larson, 1990; Scandura & Williams, 2000; Yin, 2003), involving a quantitative questionnaire, qualitative case study, and literature analysis.
However, further research is needed to validate the suggested approach in different project contexts. More specifically, because we have not addressed the quantitative validation of the utilization mechanism, future studies are required into the relationship between the utilization of project outputs and levels of project success.
Reflecting on the evolution of project management over the last few decades, this paper shows that, despite early suggestions for a goal-oriented approach (Gaddis, 1959), the current thinking is dominated by an outputs-based view of projects. We suggest that the establishment during the 1960s of professional organizations to serve the community of practicing project managers (e.g., PMI and IPMA) subtly changed the focus of the profession away from projects towards project management. We believe that while this concern with project management processes is still essential, it should be set against a wider view of projects as outcome-generating investments.
The paper also attempts to establish a closer link between projects and organizational goals and call for better integration among project stakeholders. Not only does the discussion propose an approach to give projects a role that is more goal-oriented, but it also suggests project management processes that are more meaningfully linked with organizational strategy. In particular, it is argued that, in order to achieve benefits from projects, funders should define target outcomes and tangible outputs, project managers should plan and work towards output delivery that is supportive of benefit realization, and project owners should lead an outcome realization process to ensure benefits are secured.
