The impact of project methodologies on project success in different contexts
SKEMA Business School, France
Ralf Müller, DBA, MBA, PMP
Professor of Project Management, BI Norwegian Business School, Norway
Project methodologies are an integral part of the core make-up of a project because all companies using project methodologies expect greatly improved project performance. Each company must decide the best approach to select the most suitable methodology and whether to have one or more variations of a methodology. Research is split whether implementing a standardized or customized methodology say by industry sector, project type, team experience and culture, achieves better results. Project methodologies are heterogeneous in composition and the parts (elements) of a methodology all play a role in supporting the project team throughout the project life cycle to achieve the projects’ goals. Research indicates a contingency between project success and project environment. The impact of the project environment on project success may be one of the reasons for consistently poor project success rates. This paper presents the results of the first stage of a larger study to determine the impact of the elements of a project methodology on the characteristics of project success. We investigate if different project environments impact the relationship between project methodologies and project success.
A deductive approach was applied to validate a theoretically derived research model. Nineteen interviews across seven industrial sectors and four countries were used to collect data. Pattern matching techniques were utilized in the analysis to deductively validate a theoretically derived research model.
The findings show that environmental factors do impact the use of a project methodology and its elements with resulting characteristics in the project success. New insights from the interviews highlighted the importance of understanding the origins of a company’s methodology and a way to categorize levels of methodology customization when looking at the impact of environmental factors for each level of customization.
The findings should benefit practitioners by allowing them to understand the impact of a project methodology on project success, moderated by the project environment. PMOs will understand the need to customize their organization’s project methodology(s) according to project type and environmental context. For management who are considering replacing an institutionalized methodology (including ones with derivatives of their main methodology), the findings highlight the importance of understanding context and how this is reflected in their incumbent methodology, so that an informed decision can be taken on how and whether they should replace the incumbent project methodology.
Keywords: methodologies; project context; contingency theory; governance; project success
Project failures are estimated to cost hundreds of billions of euros yearly (McManus & Wood-Harper, 2008) and are not limited to any specific region or industry (Flyvbjerg, Bruzelius, & Rothengatter, 2003; Nichols, Sharma, & Spires, 2011; Pinto & Mantel, 1990).
Project methodologies have been developed specially to help address low success rates using project-related knowledge (The Standish Group, 2010; Wysocki, 2011). Government bodies have helped to establish standards in methodologies and guidelines, with their tools, techniques, processes, and procedures ( Morris, Crawford, Hodgson, Shepherd, & Thomas, 2006). Research has shown that project methodologies provide more predictable project success than projects that do not use one, (Lehtonen & Martinsuo, 2006; Wells, 2012). This is supported in a recent literature review of project success factors (Khan & Turner, 2013) where 2 of 74 success factors variables identified, are directly related to methodologies as follows:
- Experience with using methodologies
- Correct choice of project methodology and tools
The other 72 success factors variables should be either directly or indirectly reflected within a project methodology.
For projects that do use project methodologies, there are still high project failure rates for these projects (Wells, 2012). This suggests issues in the selection of project methodology tools and techniques and/or the appropriate project management experience as described in the two success factors variables mentioned previously.
Maturity models have been developed such as “The Office of Government Commerce (OGC) PMMM1 or PMI’s Organizational Project Management Maturity Model (OPM3®)2, in response to requests from both the public and private sectors for a benchmark against which they can assess and demonstrate their corporate project management capability” (Crawford & Pollack, 2004). Maturity models have helped to improve project success on a repeatable basis, but only with the right organizational culture (Yazici 2009). There is a risk when a company implements a project methodology and improves it as outlined by a maturity model, but neglects to take into account one or more environmental factors such as culture, project type, governance, typology, adequacy of information, leadership style, and their impact on methodology and usage.
Maturity models may help to ensure a methodology and its elements are implemented within a company but provide no indication of which methodology elements are susceptible to being used or not used (switched-on or off) due to the impact of project environment. Therefore, some companies are implementing methodologies with the expectation of achieving high project success rates, but are not aware that project context has a major impact on the effectiveness of a project methodology (Shenhar & Dvir, 2002). The relationship between methodology and project success cannot be seen in isolation due to the potential impact of the project environment, which justifies the use of contingency theory in this study.
The purpose of this study is to investigate whether or not there is a relationship between a methodology and project success, and if this relationship is impacted by the project environment (such as governance or culture).This will provide the knowledge for companies to help customize project methodologies to their environment, therefore minimizing the chance of methodology elements at risk of being sub-optimally used while also allowing at-risk methodology elements to be proactively monitored.
To achieve the study’s purpose the following research question is posed:
Is there a relationship between the project methodology’s elements and project success which is influenced by the project context?
The unit of analysis is the relationship between the project methodology and project success.
The aim of the paper is to qualitatively validate the constructs of a theoretically derived research model, clarifying terminology, to gain additional insights that may help to steer the direction of a greater study on methodologies, their elements, and impact on project success.
The overall methodological approach of the study is deductive. The authors qualitatively validate the research model (see Figure 1) through interviews which are inductively analyzed.
The next section provides a literature review of the research subject, followed by a description of the methodology in this study, an analysis section, a discussion, and conclusion.
This section reviews the literature on project methodologies, the impact of project context moderating effect on methodology, and the definition and measure of project success.
The use of the terms project methods and project methodologies are sometimes misused (Avison & Fitzgerald, 2003), so definitions are provided below to ensure a common understanding for the reader.
The term methodology is derived from Greek methodologia and is defined as a system of methods used in a particular area of study or activity (Pearsall, Soanes, & Stevenson, 2011).
In the project management field methodology is defined as
- A system of practices, techniques, procedures, and rules used by those who work in a discipline (PMI Publishing Division, 2013)
- A collection of procedures, techniques, tools, and documentation aids which will help the system developers in their efforts to implement a new information system. A methodology will consist of phases, themselves consisting of sub-phases, which will guide the system developers in their choice of the techniques that might be appropriate at each stage of the project and also help them plan, manage, control and evaluate information systems projects (Fitzgerald, Russo, & Stolterman, 2002).
The term method is derived from Greek methodus and is defined as a particular procedure for accomplishing or approaching something, especially in a systematic or established manner (Pearsall et al., 2011).
In the project management field, method is defined as: Supporting some aspects of project management (OGC, 2002).
The scope of the term methodology used in the paper includes the elements (parts) of a methodology including the Knowledge Areas, capability profiles, tools, techniques, processes and methods which are applied to a project.
Literature is split on whether project methodologies directly contribute to the goals (Cooke-Davies & Arzymanow, 2002; Fortune & White, 2006; White & Fortune, 2002) or to the perceived appropriateness of project management (Lehtonen & Martinsuo, 2006). Methodologies are either referenced in the literature as a whole (The Standish Group, 2010), or by one or more aspects of project management practice (implying it is method, hence part of a methodology) and investigating the impact of these practices on project success (Cooke-Davies & Arzymanow, 2002, 2003; Milosevic & Patanakul, 2005).
Methodology elements are the underlying foundation of project success factor variables. The difference between a methodology element and a success factor variable is in the description. A success factor variable contains an adjective used to describe its syntactic role to qualify the underlying methodology element. For example project scheduling is a methodology element whereas efficient project scheduling is a success factor variable.
Taking one methodology element at a time and determining the impact on project success does not give a homogeneous picture of how the elements of a methodology impact the characteristics of the project success. Some methodology elements have a greater impact on the project success characteristics than others. Literature describes this by means of the term “project success factor variable,” which implies the importance of the underlying methodology element. Not all the methodology elements are described in a syntactic role as success factor variables, which implies either they have less impact on the characteristics of project success or that research has not uncovered their importance. During the project life cycle, the importance of a methodology element varies according to the phase within the life cycle (Loo, 2003). This is the same with each project success factor variable (Belassi & Tukel, 1996; Zwikael & Unger-Aviram, 2010). Methodology elements are used or not used (effectively switched on-or-off) according to need. Likewise, success factor variables need to be in place and executed when required during the project life cycle (Pinto & Prescott, 1988).
This leads to a research gap to understand how the elements within any given methodology impact the characteristics of project success.
Proposition 1: - There is a relationship between project methodology and project success.
Project Context Moderating Effect on Methodology and Project Success
The Standish Group put the selection and use of a project methodology as one of the top ten contributing factors to project failure (The Standish Group, 2010). The report states that project methodologies have provided improvement to project success (35%), in contrast to the rate of failure (19%) and challenged project performance (46%). The conclusion is that closer attention should be given to the correct choice and application of the methodology and tools. Cooper (2007) observed that many companies are mismanaging projects because they are using tools and techniques that are not appropriate for the project type or apply financial selection criteria that are not appropriate for the project type.
Several researchers (Fortune & White, 2006; Shenhar, Tishler, Dvir, Lipovetsky, & Lechler, 2002) show that it is not just using a methodology that leads to project success; it is the experience of using a project methodology and the ability to tailor to the context of a project that is linked to project success.
Many organizations have responded to low project success rates by requesting the project management associations to develop benchmarks and new models to help improve project management to achieve increased project success rates. These requests resulted in the introduction of maturity models such as OPM3® maturity model from PMI, Prince23 maturity model (P2MM) and the Portfolio, Program and Project Maturity Model (P3M3) from the UK government agency Office of Government Commerce (OGC). Other maturity models have also emerged with reduced scope typically in a Knowledge Area domain, for example, Risk Management Maturity Model, Earned Value Maturity Model (Stratton, 2006).
The expectation on the use of maturity models is to see long-term improvements in project success rates. Research has shown, to the contrary, limited long-term benefits, resulting from the application of maturity models within organizations (Judgev & Thomas, 2002), except when project methodologies are implemented within certain organizational cultures (Yazici, 2009).
Research on project methodologies is mainly focused on whether project methodologies should be standardized (Milosevic, Inman, & Ozbay, 2001; Milosevic & Patanakul, 2005), or tailored to the project environment (Lechler & Geraldi, 2013; Payne & Turner, 1999; Pinto & Mantel, 1990). Lehtonen and Martinsuo (2006) sum up the research dilemma by stating “The confusion in research results is reflected also in companies’ swing between standardized and tailored systems, and between formal and chaotic methodologies.”
A management theory developed over 50 years ago called contingency theory suggested that there is no single best way to manage and structure an organization (Burns & Stalker, 1961; Woodward, Dawson, & Wedderburn, 1965). Contingency theory, has since then been applied to project context with the first studies in the late 1980’s (Donaldson, 2006). Contingency theory in the field of project management has been applied to topology of projects with minor and major impacts (Blake, 1978), innovation types in business (Steele, 1975), product development project types (Wheelwright & Clark, 1992), leadership styles for project and functional managers in organization change (Turner, Müller, & Dulewicz, 2009), project procedures tailored to context (Payne & Turner, 1999), leadership styles per project type (Müller & Turner, 2007a), project type and the ability to select appropriate management methods linked to project success (Shenhar and Dvir, 1996; Boehm and Turner, 2004).
Fitzgerald, Russo, and Stolterman noted that the most successful project methodologies are those developed for the industry/organization which are aligned to the context factors (2002).
Research on success factors topics such as leadership competency profiles (Müller & Turner, 2010), stakeholder management (Turner & Müller, 2004) and HR management (Belout & Gauvreau, 2004), take into consideration project context which may or may not be reflected or used in the respective project methodologies.
There is a research gap on the impact of individual elements of the applied project methodology on project success and if this relationship is moderated by the project environment.
Proposition 2: There is a moderating effect of the project environment on the relationship of methodology and project success
To achieve a common understanding of what is project success, it needs to be defined in terms of success criteria (Müller & Turner, 2007b). Success criteria are the measures used to judge on the success or failure of a project; these are dependent variable that measure success, Morris and Hough (1987) cited by Müller & Jugdev, (2012).
To achieve project success, success factors need to be in place across the project life cycle (Pinto & Prescott, 1988). Neither Prince2 nor A Guide to the Project Management Body of Knowledge (PMBOK® Guide)—(Fifth Edition) (PMI Publishing Division, 2013) define the term success factors, but both standards make use of the term.
The following definition of project success factors will be used in this study:
- Project success factors are elements of a project, which, when influenced, increase the likelihood of success; these are the independent variables that make success more likely (Turner, 1999).
- Project success criteria are the measures used to judge on the success or failure of a project; these are the dependent variables that measure success (Morris & Hough, 1988).
These definitions for project success factors and project success criteria will be used in the interviews as well as the quantitative research to ensure a common understanding of terminology.
The literature implies the relationships shown in Figure 1, but do not indicate that it has been tested. The literature review also indicates a lack of understanding of the relationship between methodology elements and their impact on success characteristics, and the possible moderation by the project environment.
A philosophical stance of post-positivism was used in the study. Post-positivism assumes subjectivity in the collected data and allows theories, background, knowledge, and values of the researcher to influence the study. Post-positivists, like positivists, pursue objectivity by recognizing the possible effects of biases.
A deductive approach was taken to validate the model shown in (Figure 1). Data collection was done through semi-structured interviews. Interviews were used so as to gain a greater depth understanding of how the interviewees understood in what way project methodologies performed within their environment in terms of impacting the characteristics of project success and if the project environment influenced the relationship of project methodology and project success. Project methodologies are described using different terminologies and therefore a definition was required to create a generic understanding of the parts of a methodology. The findings will be used for a greater study to achieve generalizable results.
Development of Data Collection Instrument
The interview questions were derived using contingency theory as a theoretical lens for proposition 2 (see Appendix 1).
Six sets of questions were addressed:
- Nature of the organization and the type of projects run within the organization.
- Project methodology(s); how it was originally developed and evolved, project types supported, strengths and weaknesses.
- Project success; company definition.
- Impact of a project methodology on project success.
- Impact of the project environment (including governance) on the relationship between methodology and the characteristics of project success.
- Other comments from the interviewees rating to project methodology(s), project environment, and project success.
The first set of questions was to obtain an understanding on the organization’s business area, core business, size and types of projects including, complexity, technical challenge, and pace. The questions relating to project types and characteristics (urgency, complexity, technology), were taken from (Shenhar & Dvir, 2007; TSO, 2009) which is shown in Appendix 2. This should then provide some context regarding the choice of the company’s methodology(s) and level of customization.
The second set of questions relates to the methodology(s) within the organization, to understand whether it is based on an international or internally developed standard, where there are variations of the methodology for different project types, and what are the strengths and weaknesses.
The third set of questions concerns the definition and interpretation of project success, whether project success criteria were defined within the organization, and whether there are any published figures.
The fourth set of questions addresses the impact of project methodology on project success.
The fifth set of questions refers to the moderating effect of project environmental factors on methodology and project success, then focuses on one moderating environmental factor, that is, governance.
Figure 1: Research Model
A theoretical sampling method was used to determine the interviewees’ list, meaning the interviewees who have the best knowledge of the research subject. The number of interviews was determined by theoretical saturation (Miles & Huberman, 1994). The data was collected from several industries and geographies so as to find commonalities and differences in order to understand the relationship between the variables (Figure 1).
The authors conducted 19 semi-structured interviews, then theoretical saturation was reached. Participants were from 19 different organizations in 7 business areas and categorized using the Reuters categorization system (Reuters, 2013), the business areas included industrial, food and beverage, technology, financials, energy, telecom services and research that spanned four countries (Switzerland, USA, UK, and Germany). The demographic information is summarized in Appendix 2. The level of the interviewees varied from project manager, program manager, and PMO lead to CTO and COO; therefore, some relevant information especially regarding the usage of the methodologies and their purported strengths and weaknesses needed to be considered against the level of the interviewee.
The interviews were semi-structured and lasted between 60 and 90 minutes. Interview notes and recordings were written-up and compared for cross validation. When additional questions or clarity were required, follow-up was done using Skype sessions and emails.
Data Analysis Method
Every interview was recorded and notes were taken at the same time. Each interview was transcribed within a day as recommend by (Miles & Huberman, 1994). The transcripts were read several times in order to become familiar with the data in detail (Eisenhardt, 1989) and then coded into the NVIVO software tool according to the topics and questions. Data reduction, data display, and deducing conclusions and verification techniques were used to analyze the interview data following the guidelines outlined by Miles & Huberman, (1994).
As the interviews progressed, a methodology categorization was developed to show the positioning of a methodology in terms of origins and levels of customization.
Validity and Reliability
Once the findings had been derived from the interview notes and transcripts, to ensure that the findings were credible, the checklist by Miles & Huberman, (1994), pages 278-279, was used to inspect the analysis of the processes and the results. The reliability and validity are assured by considering the following:
oInterview protocols to be reviewed by peers and the data collected from a spread of industries and countries (USA, Switzerland, Germany and the UK).
oIdentified patterns were cross-validated for reliability.
oInternal validity: Concept validity was given through the theoretically derived model, which was built on existing literature, and from which the propositions were drawn. Construct validity was achieved through convergence of the interviewee data.
Analysis and Results
This section is structured into two parts: findings relating to proposition 1, findings relating to proposition 2.
Findings Relating to Proposition 1
Definition of the parts of a methodology—elements. The unit of analysis is the relationship between project methodology parts and project success. However, the term “parts,” from the authors perspective does not seem to be appropriate, therefore a commonly understood term was required. The literature does not provide an agreed term. This may be due to research focusing on impact of a methodology as a whole, on project success, or the use of one part of the methodology, such as scheduling or risk management and its impact on project success. The study looks at all of the parts of the methodology where the independent variable, project methodology, includes Knowledge Areas, methods, processes, tools and techniques, and capability profiles.
Interviewees were asked to provide a term that encompasses all parts of their methodology where the majority said “elements” was the term to use.
Project Success. To understand what project success is, success criteria need to be defined; otherwise success could mean something different to each person. The interviewees were asked if project success is defined within the company. None of the 19 interviewees said their company has a standard definition for project success. When asked how they were evaluated on their projects, the majority mentioned cost, time, budget, and sometimes customer satisfaction. For the research companies, success was described in terms of number of ideas, number of ideas moved to development, and industrialized.
The interviewees were asked whether the project methodology within the company impacts the characteristics of project success.
Figure 2: Project methodology impact on project success characteristics
Figure 2 shows that project methodology does impact the project success characteristics where the highest references were to time, cost and scope. One of the interviews stated “Yes, 100%” and then described method elements that if not executed correctly, would impact the characteristics of project success, “Requirements management not followed through—insufficient scope development and insufficient governance around changes.”
These findings support proposition 1: There is a relationship between project methodology and project success
Findings Relating to Proposition 2
Evolution of project methodologies. Project methodologies have been evolving and adapting over the years through need and perceived impact on project success. Morris and Pinto describe this by writing “it’s time to move on project management from a rather tired and dated positivist or normative origin stemming with its roots firmed in engineering companies to perhaps where it needs to reflect much more in a complex reality, such as organizational change-type projects where interpretive views of the reason for change are more appropriate” (2004). The international standards, such as PMI’s PMBOK® Guide or the UK Office of Government’s Commerce (OGC) Prince2, are updated every few years and include extensions for government, construction, defense, and the software industries.
Figure 3: Methodology origins and the levels of customization
Figure 3 was developed from the interviews to help structure the source and levels of methodology customization. Approximately 65% of the interviewed companies’ methodologies are based on an international standard, where 75% of these companies customize the international standards to varying degrees. About 35% of the companies interviewed have more than one methodology that is customizable per project type. Two of the interviewees who were both in the software consulting business, explained their companies have over 40 methodologies for different applications, industries, and project types. This shows the apparent need and benefit for some companies to employ specialized methodologies according to the application, project type, and business area. None of the companies interviewed said that their project methodologies were customized down to the level of the project team and skills, however this may be implicitly done by the project teams in companies that allow further levels of customization.
These findings indicate that environmental factors have a moderating effect on the relationship between project methodology and project success, because the interviewees’ companies are invested into creating and maintaining customized/tailored methodologies. One of the interviewees said that “the company culture impacted whether the elements of the methodology were used or not, typically change management, risk management, and issue management were not used or done properly,” reiterating the moderating effect of environmental factors.
Impact of the project environment (including governance) on the relationship between methodology and the characteristics of the project success. Environmental factors are conditions or things that are outside of the immediate control of the project team, that influence, constrain, or direct the project, program, or portfolio (PMI, 2013). These factors create the context for the project and how it should be managed. The interviewees were asked which internal environmental factors have an impact on the relationship or the way methodology elements are used to achieve process success. Referring to Figure 4, governance was the most frequently mentioned environmental factor. One interviewer commented that it is challenging to get the right governance structure for a project “as clients often felt they did not have the time for governance.” Another interviewee mentioned that governance provided the “checks and balances” for other environmental factors such as politics, power, and the effectiveness of the sponsor. Governance may have been raised more times than the other environmental factors, because it can be considered as an institutional factor, whereas the other factors are associated with individuals and change more frequently.
Figure 4: Internal environmental factors impacting project management success
Referring to Figure 5, the same question was asked as in Figure 4, but as it pertains to external environmental factors. Only a few interviewees mentioned the external environmental factors probably because the roles of these interviewees were in supporting government institutions or in the general consulting area. There was no single external environmental factor that was more prominent than others. These external factors were as important for the impact of the effectiveness of the project methodology, as the internal factors are, for those interviewees whose project environment was primarily internal. One interviewee stated that “when dealing with the government, things are never clear from the beginning” and another interview stated that “government can suddenly change priority immediately such as the government shutdown or regulations on hiring.” Interviewees working in consulting positions for companies raised the issue of client culture that is not conducive for projects including a lack of understanding of what is required in project management.
Figure 5: External environmental factors impacting project management success
Both internal and external environmental factors act as moderating variables.
Methodology elements impacted by the moderating effect of environmental factors. The interviewees were asked which elements of their project methodology(s) relationship to project success, were impacted (moderated) by environmental factors but without specifically focusing on any one environmental factor. Referring to Figure 6, stakeholder management and change management were the top two methodology elements that were impacted by environmental factors. One of the interviewees mentioned that the culture of the organization of showing good results meant that reports were changed to reflect this. Another interviewee working in an external role responded to the question “methodology elements are impacted 100%, first, requirements management; second, insufficient scope development; and third, insufficient governance around change management.” One interviewee made an interesting point by stating that “their project methodology, which was specifically developed in-house developed takes into account company culture,” so it reflected the context of the organization.
Figure 6: Methodology elements’ relationship to project success impacted by environmental factors
The interviewees were then asked to take governance as the environmental factor and say which elements of their project methodology(s) relationship to project success, were impacted (moderated) by governance. Referring to Figure 7, cost and stakeholder management were mentioned the most as being impacted by governance. The references to the impact on the stakeholder management methodology element were positive and negative (used and not-used). On the positive side, some interviewees primarily in the consulting area would design the stakeholder involvement and decision making around the governance structures to ensure a full alignment with all decisions taken. On the negative side, others mentioned stakeholders being left out of critical parts the project management life cycle mainly due to ill-fitting governance structures.
Figure 7: Methodology elements relationship to project success impacted by governance
One of the interviewees involved in software consulting said that “during the process of bidding for the work, which can last for months, the governance structures are well defined before the contract is signed so that any decision taken to deviate from the plan in terms of using the elements of the methodology are agreed in writing through the various levels.” Another interviewee explained that “governance is used at the setup of the project where the project manager has to justify why elements of a methodology will not be used.” This was the only reference given where every methodology element must be used unless there is justification on why not to use an element.
The two methodology elements—cost estimating and cost control—were raised in both the positive and negative contexts. Projects are in control with proper cost estimation (using the cost control element) and projects are out of cost control due to lack of governance impacting the use of the cost control element.
The findings from the interviews support proposition 2: There is a moderating effect of the project environment on the relationship of methodology usage and project success.
The interview results showed the importance of project methodologies as they directly impact the characteristics of project success. This is consistent with the finding in the literature (Belassi & Tukel, 1996; Zwikael & Unger-Aviram, 2010).
An applied project methodology is a collection of elements that collectively impact the characteristics of project success. However, some of the elements may have a greater impact than others. This may be supported because project success factor variables reference one or more underlying methodology elements, but not all methodology elements relate to known success factor variables.
Taking a totally different perspective on methodologies may provide additional insight into why some elements of a methodology have a greater impact on the characteristics of project success than others. A natural science comparative model of Joslin & Müller, (2013) compares project methodology elements to the genes of an organism. The genes of an organism are the building blocks of the organism (including the observable characteristics) called a phenotype (Malcom & Goodship, 2001). Genes are switched on and off throughout the life of an organism, which the authors argue is the same concept as elements of a methodology being applied, when required, to a project throughout its life cycle. Some of the genes in an organism are highly pleiotropic, meaning their impact can be seen in the organism’s phenotype, for example, hair color, eyes, and height (Stearns, 2010). The comparative explains the same is true for elements of the applied project methodology. The highly pleiotropic methodology elements noticeably impact the characteristics in project success. Returning to the interviews, some interviewees discussed the impact on project success of certain methodology elements when they were not used, due to the impact of environmental factors. The examples given included change requests, risk management, and deliverable sign-offs. The resulting consequences on the characteristics of project success included increased costs, quality issues, and reduced customer satisfaction. These could be examples of highly pleiotropic methodology elements. This alternative perspective may provide new insights that may not be possible when using a social science perspective.
The literature on project success factors variables relating to project methodologies refer to the experience of using a project methodology and the ability to tailor a project methodology to the context of a project (Fortune & White, 2006; Shenhar, et al., 2002). The words experience and tailor suggest that project context plays an important factor in achieving project success. Contingency theory within the field of project management offers insight into how to best adapt project management practices within a given environment to meet the project management goals. Contingency theory applies to selecting and tailoring the project methodology according to the environment. The findings from the interviews show that the effectiveness of the methodology to achieve project success is moderated by the project environment. Governance was the most often mentioned environmental factor impacting the effectiveness of the applied project methodology. Examples were given of ill-fitting governance structures impacting the ability to follow procedures to obtain resources, finalize requirements, test strategies, and quality assurance. The findings did not go so far as to suggest actions to enhance the positive aspects and minimize the negative aspects of the environmental governance factor.
This study’s findings show the significance of project context in terms of the number of companies customizing their methodologies, the number of methodology variants within a company, and the impact of environmental factors including governance on the relationship between project methodology and project success.
This qualitative study interviewed 19 project, program and senior IT managers from seven industries across four countries who all have detailed knowledge of their company’s methodology(s). A deductive approach was used to validate a theoretically derived research model.
The findings show that environmental factors impact the use of the project methodology and its elements with resulting characteristics of project success. This has helped to achieve the research aims to qualitatively validate the constructs of the research model, gain agreement in the use of the terms methodology elements and project success, and gain additional insights, such as the importance of understanding the methodology source and levels of customization.
The Practical Implications
- When management is considering replacement of an institutionalized methodology (including one with derivatives) the importance of context should be understood and how this is reflected in incumbent methodology. With this information, an informed decision can be taken.
- For project managers using a project methodology, there is a risk of suboptimal project performance because the effectiveness of methodology elements are negatively impacted by environmental factors. The project manager should understand which methodology elements are the foundation for success factor variables and understand and manage potential reduced effectiveness of these methodology elements which will greatly increase the risk of project failure.
- Practitioners should benefit, in general, by creating knowledge of which elements of the project methodology for different project types are impacted the most (correlated) according to the governance paradigm. The practitioner focus being on the methodology elements which are the highest correlated to project management outcome.
- Project management offices (PMOs) will be able to collect data on methodology element performance with respect to the project context and create more resilient project methodology(s). PMOs should not inhibit the ability of project managers to customize methodologies but should ask for justification in terms of understanding and quality assurance.
The Theoretical Implications
- Governance plays a major role in the moderating effect of project methodology performance.
- Contingency theory is applicable to methodology selection and its customization according to the project environment.
- A methodology’s effectiveness is constantly being impacted by the project environment where the characteristics can be seen in project success. Viewing a project methodology from a natural science perspective may bring new insights into the behavior and effectiveness of methodologies in different contexts.
Strengths and Limitations
The study collected data from various industries and countries to theoretically derive the research model. The depth of the interview discussions and the experience of the interviewers helped to provide rich data, generating new insights which would not have been possible from an on-line survey.
The study is based on interviews with a small sample size so the results cannot be generalized.
- To better understand how generic versus tailored methodologies are impacted by environment factors:
oAre the elements of methodologies that are generic more or less likely to be used than elements from a highly customized methodology?
oIs there a commonality between the environmental factors impacting the elements of a generic versus a highly customized methodology?
- To review existing literature with references to companies using standard methodologies to understand if they were generic thus standardized; or generic then customized, then standardized; or customized, therefore standardized. This information would provide a solid basis of research into methodologies, context, effectiveness and efficiency.
- To understand if there are also efficiency gains in some organizations areas when using standardize methodology elements within a customized methodology.
Contributions to Knowledge
The value of this study lies in the following:
- A methodology should be seen as a collection of elements impacting the characteristics of project success, where some elements are the foundation of success factor variables.
- Identification of environmental factors especially governance impacting the relationship of project methodology and the characteristics of project success.
- To provide empirical data for a prestudy that is in a new field of study with a new method. A natural to social science comparative was created comparing project methodology elements to genes of an organism Joslin & Müller, (2013). The results of this study in conjunction of greater study will be used to determine the validity of the new comparative.
Appendix 1: Interview Protocol
1. Nature of the company and types of projects within your company
- What types of business activities are carried out in your company?
- What types of projects are carried out in your company?
- What categories of projects are undertaken? Compare with Table 1
- What is the criterion to judge project size in terms of small, medium, and large in your company?
Table 1: Categories of project types
|Application||Project Type||Internal / External||UrgencyA||InnovationA||Size||TechnologyA||ComplexityB|
organizational change, service improvement,
|low, medium, high, |
|task, simple project, |
AThe ranges of Urgency, Innovation and Technology taken from Shenhar’s Diamond model (Shenhar & Dvir, 2007)
B The ranges for Complexity are taken from Prince 2 (TSO, 2009)
2. The project methodology(s); how it was originally developed and evolved, project types supported, strengths, and weaknesses.
- Please describe the project methodology or methodologies your company uses including whether it is based on an international standard like Prince2, Prompt, or PMBOK® Guide?
- If the methodology was based on an international standard, then was the methodology tailored to your business and, if so, was it tailored per project type or per business section?
- If the methodology was developed within your company was it developed for a specific product or service? Please describe its background.
- Are there derivatives of the methodology for different types of projects or business areas and, if so, describe why?
- Please describe the strengths and weaknesses of your project methodology
- Are there certain types of projects that your methodology is less or more suited to?
- Does your project methodology evolve to meet organizational needs and, if so, how does it evolve? Also, who is responsible for its evolution?
- What would you recommend to improve the value of your organization’s methodology?
- Looking at the methodology, what word would you use to describe the parts of the methodology (hierarchical breakdown) in a generic sense?
- Does you project methodology for any given project type—integrate the “how to build” something with the “what to build” or is the “what to build” (requirements specs) kept separately?
- Would there be any advantages or disadvantages in combining a methodology, and what needs to be built into one integrated approach?
3. Project success (success)
- Is there a definition of project success in your organization?
- Is there a definition of project success for your project(s)?
- Are there any numbers published on project success rates?
4. Impact of a project methodology on project success.
- Have you observed the project methodology, including how its elements impact the characteristics of project success?
5. Project governance paradigm based on Müller (2009) and how it relates to the goals of the organization/shareholders.
Background: The corporate governance of a company can be modeled on a continuum from shareholder orientation to stakeholder orientation. In shareholder-oriented companies, all decisions that are made are driven by the underlying desire to maximize the wealth of the company’s shareholders. In stakeholder-oriented companies, there is still a need to create profit to satisfy the needs of the shareholders, but this is only one of a variety of stakeholder groups.
- Where on this continuum would you place your company?
- Is there a management philosophy with emphasis on always getting personnel to follow the formally laid-down procedures or a strong emphasis on getting things done even when this means disregarding formal procedures?
- Is the project manager responsible for time, cost, budget and/or any other measure?
- Is the reason why the project manager is responsible or not for something due, in some way, to the governance paradigm used within your company?
6. Impact of the project environment (including governance) on the relationship between methodology and the characteristics of the project success.
- Which environmental factors have an impact on the relationship or way methodology elements used to achieve process success?
- Consider the governance paradigm impacting your project(s). How has governance, as an environmental factor, impacted the relationship or manner in which methodology elements are used to achieve process success?
7. Anything else you think is important to add?
Appendix 2 Interview Data Overview
AThe attributes of the business classification column is taken from Reuters Business classification (Reuters, 2013)
BThe attributes of the Project types column are derived from project categorization systems (Crawford, Hobbs, & Turner, 2004) CThe attributes of the Urgency and Technology columns are taken from Shenhar’s Diamond model (Shenhar & Dvir, 2007)
DThe attributes of the complexity column are taken from the Prince2 definition of complexity (TSO, 2009)
EThe scaling of small, medium and large was dependent on the interviewees’ responses. The numbers were normalized into small, medium and large.
Avison, D., & Fitzgerald, G. (2003). Information systems development: methodologies, techniques and tools. EP Patent App. 20,040,768,169 (pp. 1–12).
Belassi, W., & Tukel, O. I. (1996). A new framework for determining critical success/failure factors in projects. International Journal of Project Management, 14(3), 141–151.
Belout, A., & Gauvreau, C. (2004). Factors influencing project success: The impact of human resource management. International Journal of Project Management, 22(1), 1–11.
Blake, S. (1978). Managing for responsive research and development. San Francisco: WH Freeman & Co.
Boehm & Turner 2004. Balancing Agility and Discipline: A Guide for the Perplexed. Boston: Addison-Wesley Professional.
Burns, T., & Stalker, G. (1961). The management of innovation. University of Illinois at Urbana-Champaign’s (pp. 103–108).
Cooke-Davies, T., & Arzymanow, A. (2002). The “real” success factors on projects. International Journal of Project Management, 20(3), 471–478.
Cooke-Davies, T., & Arzymanow, A. (2003). The maturity of project management in different industries. International Journal of Project Management, 21(6), 471–478.
Cooper, R. (2007). Managing technology development projects. IEEE Engineering Management Review, 35(1), 67–77.
Crawford, L., Hobbs, B., & Turner, R. (2004). Project categorization systems. Newtown Square, PA: Project Management Institute.
Crawford, L., & Pollack, J. (2004). Hard and soft projects: a framework for analysis. International Journal of Project Management, 22(8), 645–653.
Donaldson, L. (2006). The contingency theory of organizational design: Challenges and opportunities. In Organization Design (pp. 19–40). New York: Springer.
Eisenhardt, M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532–550.
Fitzgerald, B., Russo, N., & Stolterman, E. (2002). Information systems development: Methods in action. McGraw Hill Education,Berkshire, England.
Flyvbjerg, B., Bruzelius, N., & Rothengatter, W. (2003). Megaprojects and risk: An anatomy of ambition. Cambridge, England: Cambridge University Press.
Fortune, J., & White, D. (2006). Framing of project critical success factors by a systems model. International Journal of Project Management, 24(1), 53–65.
Joslin, R., & Müller, R. (2013). A natural science comparative to develop new insights for project management research. In Drouin, Müller, & Sankaran (Eds.), Novel approaches to organizational project management research: Translational and transformational (pp. 320–345). Copenhagen: Business School Press.
Judgev, K., & Thomas, J. (2002). Project management maturity models : The silver bullets of competitive advantage. Project Management Journal, 33(4), 4–14.
Khan, K., & Turner, R. (2013). Factors that influence the success of public sector projects in Pakistan. In Proceedings of IRNOP 2013 Conference, June 17-19, 2013, Oslo, Norway: BI Norwegian Business School.
Lechler, T., & Geraldi, J. (2013). Comparing apples with apples: Developing a project based contingency theory. discovery.ucl.ac.uk. IRNOP, Oslo, Norway.
Lehtonen, P., & Martinsuo, M. (2006). Three ways to fail in project management and the role of project management methodology. Project Perspectives, XXVIII, 6–11.
Loo, R. (2003). A multi-level causal model for best practices in project management. Benchmarking: An International Journal, 10(1), 29–36.
Malcom, S., & Goodship, T. H. J. (Eds.). (2001). From Genotype to Phenotype (2nd ed., Vol 2001). Oxford, England: BIOS Scientific Publishers Ltd, Oxford, England.
McManus, J., & Wood-Harper, T. (2008). A study in project failure. British Computer Society, London, UK.
Miles, M., & Huberman, A. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: SAGE Publications Inc.
Milosevic, D., Inman, L., & Ozbay, A. (2001). Impact of project management standardization on project effectiveness. Engineering Management Journal, 13(4).
Milosevic, D., & Patanakul, P. (2005). Standardized project management may increase development projects success. International Journal of Project Management, 23(3), 181–192.
Morris, P., Crawford, L., Hodgson, D., Shepherd, M., & Thomas, J. (2006). Exploring the role of formal bodies of knowledge in defining a profession —The case of project management. International Journal of Project Management, 24(8), 710–721.
Morris, P., & Hough, G. (1988). The Anatomy of Major Projects: A Study of the Reality of Project Management. New York: John Wiley & Sons.
Morris, P., & Pinto, J. (2004). The Wiley Guide to Managing Projects. Wiley Guide to Managing Projects (1st ed.). New York: John Wiley & Sons Inc.
Müller, R. (2009). Project Governance. Surrey, England: Gower Publishing.
Müller, R., & Jugdev, K. (2012). Critical success factors in projects: Pinto, Slevin, and Prescott—The elucidation of project success. International Journal of Managing Projects in Business, 5(4), 757–775.
Müller, R., & Turner, R. (2007a). Matching the project manager’s leadership style to project type. International Journal of Project Management, 25(1), 21–32.
Müller, R., & Turner, R. (2007b). The influence of project managers on project success criteria and project success by type of project. European Management Journal, 25(4), 298–309.
Müller, R., & Turner, R. (2010). Leadership competency profiles of successful project managers. International Journal of Project Management, 28(5), 437–448. doi:10.1016/j.ijproman.2009.09.003
Nichols, K., Sharma, S., & Spires, R. (2011). Seven imperatives for success in IT megaprojects. McKinsey Quarterly.
OGC. (2002). Managing successful projects with PRINCE2. (2, Ed.). London: The Stationery Office.
Payne, J., & Turner, R. (1999). Company-wide project management: The planning and control of programmes of projects of different type. International Journal of Project Management, 17(1), 55–59.
Pearsall, J., Soanes, C., & Stevenson, A. (2011). The Concise Oxford English Dictionary (12th ed.). Oxford: Oxford University Press.
Pinto, J., & Mantel, S. (1990). The causes of project failure. IEEE Transactions on Engineering Management, 37(4), 269–276.
Pinto, J., & Prescott, J. (1988). Variations in critical success factors over the stages in the project life cycle. Journal of Management, 14(I).
PMI. (2013). A Guide to the Project Management Body of Knowledge (PMBOK® Guide)—Fifth Edition. Newtown Square, PA: Project Management Institute.
Reuters. (2013). Thomson Reuters Business Classification. London: Reuters.
Shenhar, A. J., & Dvir, D. (1996). Toward a typological theory of project management. Research Policy, 25(4), 1996, 607–632.
Shenhar, A., & Dvir, D. (2002). One size does not fit all—True for projects, true for frameworks. Proceedings of PMI Research Conference, USA.
Shenhar, A., & Dvir, D. (2007). Reinventing project management: The diamond approach to successful growth and innovation. Boston: Harvard Business School Press.
Shenhar, A., Tishler, A., Dvir, D., Lipovetsky, S., & Lechler, T. (2002). Refining the search for project success factors: a multivariate, typological approach. R and D Management, 32(2), 111–126.
Stearns, F. W. (2010). One hundred years of pleiotropy: A retrospective. Genetics, 186(3), 767–73.
Steele, L. (1975). Innovation in big business. New York: Elsevier, New York.
Stratton, R. (2006). The Earned Value Management Maturing Model. Vienna, VA: Management Concepts.
The Standish Group. (2010). Chaos Report. Boston: The Standish Group.
TSO. (2009). Managing successful projects with PRINCE2. London: Office Of Government Commerce.
Turner, R. (1999). The Handbook of Project-Based Management (2nd ed.). London: McGraw Hill.
Turner, R., & Müller, R. (2004). Communication and co-operation on projects between the project owner as principal and the project manager as agent. European Management Journal, 22(3), 327–336.
Turner, R., Müller, R., & Dulewicz, V. (2009). Comparing the leadership styles of functional and project managers. International Journal of Project Management, 2(2), 198–216.
Wells, H. (2012). How effective are project an explorative evaluation of their benefits in practice. Project Management Journal, 43(6), 43–58.
Wheelwright, S., & Clark, K. (1992). Revolutionizing New Product Development: Revolutionizing New Product. New York: The Free Press.
White, D., & Fortune, J. (2002). Current practice in project management—An empirical study. International Journal of Project Management, 20(1), 1–11.
Woodward, J., Dawson, S., & Wedderburn, D. (1965). Industrial Organization: Theory and Practice. New York: Oxford University Press.
Wysocki, R. (2011). Effective software project management (6th ed.). Hoboken, NJ: Wiley.
Yazici, H. (2009). The role of project management maturity and organizational culture in perceived performance. Project Management Journal, 40(3), 14–33.
Zwikael, O., & Unger-Aviram, E. (2010). HRM in project groups: The effect of project duration on team development effectiveness. International Journal of Project Management, 28(5), 413–421.
Robert Joslin, PhD c, BSc (Durham), PgMP, PMP, CEng, MIEEE, MBCS, is a project/ program management consultant and academic researcher. He has considerable experience in designing, initiating and providing program management delivery of large-scale business transformations, reengineering infrastructure, and strategy development. He has received recognition for his ideas and product innovation. Previously, Mr. Joslin was a consultant in a wide range of industries including telecom, banking, insurance, manufacturing, and direct marketing while working for McKinsey & Co, Logica, and his own consulting company. He is currently studying for a PhD in strategy, programme and project management at the SKEMA Business School in France. He has published several book chapters, research papers, and is in the process of authoring a book on portfolio, program, and project success factors.
1 Project Management Maturity Model (PMMM)
2 Organizational Project Management Maturity Model (OPM3)
3 PRINCE2: Project IN Controlled Environment is a structured method for effective project management. The method was first established in 1989 by the Central Computer and Telecommunications Agency
©2014 Project Management Institute Research and Education Conference
Developing a Project Management Office in the Department of Energy, Energy Information Administration
This case example, a supplement to the report, PMIAA: Strengthening the Government Delivery Foundation, highlights project and program management capability building within The U.S. Energy…