Project Management Institute

How effective are project management methodologies (PMMs)?

An explorative evaluation of their benefits in practice

Abstract

This paper investigates the benefits and supports provided by Project Management Methodologies (PMMs) to project managers for the management and delivery of Information Technology/Information System (IT/IS) projects. Using a qualitative approach, through case study strategy, the role of PMMs is examined in different business and project contexts. This paper evaluates the benefit of PMMs based on their traits and characteristics and investigates PMMs in their operational context: where PMMs come from and how they support practitioners. The findings suggest a misalignment between the intended benefit of PMMs at the strategic level and the reported benefits by project managers at the project level. Additionally, it is shown that practitioners' expertise, accountability, and attitudes all have a direct influence on the extent to which PMMs contribute and benefit the management of projects.

Introduction

A report from Health Secretary, Andrew Lansley captured the attention of many by highlighting that “Labour's IT programme let down the NHS [National Health Service] and wasted taxpayers' money by imposing a top-down IT system on the local NHS, which didn't fit their needs” (BBC News, 2011). This report highlighted significant issues associated with IT/IS project delivery. Successful delivery of projects is essential to the effective functioning of government and has a direct bearing on departments' abilities to improve public services (Improving Government Risk, 2009). Despite this central role of IT/IS projects within organisations, the high cost and failure rate continue to engage researchers and practitioners.

Improving project performance by means of ensuring successful management, development, and delivery of information technology/information system (IT/IS) projects remains the top priority of most organisations and project communities (CHAOS, 2010; Yardley, 2002; Wysocki, 2007). As a way of addressing this, Project Management Methodologies (PMMs) are regularly employed with the aim of increasing project efficiency and effectiveness. Public and private sector organisations worldwide invest significant resources into efforts, ranging from a review and tailoring of the current practices to the adoption or development of new PMMs. Despite these efforts, the benefit gained through the usage of methodologies towards improving project performance of IT/IS projects has rarely been examined or articulated. Hence, this paper, which is a sub-part of a major research (Figure 1), examines the benefits and support provided by PMMs in their business context, through first-hand empirical investigation. The following section set the context and the research background for this paper.

Research Background and Literature

Organisations achieve business benefits, produce and improve products, design and develop systems and services, and invest in company infrastructure primarily through project activities (Shenhar & Dvir, 2007; Davies & Hobday, 2005). In the United Kingdom alone, around 21% of the gross value added in manufacturing and construction is through complex products and systems development projects (Davies & Hobday, 2005). As a result, a tendency towards standardising project activities by means of formalised, generic PMMs (Gunnarson et al., 2000); for example, the drive from government and the public sector towards the promotion and usage of the PRINCE21 (OGC, 2009) PMM in recent years for the development and management of large and complex IT/IS projects. With the pressure for successful delivery of projects in the relentlessly changing business environment, new PMMs are increasingly being developed, implemented, and tailored as a means of guiding and supporting project design, development, and delivery (Morris & Pinto, 2004; Davies & Hobday, 2005; Sauer et al., 2009; Charvat, 2003; Thamhain, 1994). Despite the increased popularity of some methodologies (e.g., PRINCE2) in different organisations, including public sector departments, there are reported limitations and weaknesses associated with structured PMMs (White & Fortune, 2002). Some of the difficulties that organisations and public sector departments face is the indifference of the methodologies to their organisational business interests and benefits beyond those of a single project; complexity in tailoring and modification; leadership and strategy; and, their reliance on documentation and their inflexibility to dealing with change (Al-Zoabi, 2008; Shenhar & Dvir, 2007; Boehm & Turner, 2004).

The private sector accounts for around 45% of the IT/IS project market. These organisations do not necessarily announce their IT/IS plans in advance, nor do they publicise failures, although anecdotally there is thought to be extensive misplaced expenditure and failed projects. The public sector accounts for 55% of the whole UK market for IT (Cross, 2005). Within the public sector, plans for reforming public services are heavily dependent on new large-scale information technology systems. The UK government has been spending £14 billion a year on computer systems and services, the highest figure in Europe and, in recent years, double the sum spent in 1999. Despite substantial efforts invested since the 1960s to gain a better understanding of project failure improvements in IT/IS project success rates have been slow to materialise and examples of catastrophic failure can still be found. “The UK TV Channel 4 documentary Dispatches, How They Squander Our Billions (2009) investigated a variety of controversial public projects, which have had millions, if not billions, spent on them, including the NHS IT scheme (NPfIT, which was originally announced as costing £2.3 billion) and this later became £6.2 billion, and £12.7 billion is the latest estimate.” (http://www.wellingtone.co.uk/blog/p=61).

A recent UK report on the C-NOMIS project (2009) further exemplifies the problem. “Her Majesty's Prison Service and the National Probation Service created the National Offenders Management Information System NOMIS in 2004. In 2007, NOMIS had spent £155 million and the project was two years late and, in 2008, the decision was made to put a hold on the project. The project was reduced in scope, at a cost of £41 million and was restarted in January 2008, the original due date. Lifetime costs are now expected to be £513 million, although it will no longer support end-to-end offender management as originally planned” (Oates, 2009); this is a classic example of a project delivered late, under-achieving against its specified scope and many times over budget. During periods such as the current economic downturn, sizeable financial losses in this way inflict heavy business consequences on the project owners. This can result in the cancellation of a project with both the outright loss of the resources invested to-date and the loss of the expected benefits.

Since 1999, to date PMMs are firmly placed as one of the top ten contributing factors towards project failure, according to the Standish Group (CHAOS, 2010). In the 2008 version of the CHAOS report, it was stated that, although improvements in the rate of project success (35%) are obtained, the rate of failure (19%), and challenged (46%) project performance remain at levels that deserve further attention.

The connection between PMMs used to manage IT/IS projects and poor performances are not fully unearthed, and scant research exists examining the role and contribution of PMMs towards overall project performance. At a general level, it can be said that effective PMM implementation does not automatically signify positive or enhanced project outcome; also, poor performance does not necessarily arise from weak PMM implementation. Yet, PMMs are designed and implemented to regularise project management processes in order to aid the focus upon other critical performance matters. Therefore, PMMs are significant. The review of literature has revealed fewer than five studies tackling and examining PMMs, with very few focusing on the IT/IS discipline and even fewer using empirical evidence in their arguments. White and Fortune (2002), in their quantitative study, examined a combination of methods, tools, techniques, and methodologies, deductively concluding that a number of problems experienced by organisations are associated with PMMs, such as PRINCE and PRINCE2. The study is extensive and comprehensive, although its approach limited the findings to predetermined assumptions about success. The context in which the PMMs were examined was not considered. A study by Davies and Hobday (2005) examined the effect of using Waterfall PMMs for managing a computer-based project from a complex software project in the MoD. They revealed divergence from the PMM (rational process) to soft processes, emphasising the failure of PMMs, such as Waterfall, to reflect the reality of practice. These areas of deviation are important but leave unanswered questions such as the rationale behind selecting a Waterfall approach to a complex software project and what influences this decision. One reason associated with the complexity of management of IT/IS projects is that projects are often mismanaged because companies are said to employ the wrong processes to manage or apply inappropriate financial criteria for project selection (Cooper 2007).

To date, no single set of PMMs has been identified as ensuring the successful delivery of IT/IS projects. It has been estimated that there are one thousand ‘brand name' methodologies used by organisations globally (Jayaratna, 1994). Each methodology claims to be the answer to overcoming problems faced by projects and their organisations or an answer to commonly occurring problems in particular contexts (PRINCE2, 2009). However, many of these new methodologies have been indicted as being just a repackaging of concepts that emerged in earlier times of PMMs found in software engineering and IT/IS delivery (McManus & Wood-Harper, 2008). PMMs are considered “as a fetish used with pathological rigidity for its own sake; this demonstrates significant reliance by organisations on the application of methodologies as a routine where the reasoning behind the adoption and application of methodologies are normally unanswered and left fuzzy.” (McManus & Wood-Harper, 2008, p 28)

Figure 1, demonstrates an abstract of the overall research undertaken to examine the overall role of PMM in project management, highlighting the subsection selected for discussion in this paper on examining the benefits of PMMs. Hence, this paper focuses on the following research question.

Research Question: What are the benefits of PMMs to projects and organisations?

Figure 1: Abstraction process — generic categories.

 

Abstraction process — generic categories

Given the persistence of organisations in using PMMs, this question aims to investigate what benefits and advantages are offered by PMMs to projects and organisations. Who are the main beneficiaries and how do PMMs support them in the delivery and management of projects? This exploration mainly examines the contribution of PMMs from users', project teams', and manager's perspectives rather than the PMM owners' and promoters' alone. An independent evaluation that assesses the level of support provided to projects and project managers is the angle of the investigation. There is no empirically underpinned research available that has addressed this research question. The literature presented above illuminates the justification for needing further understanding about how PMMs support performance, that is, the successful delivery of IT/IS projects.

Research Strategy and Methodology

The research methodology is phenomenological with exploratory purpose. An inductive approach and reasoning are employed given the scarcity of other research on this complex subject. No preconceived hypothesis or conceptual models were used, although the research question was validated and fully informed by the existing literature. The research strategy employed is that of a multiple-case study approach focusing on PMMs as the unit of investigation (cf. Yin, 2003). Four case studies spanning disciplines, project contexts, and types of PMM provide anchorage into frontline management of IT/IS projects. Case One focused on PRINCE2, a widely used structured PMM. Case Two concerned an in-house structured PMM. Case Three employed a gate-phased PMM. Case Four hosted a gate-phased PMM in the process of being phased out and replaced by an agile approach. Data were collected through semi-structured interviews. Each interview lasted between 90 and 120 minutes. Practitioners were interviewed (48), each being in different roles with varying levels of accountability in the design, development, and management of projects. Some of these practitioners were key decision-makers of PMM development and application.

Table 1: Schedule of interviews by case study.

Case Interviewed (Role) Interviewed (number)
Case 1 Consultancy Director
Vice President
Senior Project Manager (4)
Independent Management
Consultant
Senior Consultant
Managing Consultant
Head of BP
Managing Consultant
Academic Researcher
Managing Director
Total 13
Case 2 Senior Project Manager (5)
Programme Manager (2)
Senior Quality Engineer (1)
Managing Consultant (1)
Principal Consultant (1)
Director (1)
Total 11
Case 3 Programme/Portfolio Manager
Solution Architect (2)
Senior Project Managers (3)
Project Managers (4)
Business Analysts
Total 11
Case 4 Head of Tools and Methodologies (1)
Tools and Techniques team (1)
Senior Agile Advocate (2)
Senior IT Project Manager (3)
Product Manger (3)
Project Manager (3)
Total 13
Total     48

The collected data were qualitative. A combination of interpretative and content analysis was employed using a general inductive approach for qualitative data analysis. The unit of investigation is PMM, examined in different business contexts. This study is also strongly empirically driven and places an importance on context (Smyth & Morris, 2007). Different types of PMMs, applied in different types of IT/IS projects, and managed in a variety of business contexts, were chosen. This was to embrace a variety of sources where different approaches and PMMs are employed, such as PRINCE2 and agile for management and governance of IT/IS projects. This interpretive approach is compatible with phenomenology, enhancing data richness, allowing “horizontal” and “vertical” in-depth analyses to co-exist. Four case studies were chosen. The first case study, Case One, focusing on PRINCE2 methodology, investigated across organisations to establish breadth. The second, third, and fourth cases were different PMMs—each employed in different organisations and investigated for depth. Case Two is an in-house comprehensive methodology; Case Three, a Waterfall type; and, Case Four is a gate-based methodology alongside an agile approach for IT/IS project delivery. Each case also demonstrated different forms of client and user engagement: internal and external. A general inductive approach is employed for organising and analysing content of qualitative data (Thomas, 2006).

Cases Study Description

Case One—PRINCE2

Case One centred on PRINCE2, commonly referred to as a structured approach to project management. Projects in Controlled Environments was developed and championed by the UK government. In 1989, it was originally intended for the management of IT/IS projects; the methodology was subsequently modified and re-launched in 1996 for use in a wider range of project situations. This modification resulted in a heavy reliance on tailoring when used for IT/IS projects. The effective application of PRINCE2 is strongly associated with finding an appropriate level of tailoring. For example, PRINCE2 is designed to ensure use of a common language for all parties involved in the project. Bringing customer and supplier together requires contracts and contract management. Although the method recognises this, the specifics must be tailored within a contractual framework according to the project and the organisation's context. PRINCE2 offers a number of structured features that can be scaled up or down, depending on the type of the project and level of expertise of the project manager. The main features are: a management structure, a system of plans, a set of control procedures, and a focus on product based delivery. The rationale behind selecting Case One is that there are now more than 300,000 PRINCE2 certified project managers worldwide. Any methodology with such a large population of proponents warrants attention in any study of PMMs. PRINCE2 is used in most government projects as well as a number of private organisations with internal and external clients at different levels of contractual engagement. Every candidate interviewed about PRINCE2 was either formally PRINCE2 certified or had managed and governed projects using PRINCE2.

Case Two—Method A

Method A is a company-wide methodology designed and developed over a number of years, outlining key processes related to fundamental aspects of the business. This in-house structured PMM was originally developed in 1975. The processes were consolidated into the company's initial quality system. Method A is strongly influenced by British Standards and is heavy on quality management and assurance. In 1998, the company intranet was launched with the first two key process area descriptions as: Win Business and Manage Projects. This provided staff with easy access to up-to-date and more detailed information. Over the years, this method has been continuously revised and several key process areas have been added. Most clients and project owners are external to the organisations with varied business structure and project needs. The parent organisation to Method A is a large international force in IT services. This London-based PLC employs 40,000 people in 39 different countries and is a systems integration organisation providing IT and business process solutions to companies worldwide. The organisation carries out project work, develops and supports new products, provides business and outsourcing services, and offers consultancy and advice across diverse markets, including telecoms, financial services, energy and utilities, distribution, and transport and the public sectors. The projects range in size from small to multi-phased programmes and vary in complexity. The company relies heavily on successful delivery of their projects to maintain margins and ensure customer satisfaction.

Case Three—Method B

Method B was used within the Information Systems and Technology (IS&T) division projects for the management and delivery of projects. Method B was initially derived from a product delivery methodology. The method was judged to contain too much external client focus while not paying enough attention to the internal customers and their needs to meet the technical and business requirements of departments within the organisation. The host organisation for Method B is a global information company providing tailored information for professionals in the financial services and corporate markets. The core strengths of the company lie in providing the content, analytics, trading, and messaging capabilities needed by financial professionals. Some 370,000 financial market professionals working in the equities, fixed income, foreign exchange, money, commodities, and energy markets around the world use the company's open information tools. This open technology, based on industry standards, enables customers to search, store, and integrate information that the organisation using Method B provides with content from other sources. Additionally, the tools help the customer to manage risk and organize data. The speed and availability of the disseminated information depend upon the efficiency of the organisation's information systems and technology infrastructure. Most systems development was undertaken to serve internal functions and departments within the organisation. Therefore, in contrast to the organisation operating, the projects' users and clients were mostly placed within the organisation and were therefore internal customers.

Case Four—Method C and Agile

Method C, used company wide, had been established as the standard for project delivery and product management. The following is a brief description of Method C. The process had six steps:

  1. Concept
  2. Specify and Plan
  3. Design and Implement
  4. Test and Trial
  5. Launch Preparation
  6. Closure

Figure 2: Case four, method C—Stage-gate product development methodology.

 

Case four, method C—Stage-gate product development methodology

The organisation that is home to Method C and an agile approach is a global telecommunications company operating in more than 150 countries. In 2004, the organisation was restructured into four separate lines of business, with each being responsible for looking after a particular group of customers. In 2007, the organisation underwent a further restructuring in order to improve the speed of project design and delivery and to improve customer experience. The new organisational structure for the company was justified as a means of responding to the radical changes faced by the whole telecommunications industry. As a result, in 2007, two additional divisions were put in place. One was a unit assembled by recruiting the entire IT design staff for the management of complex IT projects. This unit was also assigned to take control of all management of IT/IS internal and external design and development. The introduction of agile approaches for systems delivery was a focal point throughout this radical transformation.

Table 2: Summary of empirical data.

How each PMM was described by candidates interviewed.
What are the main characteristics of the PMM?
Case One Case Two Case Three Case Four
PRINCE2 Method A
(In-house structured)
Method B
(In-house Gate-Phased)
Method C
(Gate-Phased)
Agile

By far the most commonly used method in India, Hong Kong, Scandinavia, the UK, Australia, Asia, New Zealand, Singapore, Europe 50% Spain and Portugal.

P2 is an exception based project management method and it requires very little management interaction.

P2 is very prescriptive and it provides prescribed steps [sic], methods, procedures and techniques.

P2 has become the standard for most organisations, easily available, easy access, financially viable, and there is large number of trainers.

P2 is a predictive methodology; the decision has been made for you.

It has become popular because the UK government mandates its use; if you need to work with the government, you must know P2.

A strong framework that defines: how a project should be managed, how a project should be positioned within the company, and who is responsible for managing and resourcing it.

The method here is tailored from P2 to fit organisational context and IT/IS development here.

We have clear processes on how to manage projects and how to develop systems. Our team is aware of the difference

Project life cycles are fully understood and are implemented through SAP, and different ones used the RAD method. We have to follow the imposed SAP method, as the customers bring their method with them, it is condition of engagement

Method A, a third generation of quality system. Very strong on quality.

Method B was tailored from the company-wide method to address IT design and development using appropriate terminologies and IT-related processes.

Once again projects that were internal tended to operate in a narrower and more stable context, from the supplier point of view, compared with organisations with external suppliers.

Our technical work and actual development are done by external parties; this is one particular reason that we are Waterfall. We need to be absolutely Waterfall because there is a need for certainty between different parties

Within this company we have a number of methodologies, not many of us are sure of their differences and how and why they differ.

The process we use here is Waterfall type and not agile and it is gate phased. The process has six steps: 1. Concept; 2. Specify; 3. Design and implement; 4. Test and trial; 5. Launch preparation; and, 6. Closure. In product development and management, this is the process that we use.

In [Company C] most of the product development is about software development. The main product we have sold for 20 years ago is phone lines. If we made any changes to call packages or billing systems, all required making changes to our software packages as these are all software driven.

In reality, in our company the last phase is normally neglected, the report does not get written, and learning doesn't get passed on to the subsequent projects

Different parts of the company have their own ways and methods of carrying out work and delivering projects. There are different project handbooks within the company and the intention is to replace these with agile.

Selecting agile is not a direct reason to be more innovative but it is a respond to shortcomings of traditional methods.

The five core practices recognised to fit customer requirements by introducing agile were: Customer involvement, user stories, iterative development, automated testing and continuous integration (Company document)

I am responsible for implementing change and promoting agile throughout our organisation; this is not an easy task. There is no reward for being an expert agile developer and there is no reward for being an agile practitioner, whereas as a PRINCE2 practitioner there are recognised incentives.

Agile is not a methodology, it is more about cultural and behavioural change. Change in mind and heart set.

Agile principles and values can be applied to many situations. Agile is about heart and mind changes.

Findings and Analysis—The Benefits of Using PMMs in Case Organisations

The findings revealed the presence of strong strategic direction governed by the organisations alongside very little involvement from their managers. Despite this unidirectional approach to the implementation of PMMs in the cases examined, there existed a general consensus across the cases that traditional, structured PMMs were beneficial for projects and organisations. However, the types of perceived benefit and contribution were different, depending on the individual, his or her level of involvement with the project, his or her accountability, and the organisation. The responses are organised in the following tables. The comments are analysed and categorised within each case based on their content with reference to benefits. In the analysis, certain benefits appear as patterns across cases, whereas others are more case and business context—specific. This leads to the emergence of sub-categories (types of PMM benefits) and their implications.

Table 3: Case one—Empirical analyses of types of PMM benefits.

  Benefit and support from PMM to Projects and Project Managers – Empirical Data  
Case One - PRINCE2 (Structured PMM)
Findings – Empirical Evidence Analysis (Grouping findings based on meaning and content) Number of Respondents
  • [PRINCE2] helps managers to keep track of the project progress and it provides a control system to check what is working and what is not.
  • [PRINCE2] helps with imposing control and governance on individual sets of actions to avoid failure by continuously checking and balancing.
  • The main value of PRINCE2 is for senior managers to concentrate on things that matter and make decisions based on factors that are of importance.
Better Control and Monitoring on project 3
  • The advantage of using a methodology like [PRINCE2] is that everyone talks the same language; therefore, project managers understand each other.
  • PRINCE2 helps with consistency, speaking the same language, and continuity.
Standardisation and Unified language 2
  • Largely the value is that organisations get to mention [PRINCE2] at the early stage proposal and project bid.
  • Most organisations use PRINCE2 or any methodology as a hygiene factor and as an insurance.
Hygiene factor, enabling and ensuring winning proposal bids and contracts. 2
  • When actually working with the organisation, you find that people are not particularly using it.
  • Not met anyone be happy to use [PRINCE2] methodology, what would they gain from it?
No benefit through limitation or lack of usage (section PMM limitations)


No benefit – values of PMMs are subject to appropriate tailoring
2


4
Total Respondents   13

In Case One the type of support offered by PRINCE2 towards project delivery was perceived to be different according to the practitioners interviewed. It is worth affirming that the practitioners involved in this case were all senior managers with over 20 years of experience in the project management discipline, with great accountability with regards to project result and outcome. Some (3 of 13) upheld PRINCE2 for its control and monitoring traits. The main support from PRINCE2 was seen as helping managers to monitor the progress and development of tasks in projects. Financial management and prioritisation of tasks and activities were considered very helpful to these senior managers. These benefits primarily focused on the execution of projects based on traditional measures of success, time, and cost; a facet important for organisations to monitor the progress of a portfolio of projects. Only two (2 of 13) acknowledged that the values of PRINCE2 are in its support, standardisation, and the establishment of unity of best practices by avoiding reinventing the management wheel. PRINCE2 was praised for its comprehensive, structured approach to developing business cases and securing contractual bids. Two senior consultants emphasised that organisations use PRINCE2 as a way of winning contracts.

The process of proposing bids is significant in government projects; hence, the management of this process is crucial to the success of the bid, “the government usually chooses suppliers through competitive bids and a process of proposal evaluation” (Shenhar & Dvir, 2007, p. 196). The contractual benefits, including security from a legal point of view provide “the hygiene factor,” ensuring parties involved have the required standard levels of skill and knowledge in project delivery. According to Hertzberg (1968), hygiene is de-motivating if not present yet its presence will not enhance motivation. The specific findings align with this generic theoretical perspective. Others (6 of 13) offered a contrasting view describing the effectiveness of PRINCE2 as generally questionable, as at the project level, teams were unhappy with it. Despite its purported affinity to government projects, in some instances, PRINCE2 was not used. The reasoning provided for poor evaluation of the benefits of PRINCE2 was mainly associated with the drawbacks experienced by using the method. Hence, some practitioners rated the benefit of PRINCE2 as low due to lack of commitment from teams and managers and the overall limitations of the PMM.

Table 4: Case two—Empirical analysis types of PMM benefits.

Case Two - Method A (In-house structured)
Findings – Empirical Evidence Analysis (Grouping findings based on content) Number of respondents
  • The main value of Method A for the organisation is better tracking and control. Having well-structured established and understood [PMM] is crucial.
  • [PMM] is vital but nothing magical, it's a common sense way of thinking in a structured way, but it is fundamental.
  • [Method A] in this company is very strong on cost management control and the financial aspects of project delivery.
  • The key element of Method A is defining a strong framework.
Control, Monitoring, Governance 4
  • [Method A] is a third generation system and it is very strong in quality. It helps everyone at this company in different parts to speak the same language. Consistency and not perfection, it is critical to have a common system.
  • Without this method you are sunk. This method is generally based on providing good understanding of the scoped plan that recognises the reality of the available resources and allows for contingencies for money and time.
  • Method A is there to make life easy by defining an agreed on way of doing things. Stages of Method A are there for visibility by stakeholders and communication.
Standardisation and Unified language 3
  • I don't believe methodologies can ever be effective.
  • We are constrained by our quality management system and not enhanced by it.
No benefit was reported due to limitations


No Benefit reported
2


2
Total Respondents   11

In Case Two, similar to Case One, practitioners interviewed were all senior managers. The benefits of Method A were associated with its support to managers to improve tracking and control. Clear guidelines existed to support the project, such as:

“…responsibilities, performance goals, deadline dates, rewards for early delivery, penalties for missed dates, status reporting dates, milestone dates, change management procedures, acceptance test criteria, cancellation conditions, cancellation policies, and closing criteria.” (Company Document)

Method A could be considered a superset methodology of PRINCE2, better addressing issues such as risk, quality, and change where PRINCE2 falls short in explaining how these should be handled. Method A, while improving on PRINCE2, remained acceptable to the organisation's PRINCE2-using clients.

The role of Method A was seen as crucial by one director for the organisation. The strong cost control aspect of Method A was seen as important. This organisation was strict on contractual engagement and, as a result, the management and control of the financial aspects were paramount.

Method A, similar to PRINCE2, was cited as offering financial and task control and improved communication and consistency. These traits were considered important to senior managers at the organisational level. Traits such as accountability, transparency, predictability, and participation are considered the pillars of governance (CGC, 2000; King, 2002; Williams, 2003). Similar to the opinions of PRINCE2, some practitioners were apprehensive of the benefits provided by Method A, warning of the reliance of the PMM on the competence and expertise of the project manager or individual in charge of using the method. Four (4 of 11) senior managers questioned the value of Method A, based on its drawbacks, such as limiting engagement with the client and its heavy formal documentation. These senior managers disliked the lack of autonomy and the top-down enforcement by the organisation and considered Method A to be constraining rather than supportive of project delivery.

Table 5: Case three—Empirical analysis types of PMM benefits.

Case Three - Method B (In-house Gate-Phased)
Findings – Empirical Evidence Analysis (Grouping findings based on content) Number of respondents
  • I have no problem with methodologies; the people that do criticize them usually have not worked within a process environment before. How can I run and manage the progress of 30 projects at the same time and see what has been achieved at the end of each phase without the use of [PMMs]?
  • [Method B] provides a control system to check what is working and what isn't. [PMMs] are all about communication, check points, and is about making sure that the business commitment is still there, making sure that the scope of work is still right and that the work is still the right work and that the solution is still going to meet the objectives.
Better Control and Monitoring. 2
  • The benefit of the methodology is that the customer knows what to expect, if the customer knows how we are doing things and the project manager is specifying his needs from the customer by the next gate, and these questions are asked all the time, then the customer knows what to expect. If the project works in an informal way, then the customers will not know what is required from them. The consistency helps the project managers to do their job faster.
  • Our processes here are very lightweight and it allows lots of discretion but it does keep people in check in terms of going through the right check points, within a project, make sure the communication is happening when it should, and that the decision-making is happening when it should with the right level of information available.
Standardisation and Unified Language 2
  • The processes are very helpful, without the gates you would be lost in the forest. This type of PMM [Method B] and PRINCE2, I like and I am happy with. I like the gates on PRINCE2.
  • I can say the project management processes that we have here are a very useful guide for a new project manager, as it helps them to touch base and it helps the project manager to make sure that the project goes through the right gates.
  • I rely on our [PMM] for identifying risk and ensuring that risks are identified and addressed and managed.
  • [Method B] helps people to overcome the fear of the unknown and deal with uncertainty, it provides a safety factor. Gates help to ensure all the principles of PM are covered and that the project is structured appropriately
Help Guiding and Directing managers with Uncertainty and Unknown. 4
  • Do I need to get PRINCE2 certification? Will it enhance my capability to do my job? And the answer is NO. I can go and buy the book and apply what is in the book. If you are a relatively new, inexperienced project manager then perhaps yes, this would provide you with a set structure to do your job better.
  • I am too old in the tooth to start learning about new ways of managing projects.
  • I don't use a methodology at all. I rely on common sense.

No Benefit due to lack of usage or poor implementation


No benefit was reported at all

2




1
  Total Respondents   11

Case Three, comprised Method B and was somewhat different than PRINCE2 and Method A in that the practitioners interviewed had a varied level of experience, competence, and accountability. Whereas in the first two cases all practitioners were senior managers, and had more than 15 years experience, this case included novice project managers of less than 8 years. Resembling PRINCE2 and Method A, findings associated the perceived benefits of Method B with its imposition of control, monitoring, and standardisation. One programme manager correlated the benefit of Method B with her need for maintaining control and ensuring project progress and accountability. Another senior manager in the organisation argued that the benefit of Method B lies in its communication of expectations with the customer. Method B, a plan-driven, gate-phased methodology ensured strong monitoring and control of project progress and was seen as important by senior managers for cost and expenditure management purposes. The role of Method B was most relevant as a communication channel between the project and the customer. Up to this point, the benefits considered important to senior managers were similar to PRINCE2 and Method A — control and standardisation. However, a new perspective appeared in this case and was presented by less senior managers. This group of practitioners valued Method B for its guidance and general support in decision-making. Method B helped junior managers to deal with risk and uncertainty. This group of managers with less experience found Method B reassuring for tracking and guidance and compensating for the absence of tacit knowledge.

Similar to PRINCE2 and Method A, a number of practitioners viewed the benefit of Method B to be low or limited. Some senior managers believed that their experience is far more crucial than relying upon the PMM or guidelines recommended by the organisation. Reliance on tacit knowledge was the thrust of their argument. One manager explained that he does not follow any methodology and that to him reliance on his common sense is the best way forward. This sort of attitude brings into question the point of having PMMs. The reliance on tacit knowledge was not consequence free as it instigated an informal tailoring at the project level. This tailoring was purely based upon the project manager's experience, was not formally described, and remained highly subjective.

Table 6: Case four—Empirical analysis types of PMM benefits.

Case Four - Method C (Gate-Phased) & Agile
Findings – Empirical Evidence Analysis (Grouping findings based on content) Number of respondents
  • I would be the one to say that Method C is a good document. For a new project manager to touch base it is good; it can help to make sure that the projects goes through the right gate.
  • It helps other project managers inherit the project work easier.
  • I strongly believe in project management processes and methodologie as it helps me with where to go and what to do next.
Guidance, direction, helps with uncertainty and unknown. 3
  • Waterfall is extremely engineering efficient, fitting requirements perfectly where you design once, develop once, test once, and there is no need to write in a perfect environment where there is no change.
  • Traditional and Waterfall method is extremely efficient and extremely reliable and extremely unable to adapt. From an engineering viewpoint it was brilliant but not from the business perspective.
Benefits are contingent upon project type and structure of projects. 2
  • Agile emphasises the need for prioritisation, and each user story and requirement was one function. This means that we are able to manage sizes of those and keep them small. Having defined an explicit priority, things to do, created a great visibility from the customer's point of view.
  • Agile supports defining and prioritising requirements and user stories and actually taking three weeks to do the work instead of six months.
  • Agile is fine for software development, as opposed to product development where it doesn't work as well.
  • Don't think the more traditional and structured methods are the answer, they just don't work. Agile methods can work better on systems development projects and not so much with project management issues. Scaling up is not an issue. One hold-back is that some suppliers are not as advanced in their way of thinking,
  • Requirement capturing issues relate to not specifying the systems development and business requirement. The systems developers require further information that's why with agile showing someone what it could look like is more effective.
  • Where the requirements are in the form of user stories, we prioritise our user stories and the priority number one will be called the minimum market feature sets.
  • What I would say is that agile is structured; it is not chaotic and is more management-centred than traditional methods. The management side is lighter and less of a weight. The transparency is greater with agile.
No benefits from PMMs -company is moving from traditional and structured PMMs to agile approaches. Most interviewees criticised PMMs and emphasised the benefits of agile.
Benefit of the Agile Approach
  • Requirement definition, prioritisation and management.
  • Mainly beneficial for software development.
  • The transparency in project activities and progress.
  • Speed of delivery
8
  Total Respondents   13

Case Four showed the least number of endorsements for Method C, a gate-phased PMM. The organisation in which the case was explored was on a mission to promote agile approaches over traditional project delivery. It was therefore unsurprising that few benefits of Method C were acknowledged. In evaluating the benefits of Method C, it was evident that the organisational benefit of Method C was no longer relevant. Strategically, the organisation was moving towards agile approaches as the organisation-wide standard with the aim of having both project and product delivery accomplished using agile approaches. As a probable consequence of this initiative, traditional and plan-driven PMMs were heavily criticised. At the beginning of the data gathering and interviews it was requested by the head of methods and tools that the researcher should avoid where possible soliciting comments about the benefits of Method C, because this could conflict with the strategic direction towards the promotion of agile approaches for project delivery. However, despite these intentions as explained in Chapter 4, the promotion of agile approaches was not fully welcomed and supported by product managers who considered gates and controlled phases as crucial. As a result of negotiations, some aspects of agile principles had been included in Method C. In general, Method C was seen as largely unsuitable for IT/IS project management and agile approaches were considered inappropriate for product development. This exhibits a clear example of mismatch of PMM and project type. It adds to the question as to whether project managers wished to or were aware of how to tailor the PMM to the project type and context.

A small level of endorsement for Method C came from outside of the IT/IS department, where agile approaches had not yet been adopted. The end result of discussing Method C and agile was that some (3 of 13) agreed that Method C supports new project managers by guiding them through the product life cycle. IT/IS managers were exponents of agile methodologies and commented on their benefits as supportive of user requirement-gathering through encouraging customer involvement. Through this evaluation IT/IS managers tended to first highlight the issues and shortfalls they experienced with Method C and then explained how agile approaches such as XP and Scrum helped overcome these limitations. For example, one product manager stated:

“One of the biggest problems that we have is that we do not involve our customers enough in our new product launches. It often begins with a separate requirement from marketing and is not tested with the customer.” (Product Manager)

The capability of Method C to engage the customer and support requirements articulation and management was a commonly mentioned drawback of PMMs. The findings above reveal patterns across all four cases. These are classified into sub-categories labelled with the type of benefit reflected by the responses (Figure 3). The findings overall indicate that the perceived benefits and contributions of PMMs fall into the following sub-categories or patterns:

  1. PMMs support Control and Monitoring
  2. PMMs promote Standardisation and Communication
  3. PMMs guide and support managers
  4. PMMs do NOT provide any benefits due to their limitations

Figure 3: Abstraction process – Emergence of sub-categories from benefits of PMMs.

 

Abstraction process – Emergence of sub-categories from benefits of PMMs

A particular benefit appeared in Case One, where PRINCE2's value was seen in securing contracts and wining proposals. Table 7 shows the level of response for each perceived benefit.

Table 7: Response levels to different types of support by PMMs.

Type of PMM Support Case One PRINCE2 Case Two Method A Case Three Method B Case Four Method C Total Overall%
Control and Monitoring 3 4 2 9 18.8
Standardisation and Unified language. 2 3 2 7 14.5
Hygiene factor, enabling and ensuring winning proposal bids and contracts 2       2 4.2%
Guiding and Directing managers with Uncertainty and Unknown     4 3 7 14.6%
No benefit is associated with the usage of PMMs 6 4 3 10 23 47.9%
Total respondents per Case 13 11 11 13 48 100%

Table 7 outlines the results of this analysis. In comparing the results across the four cases, significantly, 47.9% of all practitioners rated the benefits of PMMs as low and considered PMMs as non-beneficial to themselves and their projects. This is alarming, because emphasis is placed upon project owners and organisations to use PMMs while playing down the financial investment in their development and promotion.

Discussion and Implications

From the findings, the generalisation can be drawn that the perception of PMM benefits to some extent relates to the levels of experience, authority, accountability, and overall responsibility of the individual. As a common occurrence in all four cases, the benefit of the PMM was subjected to personal perspectives, needs, and the level of experience.

PMM benefits and the levels of experience and authority

 

Figure 4: PMM benefits and the levels of experience and authority.

The above diagram, Figure 4 is a graphical illustration of the range of perspectives from different levels of experience and accountability and their corresponding perceived degree of benefit of PMMs. The ‘U' shaped diagram displays the levels of experience and accountability on the x-axis and the benefits of PMMs on the y-axis. Points A, B, and C represent three primary perspectives emerging from empirical evidence. Points A and C are the representations of perspectives from which the value and benefits of PMMs are considered to be relatively high.

Point A represents the view that PMM benefits lie in their support to project managers as a guidance tool. Here the value of PMMs relates to assistance with decision-making under an uncertain project outlook and hence compensating for any absence of tacit knowledge. Moreover, from the relatively inexperienced project manager's point of view, the PMM represents a fallback strategy in case of problematic project delivery. Should the project being managed encounter difficulty, then the role of the project manager can be defended by demonstrating that the PMM has been pedantically followed and the structure of the PMM could therefore account for any discrepancy between expectations and the project outcome.

Point C represents the perspective that regards the benefit of PMMs also as high, but for different reasons. Point C upholds PMMs for their promotion of standardisation and governance. From the organisational perspective the desire is for strategic directives to be enforced and monitored by the use of uniformity of processes and procedures. The prescription of a PMM facilitates the comparison of performance across multiple portfolios of projects due to the standardisation of language and progress reporting.

The perspectives represented in Points A and C are to a large extent aligned with those, which PMM owners and promoters claim to be their benefits. Furthermore, these perspectives to a large extent map to the literature's definitions of the benefits of PMMs. For example, definitions from OGC (2009), IIL, Charvat (2003), and Gardiner (2005) describe the benefits of PMMs as control and standardisation, with an emphasis on project efficiency measures as demonstrated in Table 8.

Table 8: Benefits of PRINCE2 - Control, monitoring, and standardisation.

PRINCE2 is a method for managing projects within a clearly defined framework and it describes a procedure to coordinate people and activities in a project, with guidelines on how to design and supervise the project.
Benefits: efficient control of resources, close monitoring of the project in an organised and controlled manner, provides a common language for all participants in the project, describing the management roles and responsibilities. (OGC, 2009)

Definition (IIL)2 A project management methodology addresses the principles and procedures for performing project management, where project management is a critical value-adding process that improves the probability of project success.
Benefit: Reduced risk of project failure, increased efficiency and productivity, improved quality, and improved communication

Definition (Charvat): A project management methodology is a guideline that may include the list of things to do, a specific approach, templates, forms and even checklists used over the project life cycle; a set of guidelines and principles that can be tailored and applied in a specific situation.
Benefits: Better process, flexibility, quality focus, management of complexity, standard approach, consistency, better planning, and estimation. (Charvat, 2003)

Definition (Gardiner): A project management methodology is a structured guide or framework designed to help organisations manage large and small projects in a controlled and efficient manner.
Benefit: Reduces communication and integration problem throughout the project life cycle. (Gardiner, 2005)

PMMs are often viewed as a solution to problematic project delivery, but the findings demonstrate that they are infrequently selected on this basis. In this study, the need for PMMs varied between case organisations. For example, in PRINCE2 a standardised approach and better project control mechanisms are advocated. Method C demands a better way of involving users and customers. For these cases, the goal was increased discipline into the processes. For the organisations in which the cases are embedded and reflected upon as in PRINCE2, the selection and implementation of the PMMs did not generally align with the solutions they purport to provide and therefore largely did not meet the expectations of improved project delivery. Yet, the findings illustrate that a more investigative approach rather than prescriptive approach can enhance the fit of the methodology to the type of project. However, the analysis also shows that the way the PMMs were tailored did not align with the intentions of achieving the solution purported through tailoring to context and clients, or even internal organisational factors at times. Career life cycle factors and personal dispositions played a role in the shape of tailoring (Wells & Smyth, 2011).

Conclusion

Comparison of the findings and literature found that, despite the original intent of PMMs focusing on control management and standardisation, a large proportion of practitioners (47.9%) disagreed that this fulfilled their expectations for effective project management. As analysed, the motives behind findings range from intent to trying to make the PMMs fit project type, fit context (organisation and client), fit personal factors, or trying to discard the PMM as much as possible to address one of the preceding factors. How this works out in practice was found to vary. Findings revealed what manifests as the dip in the centre of Figure 4 above, point B. Where the perceived benefits and advantages of using PMMs dramatically fall to a minimum corresponds to the middle ground of the range of perspectives. This drop was largely due to the drawbacks and limitations practitioners experienced in managing IT/IS projects. The dip in the graph at point B was also associated with the attitudes of practitioners. This is where the usefulness of PMMs was directly undermined by the mobilisation of tacit knowledge, intuitively steering project management decisions, and overriding the template-based directives from the PMM. Hence, this is where informal tailoring mostly takes place. Furthermore, this knowledge cannot be captured by any methodology because it is highly people and context sensitive. While point B is influenced by the group and is context-driven, the A and C groups are rule-driven

Ideally, the methodology usefulness should be uniformly high across the groups. The implication of the U-shaped curve is that the majority of project managers gain sub-optimal advantage from the use of any PMM, which in turn implies that PMM structure typically fails to accommodate the requirements of experienced practitioners.

It was concluded that there exists a gap between the perceived contribution of PMMs at the strategic and organisational levels compared with the perceived benefits at the project and operational levels. The findings suggested that the contributions of PMMs are first of a different type and, second, shaped by the role, experience, and accountability of the individuals questioned. The benefits of PMMs are most pronounced in assisting organisations and senior managers with their endeavours in control and monitoring, standardisation and unifying practices. Hence, the role of the PMM focuses on management and standardisation as opposed to service and product development.

The findings suggest that PMMs are found to be useful to some extent where they replace and compensate for the absence of tacit knowledge in a project, helping managers with less experience and knowledge of project management. It is concluded that a misalignment exists between the perceived value of PMMs across different groups and between the project and the strategic/organisational levels. Most project managers perceived the prime purpose of PMM to be management, control, and compliance rather than support and guidance. The investigation on this aspect reveals that 47.9% of project managers viewed PMMs as non-beneficial to their projects and claimed that using PMMs hinders their project delivery. The ‘U' distribution in Figure 4 shows that the benefits of PMMs are viewed differently by practitioners and is based on their experience, authority level, and accountability.

This paper contributes a discussion and analysis of a qualitative data through an inductive approach and the interpretivism paradigm. Although the subject of PMMs may have been addressed from various angles prior to this study, the application of this research strategy to this subject is unique. This study looks into a subject that has its foundations rooted in two different disciplines, information systems and project management. This research hence contributes to both disciplines by showing how best practices and project management approaches for management of IT/IS projects face difficulties, in part, brought about by the way in which organisations select and implement PMMs.

This research contributes to knowledge by showing a gap between the intended benefits of the PMM by senior managers at the organisation level and the actual benefits and support offered at the project level. The demise of the PMM's benefits and support at the project and operational level leads to a chasm between the intended strategic directions of the PMM and its actual contribution to projects, managers, and their teams. Hence, the purported benefits are often not realised or, furthermore, can have unintended consequences at the project level and adversely affect project success.

References

Al-Zoabi, Z. (2008). Introducing discipline to XP: Applying PRINCE2 on XP projects. The Arab International University. Damascus, Syria.

Boehm, B., & Turner, R. (2004). Balancing agility and discipline a guide for the perplexed. Addison-Wesley.

CGC (2000). Committee on Corporate Governance, The combined code: Principles of good governance and code of best practice, derived from the Committee's Final Report and the Cadbury and Greenbury Reports, United Kingdom.

Charvat, J. (2003). Project management methodologies: Selecting, implementing and supporting methodologies and processes for projects. Hoboken, NJ: John Wiley & Sons.

Cooper, R.G, (2007). Managing technology development projects. IEEE Engineering Management Review, 35(1), First Quarter.

Cross, M. (2005). Public sector IT failures. Prospect, October.

Davies, A., & Hobday, M. (2005). The business of projects, managing innovation in complex products and systems. Cambridge University Press.

Gunnarsson, S., Linde, A., & Loid, D. (2000). Is standardisation applicable to project managers of multi-project companies? In paradoxes of project collaboration in the global economy: Interdependences, complexity and ambiguity. IRNOP IV. Sydney: University of Technology, 136–146.

Herzberg, F. (1968). One more time: How do you motivate employees? Harvard Business Review 46(1), 53–62.

Improving Government Risk Handling, Personal Minute from the Prime Minister to Deputy Prime Minister, 29 March 2009.

Jayaratna, N. (1994). Understanding and evaluating methodologies. London: McGraw-Hill.

King (2002). King Committee on Corporate Governance, ‘King Report 2002.' Institute of Directors, South Africa.

McManus, J., & Wood-Harper, T. (2008). The IT trade off. Project, 20(4).

Morris, P. W., & Pinto J. K. (2004). The Wiley guide to managing projects. Hoboken, NJ: John Wiley & Sons, Inc.

Morris, P.W. (2009). Research and the future of project management. IMPA Award Article.

Oates, J. (2009). Failed probation system ‘master-class in sloppy management. 12 March 2009. The Register: www.theregister.co.uk/2009/03/12/nao_probation_report/

OGC (2009). Managing successful projects with PRINCE2. OGC.

Sauer, C., & Reich, B.H. (2009). Rethinking IT project management: Evidence of a new mindset and its implications. International Journal of Project Management 27, 182–193.

Shenhar, A.J., & Dvir, D. (2007). Reinventing project management: The diamond approach to successful growth and innovation. Harvard Business School Press.

Smyth, H. J., & Morris, P. W. G. (2007). An epistemological evaluation of research into projects and their management: Methodological issues. International Journal of Project Management, 25(4), 423–436.

Standish Group (2010). Chaos Summary for 2010, tech. report, Standish Group International.

Thamhain, H. J. (1994). A manager's guide to effective concurrent project management. PM Network, 8(11), 6–10.

Thomas, R. (2006). A general inductive approach for analysing qualitative evaluation data. American Journal of Evaluation 27, 237. SAGE. American Evaluation Association.

Wells, H., & Smyth, H. (2011). A Service-Dominant Logic: What Service? An Evaluation of Project Management Methodologies and Project Management Attitudes in IT/IS Project Business. EURAM – Tallinn.

White, D., & Fortune, J. (2002). Current practice in project management: An empirical study. International Journal of Project Management, 20, 1–11.

William, T.M. (2003). Corporate governance: A guide for fund managers and corporations, Provenance Investment & Financial Services Association Limited, Westpac Banking Corporation, Australia.

Wysocki, R. K. (2007). Effective software project management. Indianapolis, IN: Wiley Publishing Inc., 38–52.

Yardley, D. (2002). Successful IT project delivery: Learning the lessons of project failure. London: Addison Wesley.

Yin, R.K. (2003). Case study research: Design and methods, 3rd ed. Thousand Oaks, CA: Sage.

________________

1 PRINCE2: PRoject IN Controlled Environment is a structured method for effective project management. The method was first established in 1989 by CCTA (the Central Computer and Telecommunications Agency).

2 IIL: International Institute of Learning: http://www.iil.com

©2012 Project Management Institute

Advertisement

Advertisement

Related Content

  • Pulse of the Profession

    Why Social Impact Matters

    Every project has an impact—and increasingly it’s up to project leaders to make it a positive one. The same strategic mindset behind the drive for bottom-line results must also be applied to ensure…

  • Colorful Projects

    By Keil, Andrea | Doppelfeld, Dirk | Friedrich, Ralf Fast-evolving technology and innovations in collaboration systems make it easier to compose teams with members who are chosen by their expertise but might be from different locations. These project…

  • Project Management Journal

    Transition and Temporalities member content locked

    By Whyte, Jennifer | Nussbaum, Tamara As projects end and operations begin, we argue that transition involves boundary-spanning work to ensure continuity across changing forms of organizing.

  • Project Management Journal

    Facilitating Efficiency and Flexibility Ambidexterity in ProjectBased Organizations member content locked

    By Sun, Xiuxia | Zhu, Fangwei | Sun, Mouxuan | Müller, Ralf | Yu, Miao Through an exploratory multiple-case study in the context of project-based organizations in China, this study aims to identify the antecedents that facilitate three prevalent types of ambidexterity,…

  • Project Management Journal

    Re-Creating Organizational Routines to Transition Through the Project Life Cycle member content locked

    By Addyman, Simon | Pryke, Stephen | Davies, Andrew This article provides new insights into the project life cycle by proposing an alternative image to the predefined time boundary between life cycle stages.

Advertisement