Introduction
The author's background is software development and project management and in recent years an increasing number of projects have involved development and implementation of eBusiness applications. These usually have critical timelines and often involve radical development methods. One question often raised is: do traditional project management methods apply to these projects or are they a hindrance? While much has been written about eBusiness in general, there has been little or no research into the project management of these projects, hence the perceived need for this study.
The term eBusiness is used to describe business carried out over the Internet, including Business to Business (B2B), Business to Consumer (B2C), Consumer to Consumer (C2C) and Business to Employee (B2E). The study defined eBusiness projects as the development and/or implementation of eBusiness systems. It also adopted the Standish (2001) definition of project resolution as it provides industry benchmark figures for comparison:
• Successful: completed on-time, within budget and with all required features & functionality
• Challenged: completed and operational but late, over-budget or with fewer features & functions than specified
• Failed: cancelled before completion or never implemented
In order to address the research question, eBusiness project managers would be surveyed by questionnaire.
Research Methodology
The 44 processes defined by A Guide to the Project Management Body of Knowledge (PMBOK® Guide) (PMI, 2004) were selected to form the basis for the questionnaire, but there were six other potential success criteria that did not map onto these processes. PRINCE2 (OGC, 2002) business case, configuration management, issue management and management by exception provided definitions for four, while top management support and end user involvement were covered by direct questions.
The questionnaire asked subjects to state how critical to the success of an eBusiness project they believed each of the 50 processes to be and whether or not they had used it on their project. They were then asked to state any project management methodology and software development life cycle (SDLC) they had followed. They were asked to define the size of the project. They were asked to state whether the project had been completed on time, within budget and with all the required features and functionality. Finally they were asked for their age, gender and length of project management experience.
The target population were project managers who had completed an eBusiness project. Both the Project Management Institute (PMI®) and the British Computer Society (BCS) Project Management Specialist Group (PROMS-G) agreed to place details of the study on their web sites. In addition the researcher's associates and project managers working for his major client were invited to participate by email. Finally members of the PMI eBusiness Special Interest Group (EBSIG) were emailed with requests to participate.
Results
Twenty-six survey forms were returned, representing a 20% response rate from the 130 email requests (no returns were received from either web site). The project managers’ ages ranged from 27 to 62 (mean 45) years, with experience ranging from 2 to 36 (mean 14) years and 81% were male, which probably reflects industry standards.
Project size was measured in duration (months), work effort (people/days) and the maximum number of people on the project at any one time. Removal of two programmes of multiple projects produced the results shown in Exhibit 1.
Exhibit 1: Project Size
Of the 26 projects, 23 were completed and fully implemented, two were incomplete and one had failed. After discounting the incomplete projects, there were 8 successful (33%), 15 challenged (63%) and 1 failed project (4%). Standish (2004) shows a success rate of 29% with 53% challenged and 18% failed.
Analysis of the project success criteria for the 15 challenged projects (successful projects implicitly satisfy all three criteria and failed projects satisfy none) showed the most common failing was not being completed on time (60%), followed by exceeding the budget (47%) and failure to implement all features and functionality (27%). Standish (2004) found 82% late, 43% over budget and 48% failing to implement all required features and functionality.
In terms of the project management methodology used on the projects: 45% of respondents were following an in-house methodology, followed by 31% using PRINCE2, 12% using PMBOK® Guide and 4% (1 project) using SSADM. Exhibit 2 shows the cross-tabulation between the methodology used and the project resolution. The use of in-house methodology seems to give the best rate of project success.
Exhibit 2: Resolution by Project Management Methodology
In terms of the SDLC: 45% of projects used the classic waterfall life-cycle, followed by 19% using Rapid Application Development (RAD), 12% using a combination of waterfall and RAD, 8% using Evolutionary and one project using each of the other options (including none). Exhibit 3 shows the cross-tabulation between SDLC and the project resolution. The use of the classic waterfall life-cycle seems to give the best rate of project success.
Exhibit 3: Resolution by Software Development Life Cycle
Criticality of Processes
The central research question was to establish which processes were considered critical to the success of an eBusiness project. Exhibit 4 illustrates the criticality means for each of the 50 processes, which range between 4.15 and 4.81, while there was a wider variation in the percentage of projects on which the processes were used (Exhibit 5) which ranged between 39% and 100%. The succeeding paragraphs define the main highlights by knowledge area.
Exhibit 4: Criticality Means of the 50 Processes
Exhibit 5: Percentage of projects on which each process was used
Integration Management (processes 1 to 7): have some of the highest criticality means and, apart from (7) project/stage closure, were used on between 85% and 100% of projects, reflecting the strategic nature of this group. The most critical were: (5) monitoring and controlling project work; (2) preliminary scope statement; (3) project management plan; and (6) integrated change control.
Scope Management (processes 8 to 12): apart from (8) scope planning, these processes were used on between 81% and 89% of projects. Scope planning had a lower criticality mean than the other processes. The most critical process in this group was (12) scope change control.
Time Management (processes 13 to 18): these processes were used on between 81% and 100% of projects with correspondingly high criticality means except for (13) activity definition, which was rated somewhat lower. The most significant processes in this group were (15) activity resource estimating and (18) schedule change control.
Cost Management (processes 19 to 21): cost estimation (19) was used on 85% of projects, (21) cost control on 73% and (20) cost budgeting on 69% which was also reflected in the criticality means. There were no significantly critical processes in this group.
Quality Management (processes 22 to 24): these processes were used on between 65% and 85% of projects with low criticality means.
Human Resource Management (processes 25 to 28): developing the project team (27) was only used on 56% of projects, which indicated that some project managers do not see it as part of their role. The other processes were used on between 84% and 92% of projects and were considered more critical. The most significant process in this group was (26) acquiring the project team.
Communications Management (processes 29 to 32): these processes were used on between 77% and 96% of projects with criticality means in the mid-range.
Risk Management (processes 33 to 38): these processes appear to be one of the least used groups, with usage ranging from 50% to 69% and the lowest set of criticality means.
Procurement Management (processes 39 to 44): this section was only completed by respondents whose projects involved procurement of goods or services. The processes were used on between 39% and 46% of projects (except for plan contracting which was used on 58%) and the criticality means were generally low.
Other Project Management (processes 45 to 50): these were used on between 62% and 81% of projects. Of these (45) top management support was rated the most critical, followed by (46) full user involvement.
The results of the responses to the 50 processes ranked by mean, percentage used and standard deviation are listed in Appendix A. While it could be considered that they are all critical, for the purposes of this study, the 10 most critical processes, as identified in the preceding paragraphs, are:
- Monitor and controlling project work
- Scope change control
- Top management support
- Acquiring the project team
- Activity resource estimating
- Integrated change control
- Full user involvement
- Preliminary project scope statement
- Schedule control
- Project management plan
Of the ten least critical processes: five were risk management processes; two were quality processes; and the others were developing the project team, team empowerment and issue management.
Next the processes were analysed to determine if their use had a direct relationship to the success of a project. The 26 processes that were used on less than 80% of the projects (where comparison was meaningful) were analysed. Four of these showed a significant positive correlation and 12 showed some positive correlation. For the purposes of this study, it was concluded that the seven processes with a strong correlation (p < 0.1) could be beneficial to a project:
- Project closure
- Full user involvement*
- Issue management
- Communications planning
- Team empowerment
- Quality Control
- Top management support*
* also in the Top-10 critical list.
Examination of Hypotheses
In addition to the research question there were three hypotheses: project managers would use the processes they believed to be critical to the success of a project; there would be a positive relationship between the level of experience of the project manager and the success of the project; and that there would be an inverse relationship between the size of the project and the likelihood of success.
Use of Critical Processes: There was a strong correlation between the use of a process and how critically it was rated by the project manager, with all processes showing a positive difference in mean criticality supporting the hypothesis.
Project Manager's Experience: There was no significant correlation between a project manager's experience and the success of a project, so this hypothesis was not supported by the data.
Project Size: A cross tabulations of project resolution by duration, work effort and the maximum number of people on a project (Exhibit 6) showed no statistical significance by work effort or number of people. However there was a 50% success rate for projects of less than six months duration, a 40% success rate for projects of six to nine months duration and no successful projects over nine months duration which would appear to support the hypothesis. There is an anomaly in the fact that large work effort and people projects seem to be more successful than medium sized projects.
Exhibit 6: Percentage of successful projects by size criteria
Conclusions and Recommendations
Project Resolution
The success figure of 33% is only marginally higher than Standish (2004), but there is a more significant difference between the percentages of challenged and in particular failed projects in this study. This could be accurate if eBusiness projects suffer a lower rate of failure than general projects, it could be due to the subjects (experienced project managers), or reluctance to give details of a failed project. It is thought most likely that a combination of all three reasons could be responsible.
Success Criteria
While the budget results appear to be broadly comparable, the projects in this survey had a better record of being on time and functionally complete. Earl and Khan (2001) found that eBusiness project functionality evolved through the design stage and was only frozen in the build stage. MacCormack (2001) also found an evolving approach to design in successful Internet projects. In the researcher's own experience functionality tends to evolve during the project, with trade-offs often being made against time. This results in a greater likelihood of the project containing all agreed features and functionality and was the likely cause of the higher success rate for eBusiness projects.
Project Methodologies
White & Fortune (2002) found 65% of their subjects using an in-house methodology and 18% using PRINCE2 so clearly there is a more significant use of PRINCE2 and PMBOK® Guide in this survey. However, these methodologies are not intended to be used ‘out of the box’ and it could well be that many of the in-house methodologies are based on one of these methodologies. In-house methodology, developed by the business, is also more likely to be attuned to the needs of and be supported by the organisation. Whatever the reason, in-house methodologies were more frequently used and seem to have been more effective than the generic PRINCE2 or PMBOK® Guide methodologies.
Against popular perception, 42% of the projects in this study using waterfall alone were successful; against 33% of the projects using a combination of waterfall and RAD; and only 20% of the projects using RAD alone. There were no successful projects using the other life-cycles. It would appear that the classic waterfall SDLC is still the most widely used and seemingly effective life-cycle.
Criticality of Processes
While the response to the survey indicated that all 50 of the processes were considered critical, the ten most critical processes were used on between 77% and 100% of the projects. Eight of these processes are defined in the PMBOK® Guide (PMI, 2004) while two (top management support and full user involvement) were in the additional processes added by this research.
While no existing research into the criticality of project management processes was identified during this study, there were a number of studies that published factors which contributed to project success such as Standish (2001), White and Fortune (2002) and Sauer and Cuthbertson (2003). Six of the top 10 critical processes identified by this study had a good fit with more than one of their success contributors. On the other hand the remaining four processes do not seem to be identified by any of these three sources and these may therefore be more relevant to eBusiness projects than to traditional IT projects:
• Monitoring and controlling project work was rated the most critical process in this survey.
• Acquiring the project team reflects the critical need for getting the right people, with the right skills, when they are needed on an eBusiness project.
• Control of the schedule would seem to tie in closely with monitoring and controlling project work and again it is rated as critical on an eBusiness project.
• The project management plan establishes a management baseline against which a project can be kept on track. This is rated as a critical process on an eBusiness project, which often involves decisions being made ‘on the fly’, with a varying and dispersed project team.
Impact on Success
The use of the top seven beneficial processes seemed to have a good correlation to the success of a project and, as previously noted, full user involvement and top management support are already in the top 10 critical processes. None of the remaining five processes appear to have been the subject of any previous research into their potential effect on the outcome of a project. However, on the basis of this study, these further five processes should be added to the list of critical processes:
- Project/stage closure
- Issue management
- Communications planning
- Team empowerment
- Quality control
Criticality of Use
The first hypothesis was that the criticality of a process would have a direct relationship to its use and this was fully supported by the data. This would seem logical as it could be expected that project managers would use the processes they believed were the most critical to the success of their project.
Project Manager's Experience
The second hypothesis, that the level of experience of the project manager would have a direct relationship to the success of the project, was not supported by the data. However, none of the sample group had less than two year's experience and it could be postulated that there would be a lower rate of success among new and inexperienced project managers, who may be more likely to make mistakes.
Project Size
The third hypothesis, that the size of a project would have an inverse relationship to the success of the project, was supported substantially for the project duration and to a lesser extent for the work effort and number of people on the project. Keeping a project small (duration under six months and fewer than six people on the project) giving a 50% success rate. Medium duration project (six to nine months) giving a 40% success rate, but projects exceeding nine months duration were all challenged. The anomaly in these results could be explained by the fact that larger projects tend to have sufficient (often full-time) resources allocated to them while medium projects have to ‘make do’ with fewer (part-time) resources who are likely to have more pressure from their ‘day job’.
Summary
In summary this study appears to show that the traditional project management processes are still appropriate for an eBusiness project. Further that while most of the 50 processes covered by the survey are considered important, some can potentially improve the chances of success of a project. The use of an in-house project management methodology coupled with the classic waterfall SDLC appear to be the most popular and most successful combination and the old wisdom about keeping a project's duration as short as possible is especially true for an eBusiness project.
Recommendations
Based on the results of this study, it would seem that there is a strong case to be made, for any business carrying out eBusiness projects to define and implement an in-house project management methodology. The study has confirmed the criticality of the 44 processes defined by the PMBOK® Guide. The study also identified a good case for the use of the classic waterfall software development life cycle. While not covered by the PMBOK® Guide processes, end user involvement, issue management, team empowerment and top management support all appear to improve a project's chance of success. Finally project duration was clearly the most significant size factor in this survey and eBusiness projects exceed nine months duration at their peril. These would seem to represent the factors that can improve the chances of success for an eBusiness project.