An exploratory analysis of the relationship between stakeholder management and information technology project success
Feng-Ching Lu, BBusSci (UCT)
Post-graduate honors degree candidate
Department of Information Systems
University of Cape Town, South Africa
Lauren Mandy, BBusSci (UCT)
Post-graduate honors degree candidate
Department of Information Systems
University of Cape Town, South Africa
Derek C. Smith, PMP
Department of Information Systems
University of Cape Town, South Africa
11-14 July 2004 – London, UK
According to the CHAOS Chronicles from the Standish Group (1994, 1999), only one-third of information technology (IT) projects are successful. This research literature argues that it is the behavioral, organizational, and management issues—stakeholder management, in particular—that require urgent attention to improve this situation.
These reports examines whether an IT project management approach that focuses on the needs of stakeholders leads to an improved degree and rate of IT project success. We develop five hypotheses to test this assertion. While the quantitative data was useful for statistical testing, the qualitative data provided substantially more richness, offering insight into the perceptions, attitudes, and behaviors of project managers (PMs).
We determined that the level of effort expended on troubled projects was substantially higher than on successful projects. When comparing the average level of stakeholder involvement in successful and challenged projects, we found that the stakeholders were significantly more involved in the successful projects. The qualitative data suggests that the nature of this involvement is particularly important.
As the extent of stakeholder involvement increases, there is, however, a corresponding increase in associated problems. Stakeholders who were skilled and experienced knew what they wanted and had decided what their requirements were prior to attending project meetings. This made for particularly valuable interaction. However, PM who continually attempted to accommodate and involve stakeholders who lacked IT knowledge led to valuable development time being misspent, resulting in an increase in time overruns.
A good deal of effort was put into managing stakeholders. Project managers (PMs) realized that managing stakeholders was important to project success. Therefore, they actively tried to manage them. And when projects did not appear to be progressing according to plan, PMs shaping expectations became essential. Support from senior management emerged as a distinguishing factor between successful and challenged projects.
Finally, PMs need to assess the level of IT competency among stakeholders and adapt their management approach accordingly. Where stakeholders have little understanding of technical issues, more emphasis should be placed on ensuring that stakeholders understand correctly how software will function.
The IT project environment is characterized by a decreasing—but still high—incidence of failure (Standish Group International, 1994; Wateridge, 1995).
The CHAOS Chronicles classify projects as successful (completed on-time and on-budget, with all of the features and functions, as initially specified), challenged (completed and operational, but over-budget and over the time estimate with fewer features and functions than originally specified), or impaired (cancelled at some point during the development cycle) (Standish Group International, 1994, p.1).
Exhibit 1 summarizes the findings of the CHAOS Chronicles. IT project success rates remain extremely low and nearly half of all IT projects were not completed successfully according to the original project plans.
Exhibit 1: CHAOS Chronicles summaries (Standish Group International, 2003)
During these 8 years, the sample of projects measured has increased dramatically, thus the 34% success of a large sample of projects in 2002 far exceeds the 16% success rate in 1994. In contrast, the impaired (failed) projects have halved. The challenged projects remain, however, a serious concern. Since 1994, project cost overruns have averaged 189%. In 2002, this figure dropped drastically to 45%. Although schedule overruns were 222% in 1994, by 2002 this figure dropped to 63%.
According to Morris (2000), narrowly focused IT project management methodologies tend to concentrate on traditional, measurable project management goals and neglect criteria for project success, as determined by project stakeholders. It would appear that soft issues—difficult to define, and therefore manage, matters pertaining to people, relationships, attitudes, and other intangibles—are repeatedly neglected. The Standish Group International (1999) report argues that “project management is only beginning to understand the different roles and responsibilities of the stakeholders. Major research must be conducted to better understand the motivations of each of these constituencies” (p. 12). Our ensuing study explores the effects of stakeholders on IT projects and investigates whether these effects contribute to project success.
Review of the Literature
We begin by drawing a distinction between project success (evaluated against the project's overall goals) and project management success (evaluated according to the triple constraint of time, cost, and quality). We believe that it is important to note the differences between success criteria (measures for judging project success or failure) and success factors (inputs to the management system that lead directly or indirectly to the success of the project) (Cooke-Davies, 2002; De Wit, 1988).
While project management literature (Gilbert, 1983; Karlsen, 2002; Standish Group International, 1999) frequently concentrates on the management of timescales and budgets (project management success), Cleland (1986), Jergeas, G. F., Williamson, E., Skulmoski G.J. & Thomas, J.L. (2000), and Wateridge (1995) state that PMs should concentrate on success criteria relating to users and sponsors, which are major contributing factors driving success criteria (project success).
Cooke-Davies (2002), Hofmann and Lehner (2001), Karlsen (2002), Robertson (2000), and Wateridge (1995) agree that since stakeholders are the ones who decide whether a project is successful, the criteria for project success should clearly define criteria that all parties agree to. The criteria should undergo reviews as the project progresses. This becomes difficult when there is discord between the various stakeholders as to what constitutes success (Hartman & Ashrafi, 2002) or amongst management as to which metrics are appropriate (Cooke-Davies, 2002). The critical success factors (CSFs) from the Standish Group's 1994 and 1999 reports are listed in Exhibit 2. Executive support and user involvement are reported as the top two factors for success.
Exhibit 2: Project success factors
Why Are IT Projects Difficult to Manage?
IT project management is distinct from other project management. Software is intangible and software projects are often novel or innovative. There is an immature body of knowledge to guide their management. Moreover, each project is specifically designed to meet the user's requirements, thus it is unique in nature. Finally, IT projects are often undertaken to improve or solve certain business problems and therefore usually only last for a certain period of time (McLeod & Smith, 2001). Owing to these three features, the pressures of uncertainty appear; the need for integrating both business and project objectives is required, and planning and estimating are undertaken iteratively at each stage of the development life cycle.
In a conventional IT project, there are various parties involved in the system development life cycle, viz. sponsors, stakeholders, the design and implementation team, and finally the users. Frequently it is the interaction between these players that makes IT projects so difficult to manage. According to Thomsett (2002), the more time the PM spends with the stakeholders, the better.
In contrast to traditional project management, which focuses on the content of the project, the technical deliverables, technology, tools, and methodologies involved in a project, Thomsett's (2002) extreme project management (XPM) methodology is more concerned with the context of the project—the organizational, social, political, and financial environment in which a project is developed. He argues that what happens after the project is over is more important than what happens during the project. He claims that satisfying customers at delivery time is crucial, yet only recently has the post-implementation review received much attention. Most organizations focused on the development component, and very few accurately tracked either the support costs or the realization of benefits. Consequently, most senior managers do not realize the benefits brought by the implementation of IT projects; instead, they view IT as a cost center and are, therefore, often reluctant to support the IT project (Thomsett, 2002).
Most projects take place in a context where stakeholders play a major role in the accomplishment of tasks (Karlsen, 2002). According to various authors, many of the problems associated with IT projects are not technical in nature, but rather concerned with management, organizational, or behavioral issues (Johnston, 1995; Martin, 1994; Whitten, 1995; Yeo, 2002). Additionally, a project may be sensitive to actions and decisions taken by stakeholders; stakeholders, in turn, may also create both problems and uncertainty for the PM.
Stakeholders have power because they control information and resources: Many PMs have encountered stakeholders making greater demands on project execution (Karlsen, 2002). Part of a stakeholder's power lies in their being ultimately responsible for determining project success, given that they define and finance the project, though end users ultimately decide the usefulness of the project results. Indeed, “the issue of system acceptance may go beyond the usability and technical quality of the final product; extending to other more complex soft issues that are social and cultural in nature, including politics in information management” (Yeo, 2002, p. 241).
Determining the right stakeholder groupings, and understanding their interests and priorities, is vital for effective project management. Exhibit 3 summarizes the possible involvement of various stakeholder groups.
Exhibit 3: What stakeholders want and what they offer
Based on the shortcomings and issues identified in the literature, the following research question was posed:
Will an IT project management approach that focuses on the needs of IT project stakeholders and takes cognizance of soft project issues lead to an improved degree and rate of IT project success?
The definition of impaired, challenged, and successful projects used was that of the Standish Group International's CHAOS Chronicles (1994).
We define stakeholders as individuals and organizations that are actively involved in a software project or whose interests are affected by the project. There can be both internal and external stakeholders (i.e., members of the development team as well as clients and sponsors).
From our examination of the research literature, we developed the following hypotheses to show the various stakeholder variables that influence project outcomes and project success.
H1: Stakeholder involvement is related to project success.
H2: The extent of stakeholder management is related to project success.
H3: The extent of communication between the project manager and the project stakeholders is correlated with project success.
H4: Of the eight identified stakeholder groups, certain of these are perceived to have a greater impact on project success than others.
H5: There is general consensus by PM regarding stakeholder problems.
The target audience for this research was a group of experienced South African IT PMs. A survey approach was selected because it offers an accurate and anonymous means of collecting data. A validated questionnaire was emailed to PMs as a Word attachment. PMs were requested to think of a successful project and a challenged project in their recent experience and answer questions related to stakeholder involvement for both projects. In addition, a text box was included for most questions to allow the respondents to provide further information where appropriate. This format resulted in a questionnaire that was long and unattractive. Of the 139 emailed questionnaires, 18 valid responses were received by the deadline (a 13% response rate). While the sample is small, the data collected is comprehensive.
Analysis of Results
Seventeen of the eighteen responding PMs had at least a post-graduate university degree. The number of years of project management experience ranged from 1-to-23 years, with an average of 4.8 years. The successful project duration ranged from 1 month-to-28 months, with an average 11.1 months. The shortest running challenged project lasted 4 months and the longest persisted for 42 months; the average lifetime was 12.8 months. When comparing the effort expended on successful projects and challenged projects, a great deal more effort was expended on the challenged projects (average 87 person-months) than on the successful projects (average 38 person-months). The small sample was deemed one with rich IT project management experience; the analysis focuses specifically on the qualitative data captured.
Fourteen out of eighteen respondents indicated that stakeholder involvement was essential or critical to project success. Respondents substantiated these assertions with statements including “lesser involvement would have likely lowered the success” and “this made all the difference in the successful project.” One respondent indicated that “not using end users as stakeholders was problematic in challenged projects.” It is interesting to note that three respondents stated the opposite: that end users were not involved in their projects, nor were their opinions sought. One PM indicated that users are used to being told what functionality they require rather than the other way around.
Six of the PMs stated that managing stakeholders was essential. The reasons for this included that stakeholders needed to know what their specific roles were and what was required from them and when. One PM indicated that they managed stakeholders but not end users; this distorted the project's requirements.
With regards to the challenged projects, one PM stated that he felt he might have failed to communicate to the stakeholders what was required of them. Another indicated that managing the stakeholders was particularly difficult, as they didn't want a major part in the progress of the project.
When asked about the tools used to establish communication with, and obtain feedback from, stakeholders, nine respondents cited meetings or informal discussions, six cited telephone calls, and seven cited email. Regarding collaborative documents and tools, the respondents cited such tools as specification documents that were highlighted to indicate changes, an internal website or LAN (three references), and spreadsheet templates for issue logging.
Thirteen PMs stated that shaping stakeholder expectations was extremely important. Six indicated that the reason for this was the stakeholder's lack of IT knowledge. This lack of knowledge often lead to unrealistic expectations. Additionally, it was mentioned that expectations need careful management when things might be heading off track with respect to budget or deadlines. Three PMs felt that they ought to have managed the challenged project's stakeholders more extensively.
Fifteen of the respondents stated that they had been involved in communicating and selling their project vision, and viewed it as very important. Seven respondents indicated that they had sold their project vision using demonstrations or road shows. Others used meetings or email.
Thirteen PMs thought that their management style had a positive effect on stakeholders, encouraging them to ask questions. It was also mentioned that while coming forward with ideas and suggestions was important, the PM had to know when to say enough was enough and when to determine that a decision needed to be made. One PM felt his management style encouraged stakeholder buy-in.
Sixteen of the PMs indicated that for the successful projects, good outside support already existed. It appears that the nature of this support was more important to project success or failure, but unfortunately little information was gathered on this. One PM noted that while they had financial support, there was little appreciation of the risks involved and the complexity of the technical aspects of the project. Another stated that support was not timely. Where stakeholders were extensively involved and changes to specifications were suggested, developing support was essential but tricky.
Commitment: “stakeholders do not commit enough time to the project.”
Responses to this were mixed: three of the PMs said that end users were not permitted to take time to spend on the project and were certainly not encouraged to actively participate in the project. If this is linked back to the first question, the same PMs said that end users weren't actually asked what they wanted. Three said stakeholders were very cooperative and six agreed that lack of commitment was something they had experienced.
Skill: “The stakeholders do not have the skills necessary to participate in the business of gathering requirements; they describe solutions, not requirements, and they change their minds.”
Eleven PMs indicated that they agreed with this. It was only specified by two PMs that this was only the case on the challenged project. A recurring problem was that stakeholders did not have the IT skills necessary to deliver useful or precise requirements. Another PM indicated that stakeholders didn't fully take responsibility for “thinking requirements though.” Two PMs mentioned that requirements were incomplete and therefore changed. One PM indicated that the stakeholders had known exactly what they were supposed to do (i.e., their precise role in the project).
Discovery: “We can't discover who the appropriate stakeholders are; it's very difficult to find all the people who need to be involved in the process of discovering, inventing, and specifying requirements”
Five PMs indicated that while they knew who the appropriate stakeholders were, it was not practical to get in touch with all these people due to the sheer number of end users and the effort required in coordinating this. Of the PMs who had complained about lack of technical knowledge among stakeholders, one indicated that if they had dealt with all the end users, this problem would have been magnified. Five indicated that discovery was a problem.
Maintaining: “We can't keep the stakeholders interested and involved for the duration of the project.”
Ten PMs indicated that maintaining stakeholder interest was not a problem. While one PM indicated that the stakeholders were only interested in the financial aspects of the project. (In this case, the stakeholders were only sponsors with a financial/investment interest, which may explain this interest.) This PM did not say discuss the stakeholder interest in regards to other project matters. Of the two PMs who indicated that maintaining stakeholder interest was tricky, one PM indicated that this was only the case when they (the project team) actually required something from the stakeholder. The other PM was more vague, stating that it is more difficult to maintain stakeholder interest when the prospective outcome is less tangible.
Involvement: “The project would have achieved a higher degree of success with more stakeholder involvement”
Eight PMs felt that the project would have achieved a higher degree of success with more involvement. Three PMs stated that stakeholders were already extensively involved; four indicated that they felt that it was unlikely that greater involvement would lead to greater success because at a certain point in the project the team actually had to just get on with things or because dealings with the stakeholders had been problematic. Greater stakeholder involvement, therefore, was not desired.
Evaluation of hypothesis 1: Stakeholder involvement is related to project success.
This hypothesis seeks to determine whether stakeholders were more involved in the successful projects than in the challenged projects. The testing of this hypothesis involved the calculation of the mean level of stakeholder involvement in the successful and challenged projects.
By visually assessing the stakeholder involvement in projects successful and projects challenged (displayed in Exhibit 4), we saw that differences do exist between the two classifications. The largest differences can be seen in the extensively and limited categories. For the successful projects, the mean level of extensive involvement was more than twice that of project challenged. Similarly the mean number of PMs citing limited stakeholder involvement in successful projects was less than half of the number in the challenged projects.
Exhibit 4: Stakeholder involvement
To see whether general stakeholder involvement is correlated with project success, a correlation matrix was produced. No significant correlation was found between project success and the extent of stakeholder involvement in successful and challenged projects. To further assess the influence that stakeholder involvement has on project success, a simple linear regression was performed. The resultant coefficient was 0.282, which means that for each additional unit of stakeholder involvement, the project will become 0.282 more successful. While this is significant at the 5% level, the coefficient is very small and is not significant.
An attempt was made to establish at what systems development phase stakeholders were most involved, if any. A frequency count determined the number of times that PMs stated stakeholders had been involved during the development stage of requirements specifications, testing, or implementation. The frequency counts were converted to percentages for ease of comparison (see Exhibit 5 for results.)
Exhibit 5: Stakeholder involvement during the project life cycle
Stakeholders were involved in requirements specification (RS) in approximately a quarter of all projects (more than during the testing or implementation stages in either project). There was concern though that this analysis may not be sufficiently rigorous due to the overlap of categories. A PM may have indicated that feedback was obtained throughout the project from stakeholders. The PM may or may not have indicated that this meant the stakeholder was involved in particular stages in addition to giving feedback.
This qualitative data suggests that stakeholder involvement is strongly correlated with project success. The PMs reported that:
- “Greater involvement has resulted in more successful projects”
- “Successful project—lesser involvement would have likely lowered the success.”
- “Extensive stakeholder involvement is critical to the success of a project.”
- “Stakeholder involvement was essential.”
- “Stakeholder involvement essential; this made all the difference in the successful project.”
Furthermore, one PM said that “greater stakeholder involvement would have been better” and that “the lack of stakeholder involvement is what caused the (challenged) project to overrun time and cost”.
In light of these statements, the results in Exhibits 5 and 6 seem anomalous. It is unclear why less than half of the responding PMs obtained feedback from stakeholders throughout the project when the majority indicated that the extent of stakeholder involvement was an important variable in terms of project success. Similarly, another PM, who dealt exclusively with financial stakeholders, said that she “wouldn't have wanted other stakeholders involved later than in requirements; the project needs to get going and not always have outside interference.”
Where only a select group of stakeholders was involved, extensive involvement by these stakeholders did lead to project success. One PM stated that in the successful project, having experienced stakeholders who represented the needs of the end users well, led to project success.
It should be noted that the nature of stakeholder involvement was also important to project success. One respondent said that it would have helped the challenged project if stakeholders had reached consensus as to their requirements prior to the project meetings. One PM indicated that less involvement would have meant less scope-creep but that ultimately stakeholders did need to be involved to ensure acceptance of the product later on. The authors feel that this is particularly relevant due to Hartman and Ashrafi's (2002) assertion that user acceptance of the final product is a fundamental aspect of project success.
In terms of the actual stakeholders involved, it is inferred in six responses that a full range of stakeholders was not consulted. Clients and financial stakeholders were specifically mentioned as being important. Users were intentionally excluded by five respondents. Only one respondent indicated that this might have been problematic.
Evaluation of hypothesis 2: The extent of stakeholder management is related to project success.
The aim of this hypothesis was to determine whether there was a relationship between the extent to which stakeholders are managed and project success. In this hypothesis, various ways in which stakeholders may be handled were identified and tested for significance:
1) Managing the effective use of stakeholders.
2) Managing stakeholder expectations.
3) Shaping stakeholder expectations.
4) Establishing an environment conducive to commitment and working together.
5) Collecting information regarding team member abilities.
6) Creating a facilitative environment.
7) Performing a people issue audit.
A basic multiple regression analysis was applied because of the relatively small sample size. The coefficient of the determination (R2) value was very low (0.289). This shows that the extent of stakeholder management, viewed as a cohesive variable incorporating the aspects listed above, added very little explanatory value to a successful project outcome. A greater or lesser degree of stakeholder management did not substantially influence the project outcome. In addition, the F-test (F = 1.511 and p-value = 0.207) indicates a weak relationship.
Of the seven variables tested, only “Managing the effective use of stakeholders” was significant at the 5% level. “Managing stakeholder expectations” was significant at the 10% level.
The results generated from the t-test showed that the variable “Managing the effective use of stakeholders” has a t-value of 2.817, which is less than 2.898 (p = 0.005). The mean level of “Managing the effective use of stakeholders” in the successful projects was therefore highly significantly greater than in the challenged projects, at the 0.05% level. The average level of “Managing stakeholder expectations” was established to be significantly higher (at the 1% level (t = 2.149 < 2.567)) in the successful projects.
Several PMs said that stakeholder management was essential and several suggested that increased stakeholder management would have led to a greater chance of project success. In support of this statement, a number of PMs stated that they had either failed to manage the stakeholders satisfactorily, which led to the project being challenged, should have managed the stakeholders more, or had managed the wrong stakeholders by excluding the users. Another respondent felt that the challenged project might have experienced greater success if they had actively shaped stakeholder expectations. Another PM indicated that input from stakeholders who had experience with projects of a similar nature and therefore did not need much management led to project success. Another respondent said that managing stakeholders was extremely difficult—though necessary—and they suggested that this was a key factor in their project being challenged.
“Managing stakeholder expectations” is seen to include managing and shaping stakeholder expectations. PMs suggested that expectations of scope required active management. Seven respondents said that shaping expectations became increasingly important when the project was not progressing according to plan. On projects where stakeholders were not familiar with IT, the PMs shaping of stakeholder expectations was essential to ensure that stakeholders knew what to expect and would be happy with the outcome. When a stakeholder's actions prevented a PM from managing or shaping expectations, this led to inadequately met expectations and to implementation difficulties.
Creating a facilitative environment where stakeholders were encouraged to ask questions about the project and offer suggestions was an important aspect of managing stakeholders. The environment is where expectations may be managed or shaped. Fourteen PMs suggested that this environment was important to project success.
The environment was influenced by the PM’s management style, although only one PM indicated that her management style differed in the challenged and successful projects. Ten PMs said that they adopted a flexible, open, facilitative, or participative approach that they had hoped would have made stakeholders feel confident to ask questions.
Evaluation of hypothesis 3: The extent of communication between the project manager and the project stakeholders is correlated with project success.
To test this hypothesis, two variables were used:
1) Extent of management of the communication of project information to the external stakeholders (defined as those stakeholders outside of the development team)
2) Ability to communicate and sell the project vision to the stakeholders
A correlation matrix was produced to test the correlation between the two variables and project success. There is no correlation between “Extent of management of the communication of project information” and project success, as indicated by a low correlation coefficient (0.05). However, there is significant correlation (at the 5% level) between “Ability to communicate and sell the project vision” and project success (correlation coefficient = 0.381).
A t-test was conducted to analyze the difference between the means of both variables in successful and challenged projects. For the variable “Extent of management of the communication...,” the t-statistic is 0.286 with a p-value of 0.7767. There is no significant difference between the mean extents of “Extent of management of the communication…” in the two project classifications.
Testing the variable the “Ability to communicate and sell the project vision…,” the mean number of PMs who indicated that they were able to communicate and sell the project vision in the successful project was greater (at the 2% level) than in the challenged project.
PMs used several tools to communicate with—and obtain feedback from—stakeholders. While meetings, email, and telephone conversations featured most frequently, details as to the nature of meetings or interaction were not given.
Spreadsheets were cited twice and forms of collaborative document systems (such as Lotus Notes, spec docs mailed with highlighted changes, and a Microsoft Share point portal) were also used by three respondents. The authors feel that the investment in this type of shared system indicates the communication of project information was seen to be important to project success and therefore facilitated.
Seven respondents believed that project vision was very important to project success. A number of tools were used to communicate and sell vision to the stakeholders. Demonstrations or presentations were mentioned as well as such ideas as a road show, hard sell, and selling. These suggest that the PMs put significant effort into the communication and selling of project vision.
Evaluation of hypothesis 4: Of the eight identified stakeholder groups, certain of these are perceived to have a greater impact on project success than others.
The aim of this hypothesis is to identify those stakeholders that are perceived to contribute more to project success than others. Of the eight identified stakeholder types, certain of these are perceived to have a greater impact on project success than others.
To test this hypothesis, a multiple regression analysis was performed. The F-test (F = 0.477225 and p-value = 0.86058) indicated that most of the variation in project outcome is unexplained by any particular stakeholder's involvement. The coefficient of determination (R2) is 0.1325, which indicates that only 13.25% of the variation in the project outcome is explained by the involvement of particular stakeholders.
T-tests were performed to assess whether the mean level of involvement by each stakeholder was greater in either the successful or challenged project. While a minor difference in mean involvement between the two project outcomes was found in users, senior management, auditors/quality assurors and legal advisors, there was no significant difference for any of the eight stakeholders. All correspondent p-values were substantially higher than the 0.05 significant level. None of the eight identified stakeholders had a greater impact on project success.
Sponsors became particularly important in projects where budgets were aggressively managed. There is extensive support for involving programmers (techies) early on. PMs stated that involving programmers in requirements specification integrates a practical, technical perspective early on, which assists in the estimation of time scales. A number of PMs indicated that programmers performed systems analysts jobs as they had a very clear understanding of practical issues.
Evaluation of hypothesis 5: There is consensus among PMs regarding stakeholder problems.
The aim of this hypothesis was to determine whether PMs could reach consensus in terms of the five identified stakeholder problems. For this analysis, the degree of agreement by PMs as to each stakeholder problem, as identified by the Atlantic Systems Guild in Robertson (2003), was compared in all the projects. The incidence of each level of consent was counted for each problem and converted to a percentage for ease of comparison. The results are shown in Exhibit 6.
The statement concerning skill, discovery, and maintaining showed significant consensus by PMs. There was a reasonable level of consensus regarding the commitment and involvement statements, with which 53% and 55% of PMs agreed respectively. It is important to note that this consensus was not always in agreement with the problem statements. 62% of PMs disagreed that discovering the correct stakeholders was problematic. A noteworthy 76% of PMs did not agree that maintaining stakeholders’ interest for the duration of the project was a problem.
Exhibit 6: PM consensus on stakeholder problems
To examine the consensus in project successful and project challenged separately, a value of 1 to 5 (completely disagree = 1; completely agree = 5) was assigned to the levels of agreement and the mean was calculated and converted to a percentage for simplicity (see Exhibit 7). It is important to note that for the problems where there was agreement with the problem statement, the mean extent of agreement is considerably higher in project challenged. This indicates that this was more of a problem in the challenged projects.
For skill, the mean was 4.25 in project challenged. Fourteen PMs indicated that they agreed somewhat or completely that stakeholders did lacked the skills necessary to participate in the business of gathering requirements, described solutions and not requirements, and changed their minds.
Similarly, in project challenged, means of 3.56 and 3.69 were attained for commitment and involvement respectively, and ten PMs agreed somewhat or completely with the problem statement. This is compared to eight PMs who agreed somewhat or completely that commitment was a problem in the successful projects (mean 3.06). For the involvement statement in project successful, nine PMs agreed somewhat or completely (mean = 3.17).
When looking at the problems with which PMs disagreed, it should be noted that in the challenged projects, the level of disagreement was lower, indicating that while PMs didn't disagree with the statements, the problems were more significant in the challenged projects.
Exhibit 7: PM consensus on stakeholder problems by successful/challenged projects
Furthermore, to determine whether any of the common five stakeholder problems was seen to be particularly problematic, each level of agreement was again assigned a value from 1 (completely disagree) to 5 (agree completely). With a sample size of 34, and the highest score attainable a 5, the highest possible total was 170. Totals attained for each problem were converted to a percentage.
Exhibit 8: Analysis of the 5 stakeholder problems
As shown in Exhibit 8, skill is deemed to be a significant problem by the majority of the PMs (74%). The most significant skill lacking is that of IT or technical knowledge, cited by seven respondents. This was particularly problematic in RS. When stakeholders requested changes to project scope, PMs complained that the stakeholders did not grasp the complexity of the changes being requesting, and therefore stakeholders frequently did not understand a corresponding change in budget or deadline. The implication of lack of involvement by external stakeholders was that when PMs required input from these stakeholders, they either shirked or did not follow through quickly enough.
This report has drawn attention to the increasingly ubiquitous nature of IT. While the much-watched results of the Standish Group's (1994, 1999) CHAOS Chronicles indicate a slight improvement in the number of successful projects, almost half of all IT projects were classified as challenged in the 1999 report. The challenged IT projects discussed in the 1999 report are completed and operational but over-budget, over the time estimate, and offer fewer features and functions than originally intended. The report also highlighted, through a review of existing literature, that it is not technology that is failing (or resulting in the challenged classification), but rather the softer issues related to behavioral, organizational, and management performance.
Based on the substantial gap found in the literature concerning stakeholder management, this study attempted to ascertain whether an IT project management approach that focuses on the needs of IT project stakeholders and takes cognizance of soft project issues leads to an improved degree and rate of IT project success. Five hypotheses were used to test this question. While the quantitative data was useful for statistical testing, the qualitative data provided substantially more richness, offering insight into the PMs’ perceptions, attitudes, and behaviors. This was particularly important given that soft—and frequently unquantifiable—issues were being explored.
The level of effort expended on challenged projects was substantially higher than on successful projects. The authors conclude that this may not necessarily have exclusively been a result of the challenged nature of the project but could also have been due to a greater degree of project complexity.
When comparing the average level of involvement by external stakeholders in the successful and challenged projects, the stakeholders were significantly more involved in the successful projects. The qualitative data suggest that the nature of this involvement is particularly important.
As the extent of stakeholder involvement increases, there is a corresponding increase in associated problems, which may explain the anomalous results. Where stakeholders were skilled and experienced, knew what they wanted, and had decided what their requirements were prior to project meeting times, they were able to help generate particularly valuable interactions. However, continually attempting to accommodate and involve external stakeholders, particularly where knowledge of IT was poor, may lead to valuable development time being misspent and consequently, time overruns. One particular PM indicated that in the challenged project, the stakeholders were very difficult to deal with; this PM did not want further involvement with the stakeholder. This implies that a delicate balance needs to exist between ensuring that the needs of the stakeholder are being met and ensuring that development continues according to schedule.
It was significant that certain external stakeholders are seldom involved in projects. In the case of users, PMs frequently and specifically excluded them, often citing the impracticality of getting so many people's input as the motive. With regard to internal stakeholders, programmers were viewed as particularly valuable to the project team for their practical outlook. However, in a project team where an external stakeholder's IT knowledge is poor, communication difficulties may arise. Only one PM indicated that the programmers were good at explaining technical issues to external stakeholders. A significant problem mentioned was that external stakeholders indicated that the technology was “not their problem.” In such a case, where stakeholders refuse to take responsibility or become overwhelmed by their responsibilities, the PM needs to appeal to the sponsor.
It was found that the extent of stakeholder management had a strong influence on project success. A good deal of effort was put into managing stakeholders, and the authors’ inference is that PMs were aware that managing stakeholders was important to project success and therefore actively tried to manage them.
“Managing the effective use of stakeholders” and “Managing stakeholder expectations” were particularly important to project success. Significantly, when projects did not appear to be progressing according to plan, shaping expectations became essential.
While the extent of communication between PMs and stakeholders was not a significant factor leading to project success, communicating and selling project vision was seen to be very important. However, this is largely seen to be the role of senior management as opposed to the PMs themselves, as it is more strategic in nature than many of a PM’s daily responsibilities. Communication remains important and stakeholders need to understand their roles and responsibilities in the project development process.
No particular stakeholder was found to have a more significant effect on project success than any other stakeholder. Support by senior management did, however, emerge as a distinguishing factor between successful and challenged projects in the final model. This is not surprising given that the successful and challenged projects (with the same PM) took place in the same development environment. Additionally, senior management support is among the top ten factors, cited by the Standish Group's (1994, 1999) CHAOS Chronicles, as important for project success.
The stakeholder problems section of our questionnaire yielded interesting and striking results, with stakeholder skill emerging as the most significant problem. The mean level of agreement with all stakeholder problems was consistently higher in the challenged projects. The IT illiteracy of stakeholders was consistency mentioned as a problem. The authors feel that the implications for PMs are as follows:
- PMs need to assess the level of IT competency among external stakeholders and adapt their management approach accordingly. Where stakeholders have little understanding of technical issues, more emphasis should be placed on ensuring that stakeholders understand correctly how software will function.
- PMs will need to be very particular about the way scope changes are handled.
- PMs should anticipate that external stakeholders will not grasp the complexities of their requests and should therefore explain clearly what their requests will mean in terms of budget and deadline adjustment.
- Patience will also be required so that PMs can obtain as much information from IT-illiterate stakeholders as possible. These stakeholders must not be seen as having nothing to add, but rather that more effort will be required by the PMs to elicit useful information.
Stakeholder management is inherently difficult given the intangible nature of the variables involved. The type of project will also significantly affect the number and type of stakeholders involved, making definitive project management approaches hard to prescribe. For particularly large projects, stakeholders may change over the SDLC, making the PM’s task even more difficult. Additionally, while PMs may adopt a particular approach to influence or manage stakeholders, these stakeholders retain the ultimate power of deeming a project successful upon completion.
Recommendations for Further Research
Further research using a larger sample size is required for to obtain a more rigorous quantitative analysis. Although this research concerns stakeholders, the survey was only based on the perspectives of PMs. Further research could be conducted with the same focus, but from a particular stakeholder's point of view to see how perceptions differ between each group.
This research has largely viewed stakeholders as a general, cohesive group; in reality, this is seldom the case. Only hypothesis 4 was split to assess the contribution of various stakeholders, but it was nonetheless discussed relatively broadly. It is therefore recommended that future research explore the involvement, contribution, and influence of an individual stakeholder or a group of stakeholders deemed to be most influential.
Different aspects of project management that influence stakeholders could also be investigated. Does a different leadership style, for example, influence stakeholder involvement in a particular way or lead to a better project outcome?
The relationship between project outcome and a PM’s number of years of experience could also be investigated. The senior manager and PM are two crucial stakeholders, as identified by the Standish Group International (1994, 1999), and it would be interesting to ascertain whether an increased level of experience means improved stakeholder management.
Cleland, D. I., (1986). Project stakeholder management. Project Management Journal, 17(4), 36-44.
Cooke-Davies, T. (2002). The “real” success factors on projects. International Journal of Project Management, 20, 185-190.
De Wit, A. (1988). Measurement of project success. International Journal of Project Management, 6.
Gilbert, G. P. (1983). The project management environment. International Journal of Project Management, 1(2), 83-87.
Hartman, F., & Ashrafi, R. (2002). Project management in the information systems and information technology industries. Project Management Journal, September 5-15.
Hofmann, H. F., & Lehner, F. (2001). Requirements engineering as a success factor in projects. IEEE Software, Vol.18 (4), July/August, 58-66.
Jergeas, G. F., Williamson, E., Skulmoski G.J. & Thomas, J.L. (2000). Stakeholder Management on Construction Projects. 2000 AACE International Transactions, PM 12.1-12.6.
Johnston, A. K. (1995). A hacker's guide to project management. Oxford, UK: Butterworth-Heinemann.
Karlsen, J. T. (2002). Project Stakeholder Management. Engineering Management Journal, 14(4), 19-24.
Martin, J. E. (1994). “Revolution, risk, runaways: Three Rs of IS projects. Proceedings of the Project Management Institute's 25th Annual Symposium, Vancouver, Canada, 266-272.
McLeod, G., & Smith, D. (2001). Managing information technology projects. Cape Town, South Africa: Inspired Press.
Morris, P. (2000). Researching the unanswered questions of project management. In Proceedings of PMI Research Conference (pp. 87-102). Paris, June 21-24. Newtown Square, PA: Project Management Institute.
Murray, J. P. (2001). Recognizing the responsibility of a failed information technology project as a shared failure. Information Systems Management, 18(2). Retrieved May 21, 2004, from http://www.auerbach-publications.com/ejournals/articles/article.asp?id=31274.
Robertson, S. (2000). Project sociology: Identifying and involving the stakeholders. Retrieved June 24, 2003, from http://www.guild.demon.co.uk/ProjectSociology.pdf.
Robertson, S. (2003). Stakeholder Concerns Survey. Retrieved June 21, 2003, from http://www.systemsguild.com/GuildSite/SQR/stakeconcerns.html.
Roetzheim, W. (1993). Managing software projects: Unique problems and requirements. In P.C. Dinsmore (Ed.), AMA Handbook of Project Management (pp. 347-352). New York: Amacom.
Sauer, C., & Cuthbertson, C. (2003). The key character traits and skills of today's effective project manager. Retrieved April 4 2003, from http://www.cw360ms.com/pmsurveyresults/index.asp.
Standish Group International. (1994). CHAOS Chronicles. Retrieved April 11, 2003, from http://www.pm2go.com/sample_research/unfinished_voyages_3.php.
Standish Group International. (1999). Chaos: A recipe for success. Retrieved April 11, 2003, from http://www.standishgroup.com/sample_research/PDFpages/chaos1998.pdf.
Standish Group International. (2003). Latest Standish Group CHAOS Report Shows Project Success Rates Have Improved by 50%. Retrieved May 20, 2004, from http://www.standishgroup.com/press/article.php?id=2
Stratigos, A. (2003). Managing up: Stakeholder relationship imperatives. Online, March/April, 69-71.
Thomsett, R. (2002). Radical project management. Upper Saddle River, NJ: Prentice-Hall.
Wateridge, J. (1995). IT projects: A basis for success. International Journal of Project Management, 13(3), 169-172.
Whitten, N. (1995). Managing software development projects (2nd ed.). New York: John Wiley and Sons.
Yeo, K. T. (2002). Critical failure factors in information system projects. International Journal of Project Management, 20, 241-246.
PMI research shows project teams that draw from an array of perspectives and skillsets deliver powerful outcomes.