Abstract
The Standish Group's (2009) Chaos Summary reported that 32% of information technology (IT) projects undertaken in the United States in 2008 were successful, whereas 68% were late, over budget, or canceled. High failure rates among IT projects have been attributed to a wide range of organizational, process, and technological causes. One underlying cause of project failure may be unrecognized project complexity. The purpose of this study was to gather data on a sample of IT projects to assess the relationship between IT project complexity and complication and then compare their individual relationships with IT project success. A conceptual model was developed depicting the relationships among IT project complexity (ITPCx), IT project complication (ITPCn), and IT project success (ITPS). Three quantitative constructs representing the study variables were synthesized from prior literature. A survey instrument was developed to gather data about participant demographics, organizational characteristics, and each of the factors of ITPCx, ITPCn, and ITPS. Survey invitations were sent by email directly to the membership of Project Management Institute's Information Systems Community of Practice (PMI IS CoP). A total of 235 qualified responses were received, exceeding the minimum sample size of n = 115 as determined by power analysis. After correction for a non-normal distribution of the responses for ITPS with a rank order normal transformation, Pearson's correlation coefficients were derived for the relationships between the independent variable constructs ITPCx and ITPCn, and between the independent constructs and the dependent NITPS normal transform. To confirm the validity of the parametric correlations with the normal transform, nonparametric Kendall's taub and Spearman's rho correlations were also calculated between the independent variable constructs ITPCx and ITPCn and the dependent ITPS construct. Results indicated that IT project complexity and IT project complication were positively correlated but were distinct sets of project characteristics. In addition, IT project complexity had a greater negative correlation with IT project success than did IT project complication.
Keywords: project complexity; project complication; project success; information technology; complex adaptive systems
Introduction
The Standish Group's (2009) Chaos Summary reported that only 32% of information technology (IT) projects undertaken in the United States in 2008 were considered successful, whereas 68% were late, over budget, or canceled. Some researchers have suggested the criteria used in the widely-cited Standish studies are too restrictive, resulting in inflated failure rates (El Emam & Koru, 2008; Eveleens & Verhoef, 2010; Glass, 2006). Regardless of the specific success criteria, IT project failure is a significant and growing problem worldwide (Bharadwaj, Keil, & Mähring, 2009; Nelson, 2007). Total direct and indirect costs in the United States alone were estimated recently at more than US$1.2 trillion a year (Sessions, 2009).
Historically, high failure rates among IT projects have been attributed to a wide range of organizational, process, and technological factors. Hartman and Ashrafi (2002) cited a lack of alignment between business and project objectives, misunderstood requirements, and overly optimistic schedules and budgets among the managerial and organizational causes. Jugdev and Thomas (2002) found a lack of process maturity and inadequate application of existing project management practices and guidelines. Kappelman, McKeeman, and Zhang (2006) included a lack of management support, inadequate project manager skills, insufficient stakeholder involvement, poor requirements management, and inadequate change control among 53 leading indicators of IT project failure.
A potentially significant underlying cause of project failure, however, may be unrecognized project complexity (Benbya & McKelvey, 2006). To help investigate its effects, a distinction was made in this study between project complication and project complexity. Project complication is related to factors of scale, including extensive and detailed requirements, large and geographically dispersed project teams, high project cost, and long project duration (Cilliers, 1998; Hass, 2009). Project complexity, on the other hand, emerges when project objectives are not clearly defined, requirements are unclear and volatile, the project incorporates significant technological or organizational change, or the project environment exhibits extensive political and social influences, dependencies, and constraints (Baccarini, 1996; Jaafari, 2003; Whitty & Maylor, 2009). Although complication can make a project difficult to manage, complexity can cause project behavior to become unpredictable and unmanageable (Benbya & McKelvey, 2006; Xia & Lee, 2004).
The purpose of this study was to gather data on a sample of IT projects in order to assess the relationship between IT project complexity and complication, and then compare their individual relationships with IT project success. A model of IT project complexity (ITPCx), IT project complication (ITPCn), and IT project success (ITPS) was synthesized from existing literature. A survey instrument was developed and tested, and used to collect data on IT project complexity, complication, and success from a sample of members of Project Management Institute's Information Systems Community of Practice (PMI IS CoP). Complexity and complication were found to be positively correlated but were distinct sets of project characteristics (r = .530, r2 = .281, p < .001). Project complexity had a greater negative correlation with project success (r = -.350, r2 = .123, p < .001) than did project complication (r = -.228, r2 = .052, p < .001).
IT Project Complexity, Complication, and Success
In previous studies of IT project complexity, factors and elements of project complexity have been analyzed and organized according to a number of approaches and schemas. For the purposes of this study, a complex adaptive systems view was applied to distinguish characteristics related to IT project complication from those related to IT project complexity. The relationships between these sets of project characteristics and IT project success were then assessed using a definition of project success similar to that applied in the Standish (2009) Chaos Studies.
Projects as Complex Adaptive Systems
As temporary organizations (Turner & Muller, 2003), projects can be studied in the context of the organizational paradigms from general systems theory. The rational, natural, and open systems views have been the predominant systems paradigms of sociological structures and organizations since the early 1960s (Katz & Kahn, 1966; Parsons, 1960; Thompson, 2003). More recently, however, a complex adaptive systems view has emerged. It has since been applied to the study of a wide range of sociological entities including, for example, societies (Buckley, 1968), economies (Durlauf, 1997), communities (Stackman, Henderson, & Bloch, 2006), organizations (Thietart & Forgues, 1995), and projects (Jaafari, 2003).
The complex adaptive systems view recognizes that the behavior of complex systems can often appear nondeterministic, nonlinear, and unpredictable (Rittel & Webber, 1973). It identifies characteristics of systems such as sensitivity to small variations in environment and inputs, instability and rapid change, and evolutionary learning and adaptation that tend to lead to emergent and unanticipated patterns of behavior (Maturana & Varela, 1980).
Depending on the extent of the interconnections between their component parts, the degree to which they address unfamiliar and changing problems, and the extent to which they interact with and influence their environments, projects can exhibit the characteristics of complex adaptive systems (Churchman, 1967; Rittel & Webber, 1973). Several decades after it was first applied to sociological entities, however, complex adaptive systems theory has not made its way into general project management practice. Jaafari (2003) suggested the rapid evolution of complexity theory itself is one possible explanation for the lack of practical application. Recent progress indicates the field may soon benefit from a more extensive integration of general systems theory, organization theory, complexity theory, and project management theory.
IT Project Complexity and Complication
In the 20th anniversary edition of his classic work on software development, Brooks (1995) concluded, “complexity is the business we are in, and complexity is what limits us” (p. 226). Many IT projects are complicated; however, it is not necessarily the volume of requirements and subsystems, or the number of stakeholders and team members that makes a project complex (Cilliers, 1998). Complexity emerges when the interactions among the subsystems, stakeholders, team members, and environment are interdependent and adaptive, with each influencing the others and causing them to change, often unpredictably (Holland, 1992; Lewin, 1999). Because of their inherent intangibility and interconnectivity, information technology projects tend to be among the most complex of all project types (Xia & Lee, 2004).
Project complexity tends to be exacerbated when certain project conditions exist (Austin, Newton, Steele, & Waskett, 2002; Baccarini, 1996; Brockhoff, 2006; Cooke-Davies, Cicmil, Crawford, & Richardson, 2007):
- The problem or opportunity is not clearly defined.
- There are a large number of unknowns.
- The requirements are unclear and volatile.
- The outcome is unpredictable.
- The project schedule is overambitious or overconstrained.
- The project uses or creates new technology.
- There is a rapid rate of technological change.
- There are significant political and social influences.
- There are critical external dependencies and constraints.
- The project itself creates significant change.
Xia and Lee (2004) investigated IT project complexity by grouping project attributes into structural and dynamic dimensions of organizational and IT complexity. In the study, they found significant negative correlations between structural organizational complexity and elements of IT project success (r = -.311 to - .395, p < .01), weaker and less significant correlations between dynamic organizational complexity and IT project cost (r = -.085, p < .1) and between dynamic IT complexity and project functionality (r = -.091, p < .05), and no correlation between structural IT complexity and IT project success.
Shenhar and Dvir (2007) and Sauser, Reilly, and Shenhar (2009) developed a novelty-technology-complexity-pace (NTCP) model of project complexity from their work on contingency approaches to project classification. The four dimensions were depicted graphically as a diamond, offering both visual and quantitative representations of project complexity. In some cases, they combined factors of both complication and complexity; for example, the extent of technology used in a project appeared to have less of an influence on complexity than did the degree to which the technology itself was changing.
Hass (2007a, 2007b, 2009) expanded on the NTCP model to create a project complexity model (PCM) with 11 dimensions of complexity, including relatively straightforward scale-related factors such as duration, cost, and team size, as well as more disruptive factors such as requirements volatility, organizational and technological changes, and strategic and political influences.
IT Project Success
Project success has been measured with a wide variety of criteria (Shenhar, Dvir, Levy, & Maltz, 2001). Traditional measures like the triple constraint emerged from the field of operations research (Morris, 1994). More recent standards have incorporated the process and product of the project (Baccarini, 1999), as well as the project's contribution to the organization, stakeholders, and environment (Brown, Dillard, & Marshall, 2006). Researchers have also considered contingency-based models (Evaristo & van Fenema, 1999), and found that the perception of project success often differs from its quantitative measurement (Besner & Hobbs, 2006; White & Fortune, 2002).
The traditional triple constraint approach has been applied to IT projects with varying degrees of effectiveness. Agarwal and Rathod (2006) found differences in the priorities of scope, cost, schedule, and additional criteria for project quality among Indian contract software development organizations with high Capability Maturity Model (CMM) ratings. Atkinson (1999) pointed out the difficulty of estimating and measuring project cost and duration, and balancing them against indicators of project quality.
In spite of such limitations, the Standish Group (1994, 1999, 2009) research has consistently used project success categories based on the triple constraint:
- Successful: Completed on time and on budget, with all features and functions as initially specified
- Challenged: Completed and operational but over budget, late, with fewer features and functions than originally specified
- Failed: Canceled before completion or never implemented
Research Questions and Hypotheses
The purpose of this study was to investigate the relationships among IT project complexity, complication, and success. The specific research questions and hypotheses addressed included:
| RQ1: | To what extent, if any, is IT project complexity related to IT project complication? |
| H10: | IT project complexity is not correlated with IT project complication |
| H1A: | IT project complexity is correlated with IT project complication |
| RQ2: | To what extent, if any, is IT project complexity related to IT project success? |
| H20: | IT project complexity is not correlated with IT project success |
| H2A: | IT project complexity is correlated with IT project success |
| RQ3: | To what extent, if any, is IT project complication related to IT project success? |
| H30: | IT project complication is not correlated with IT project success |
| H3A: | IT project complication is correlated with IT project success |
| RQ4: | To what extent, if any, is IT project complexity more strongly related to IT project success than is IT project complication? |
| H40: | IT project complication has an equal or greater correlation with IT project success than does IT project complexity |
| H4A: | IT project complexity has a greater correlation with IT project success than does IT project complication |
Methodology and Data Collection
A conceptual model was developed depicting the hypothesized relationships among the study variables (see Figure 1). Three quantitative constructs corresponding to the study variables were also developed. Elements and factors of ITPCx and ITPCn were derived primarily from the Hass (2009) PCM model, with 26 factors of ITPCx aggregated to 13 elements (see Table 3) and 14 factors of ITPCn aggregated to 9 elements (see Table 4). Criteria for ITPS were based on the triple constraint and Standish Group (2009) categories, with 8 factors aggregated to 3 elements (see Table 5). All factor and element aggregations were performed by summing factor scores with equal weight and dividing by the number of factors to obtain an aggregated mean score for each element, then by summing each element score with equal weight and dividing by the number of elements to obtain an aggregated mean score for each construct.
Figure 1: Conceptual model.
No existing instruments differentiating between IT project complexity and IT project complication were identified in the literature review. A new survey instrument was developed to gather data about participant demographics, organizational characteristics, and each of the factors of IT project complexity, IT project complication, and IT project success. Survey questions used mostly five-point Likert-type scales with low-to-high response ranges. Prior to data collection, the survey was field tested, then validated by a pilot test with 42 experienced IT project managers.
The target population for the study was U.S. IT project management practitioners. Survey invitations were sent by email directly to the entire membership of the PMI IS CoP, and the survey was hosted at SurveyMonkey.com. For purposes of comparison with prior studies, only responses from U.S. members were included in the initial study analysis. A total of 235 qualified responses were received and included in the data set, which was imported into SPSS® for further processing and analysis.
Data Analysis and Results
For a study population of approximately 6,000 U.S.-based PMI IS CoP members, sample power analysis indicated a minimum reliable sample size of n = 115 with sample power (1-ß error probability) = .95 and desired correlation effect ρ = .30. To maximize response, a 100% probability sample was used. The survey response rate of 3.9% was lower than the 6% to 15% rate exhibited in previous studies using the same population (Mishra, Sinha, & Thirumalai, 2009; Wallace, Keil, & Rai, 2004; Xia & Lee, 2005). However, post hoc power analysis with the actual sample size n = 235 yielded sample power (1-ß error probability) = .9989, indicating the sample had more than sufficient statistical power for correlation analysis (Kraemer & Thiemann, 1987; Murphy & Myors, 2004).
Because the survey instrument had not been previously validated, internal consistency of the survey questions and constructs for ITPCx, ITPCn, and ITPS was assessed with Cronbach's alpha (α). An overall score for standardized items of α = .738 for the survey instrument, and scores of α = .847 for ITPCx, α = .546 for ITPCn, and α = .766 for ITPS indicated acceptable reliability for the survey instrument and constructs (Cortina, 1993; Nunnally, 1978).
Demographics
The 235 survey participants represented more than 20 industries including finance, insurance, and banking (n = 51, 21.70%), information technology (n = 48, 20.43%), healthcare (n = 27, 11.49%), and others (n = 25, 10.64%) for which the most common responses included pharmaceuticals, media, and government. Organization size ranged from fewer than 10 to more than 10,000 employees, with the most common response of 1,000 or more employees (n = 120, 51.1%) followed by 10 to 99 employees (n = 49, 20.9%).
The most common project type cited by participants (Table 1) was information technology (n = 92, 39.15%), followed by software development (n = 77, 32.77%) and application package implementation (n = 26, 11.06%).
Table 1: Project type frequency distribution.
| Project type | n | % |
| Information Technology | 92 | 39.15 |
| Software Development | 77 | 32.77 |
| Application Package Implementation | 26 | 11.06 |
| Other | 19 | 8.09 |
| Business Change/Reorganization | 18 | 7.66 |
| Engineering | 3 | 1.28 |
| Total | 235 | 100.00 |
The most commonly reported project roles (Table 2) were project manager (n = 130, 55.32%), program manager (n = 59, 25.11%), and project team member (n = 17, 7.23%). A majority of survey participants (n = 178, 75.74%) also reported that they or the primary project manager held the Project Management Professional (PMP)® certification.
Table 2: Project role frequency distribution.
| Project role | n | % |
| Project Manager | 130 | 55.32 |
| Program Manager | 59 | 25.11 |
| Project Team Member | 17 | 7.23 |
| Other | 12 | 5.11 |
| Consultant | 11 | 4.68 |
| Stakeholder or Customer | 3 | 1.28 |
| Project Sponsor | 3 | 1.28 |
| Total | 235 | 100.00 |
IT Project Complexity
Means for survey response distributions of factors and elements of ITPCx (Table 3) ranged from a low of M(SD) = 1.97 (0.92), 95% CI [1.82, 2.05] for clarity of project objectives (ITPCx2a) to a high of M (SD) = 4.06 (0.97), 95% CI [3.94, 4.18] for strategic environmental influences (ITPCx8b). The aggregated ITPCx construct exhibited a mean and standard deviation M (SD) = 2.91 (0.54), 95% CI [2.84, 2.98].
Skewness tests of the factors and elements and factors of ITPCx identified individual distributions skewed as slightly as -0.01 leftward for the schedule element (ITPCx6) and as greatly as 1.07 rightward for the opportunity clarity factor (ITPCx2a). Similarly, kurtosis tests revealed kurtosis as small as 0.02 for the methodology element (ITPCx5) and as great as -1.26 for the schedule reasonableness factor (ITPCx6a). Summary tests for the ITPCx construct indicated slight rightward skewness = 0.15 and moderate platykurtosis = -0.30 or a moderately flatter top and thinner tails than the normal distribution (DeCarlo, 1997).
Table 3: Summary statistics for elements and factors of IT Project Complexity (ITPCx).
| Element/Factor | Variable | n | M (SD) | 95% CI | Skewnessa | Kurtosisb |
| Objectives | ITPCx1 | 235 | 2.23 (1.14) | [2.08, 2.37] | 0.81 | -0.35 |
| Opportunity | ITPCx2 | 235 | 2.05 (0.84) | [1.95, 2.17] | 0.75 | 0.35 |
| Clarity | ITPCx2a | 235 | 1.97 (0.92) | [1.82, 2.05] | 1.07 | 0.86 |
| Familiarity | ITPCx2b | 234 | 2.18 (1.08) | [2.04, 2.31] | 0.90 | 0.03 |
| Solution | ITPCx3 | 234 | 2.57 (1.14) | [2.42, 2.72] | 0.24 | -0.82 |
| Familiarity | ITPCx3a | 234 | 2.66 (1.31) | [2.49, 2.83] | 0.27 | -1.22 |
| Availability | ITPCx3b | 233 | 2.49 (1.25) | [2.33, 2.65] | 0.50 | -0.86 |
| Team | ITPCx4 | 235 | 2.48 (1.05) | [2.35, 2.62] | 0.44 | -0.51 |
| Experience | ITPCx4a | 235 | 2.35 (1.14) | [2.20, 2.50] | 0.61 | -0.58 |
| Track Record | ITPCx4b | 235 | 2.62 (1.23) | [2.46, 2.78] | 0.36 | -0.92 |
| Methodology | ITPCx5 | 235 | 2.34 (0.95) | [2.21, 2.46] | 0.58 | 0.02 |
| Formality | ITPCx5a | 235 | 2.50 (1.20) | [2.35, 2.65] | 0.62 | -0.63 |
| Consistency | ITPCx5b | 233 | 2.18 (1.03) | [2.04, 2.31] | 1.05 | 0.59 |
| Schedule | ITPCx6 | 235 | 3.08 (1.05) | [2.94, 3.21] | -0.01 | -0.76 |
| Reasonableness | ITPCx6a | 235 | 2.98 (1.24) | [2.82, 3.14] | 0.18 | -1.26 |
| Flexibility | ITPCx6b | 234 | 3.17 (1.25) | [3.01, 3.33] | -0.14 | -1.13 |
| Requirements | ITPCx7 | 235 | 3.20 (1.00) | [3.06, 3.33] | -0.13 | -0.59 |
| Clarity | ITPCx7a | 235 | 2.71 (1.13) | [2.57, 2.86] | 0.38 | -0.85 |
| Stability | ITPCx7b | 235 | 3.69 (1.13) | [3.54, 3.83] | -0.80 | -0.22 |
| Environmental | ITPCx8 | 235 | 3.31 (0.74) | [3.22, 3.41] | -0.04 | -0.02 |
| Political | ITPCx8a | 232 | 3.81 (1.23) | [3.65, 3.97] | -0.80 | -0.36 |
| Strategic | ITPCx8b | 235 | 4.06 (0.97) | [3.94, 4.18] | -0.98 | 0.56 |
| Stakeholders | ITPCx8c | 234 | 3.33 (1.15) | [3.18, 3.48] | -0.16 | -0.77 |
| Dependencies | ITPCx8d | 234 | 3.50 (1.16) | [3.36, 3.65] | -0.39 | -0.68 |
| Regulatory | ITPCx8e | 222 | 2.35 (1.34) | [2.17, 2.53] | 0.65 | -0.77 |
| Legal | ITPCx8f | 227 | 2.74 (1.38) | [2.56, 2.93] | 0.28 | -1.20 |
| Information technology | ITPCx9 | 235 | 3.55 (0.82) | [3.44, 3.65] | -0.27 | -0.05 |
| IT complexity | ITPCx9a | 235 | 3.74 (0.92) | [3.62, 3.85] | -0.35 | 0.07 |
| Innovation | ITPCx9b | 233 | 3.36 (1.01) | [3.23, 3.49] | -0.20 | -0.44 |
| Technological change | ITPCx10 | 234 | 3.28 (1.27) | [3.12, 3.45] | -0.20 | -1.09 |
| Organizational change | ITPCx11 | 234 | 3.15 (1.11) | [3.01, 3.30] | 0.04 | -0.89 |
| Business processes | ITPCx11a | 234 | 3.46 (1.17) | [3.31, 3.61] | -0.22 | -0.90 |
| Organizational scope | ITPCx11b | 230 | 2.84 (1.30) | [2.67, 3.01] | 0.12 | -1.10 |
| Staffing | ITPCx12 | 235 | 3.30 (1.24) | [3.14, 3.46] | 0.08 | -1.15 |
| IT integration | ITPCx13 | 230 | 3.33 (1.26) | [3.16, 3.49] | -0.03 | -1.06 |
| IT project complexity | ITPCx | 235 | 2.91 (0.54) | [2.84, 2.98] | 0.15 | -0.30 |
Note. aStd. Error = 0.16. bStd. Error = 0.32.
The histogram of the distribution for ITPCx (Figure 2) displayed reasonable fit with the normal curve, with some bimodality below the mean corresponding to the indication of platykurtosis in the summary statistics.
Figure 2: Distribution of ITPCx with normal curve.
IT Project Complication
Means for survey response distributions of factors and elements of ITPCn (Table 4) ranged from a low of M (SD) = 1.88 (1.03), 95% CI [1.74, 2.01] for leadership competence (ITPCn1b) to a high of M (SD) = 3.71 (1.14), 95% CI [3.57, 3.86] for organizational units involved (ITPCn8). The aggregated ITPCn construct exhibited a mean and standard deviation M (SD) = 3.04 (0.48), 95% CI [2.98, 3.10].
Skewness of the factors and elements and factors of ITPCn ranged from as little as 0.00 for planned cost (ITPCn4a) to as much as 1.38 rightward for leadership competence (ITPCn1b). Kurtosis ranged from as little as -0.01 for the organizational support (ITPCn7) to as much as -1.72 for the planned cost (ITPCn4a). The ITPCn construct exhibited slight rightward skewness = 0.24 and moderate platykurtosis = -0.28.
Table 4: Summary statistics for elements and factors of IT Project Complication (ITPCn).
| Element/Factor | Variable | n | M (SD) | 95% CI | Skewnessa | Kurtosisb |
| Leadership | ITPCn1 | 235 | 2.01 (1.01) | [1.88, 2.14] | 1.14 | 0.89 |
| Experience | ITPCn1a | 234 | 2.13 (1.13) | [1.99, 2.28] | 1.02 | 0.30 |
| Competence | ITPCn1b | 234 | 1.88 (1.03) | [1.74, 2.01] | 1.38 | 1.47 |
| Duration | ITPCn2 | 234 | 3.27 (1.30) | [3.10, 3.44] | -0.12 | -1.20 |
| Team size | ITPCn3 | 235 | 3.57 (1.20) | [3.41, 3.72] | -0.16 | -1.05 |
| Cost | ITPCn4 | 224 | 3.23 (0.93) | [3.11, 3.35] | -0.03 | -0.68 |
| Planned | ITPCn4a | 209 | 3.07 (1.70) | [2.84, 3.30] | 0.00 | -1.72 |
| Flexibility | ITPCn4b | 222 | 3.45(1.02) | [3.31, 3.58] | -0.34 | -0.07 |
| Scope | ITPCn5 | 234 | 3.26 (1.03) | [3.13, 3.40] | -0.27 | -0.12 |
| Technology content | ITPCn6 | 234 | 3.62 (1.20) | [3.47, 3.77] | -0.56 | -0.59 |
| Organizational support | ITPCn7 | 235 | 2.28 (0.79) | [2.18, 2.38] | 0.36 | -0.01 |
| Executives | ITPCn7a | 233 | 2.02 (0.93) | [1.90, 2.14] | 0.75 | 0.24 |
| Users | ITPCn7b | 228 | 2.54 (0.96) | [2.42, 2.67] | 0.27 | -0.22 |
| Organizational units | ITPCn8 | 234 | 3.71 (1.14) | [3.57, 3.86] | -0.15 | -1.38 |
| Contractors | ITPCn9 | 230 | 2.39 (0.91) | [2.27, 2.51] | 0.08 | -0.83 |
| Number | ITPCn9a | 229 | 2.59 (1.19) | [2.43, 2.74] | 0.62 | -0.37 |
| Familiarity | ITPCn9b | 191 | 2.61 (1.23) | [2.44, 2.79] | 0.38 | -0.96 |
| Track record | ITPCn9c | 187 | 2.39 (0.93) | [2.25, 2.52] | 0.34 | -0.54 |
| IT project complication | ITPCn | 235 | 3.04 (0.48) | [2.98, 3.10] | 0.24 | -0.28 |
Note. aStd. Error = 0.16. bStd. Error = 0.32.
The histogram for the distribution for ITPCn (Figure 3) also showed reasonable visual fit with the normal curve. Apparent bimodality was less than that for ITPCx; however, there was a spike slightly above the mean and visible indication of moderate platykurtosis as indicated by the summary statistics.
Figure 3: Distribution of ITPCn with normal curve.
IT Project Success
Means for the distributions of factors and elements of ITPS were consistently high (see Table 5), with a low of M (SD) = 3.32 (1.26), 95% CI [3.16, 3.48] for schedule performance against baseline 1 (ITPS2a) to a high of M (SD) = 4.40 (0.94), 95% CI [4.28, 4.52] for scope completion (ITPS1a). The ITPS construct exhibited an unexpectedly high mean and standard deviation M (SD) = 3.92 (0.78), 95% CI [3.82, 4.02].
Skewness of ITPS elements and factors was also high. The least skewed was schedule performance against baseline 1 (ITPS2a) with slight left skew = -0.32. Scope completion (ITPS1a), however, was highly left skewed = -1.89. Skewness for ITPS was the greatest of the three aggregated constructs with extensive left skewness = -1.00. Kurtosis ranged from 0.15 for budget performance against the final baseline n (ITPS3b) to 3.49 for scope completion (ITPS1a). The ITPS construct itself displayed extensive leptokurtosis = 0.94, indicating a narrower peak and broader tails than the normal distribution (DeCarlo, 1997), typical for a highly skewed distribution (Hopkins & Weeks, 1986).
Table 5: Summary statistics for elements and factors of IT Project Success (ITPS).
| Element/Factor | Variable | n | M (SD) | 95% CI | Skewnessa | Kurtosisb |
| Completion | ITPS1 | 234 | 4.32 (0.91) | [4.20, 4.44] | -1.76 | 3.08 |
| % Completed | ITPS1a | 233 | 4.40 (0.94) | [4.28, 4.52] | -1.89 | 3.49 |
| % Implemented | ITPS1b | 231 | 4.24 (1.07) | [4.10, 4.38] | -1.66 | 2.24 |
| Performance (baseline 1) | ITPS2 | 232 | 3.57 (0.91) | [3.45, 3.69] | -0.54 | -0.06 |
| % Schedule | ITPS2a | 228 | 3.32 (1.26) | [3.16, 3.48] | -0.32 | -0.89 |
| % Budget | ITPS2b | 212 | 3.62 (1.23) | [3.46, 3.79] | -0.59 | -0.59 |
| % Scope | ITPS2c | 231 | 3.82 (1.32) | [3.64, 3.98] | -0.87 | -0.51 |
| Performance (baseline n) | ITPS3 | 231 | 3.87 (0.94) | [3.75, 3.99] | -0.59 | -0.66 |
| % Schedule | ITPS3a | 228 | 3.72 (1.28) | [3.55, 3.89] | -0.79 | -0.48 |
| % Budget | ITPS3b | 210 | 3.90 (1.20) | [3.74, 4.06] | -1.01 | 0.15 |
| % Scope | ITPS3c | 228 | 4.00 (1.30) | [3.83, 4.17] | -1.21 | 0.24 |
| IT project success | ITPS | 235 | 3.92 (0.78) | [3.82, 4.02] | -1.00 | 0.94 |
Note. aStd. Error = 0.16. bStd. Error = 0.32.
The frequency distribution histogram for ITPS confirmed the extensive left skewness = -1.00 and leptokurtosis = 0.94 indicated by the summary statistics (see Figure 4). The superimposed normal curve peaked near 4.0 but the peak of the actual distribution was farther to the right, at 4.2 with some bimodality and another local peak at 3.2.
Figure 4: Distribution of ITPS with normal curve.
Tests of Normality
To confirm the ordinal survey data could be interpreted as interval data for purposes of correlation analysis, frequency distributions of the responses for the ITPCx, ITPCn, and ITPS constructs were compared with the normal distribution using the Kolmogorov-Smirnov and Shapiro-Wilk tests. Results of the analysis (see Table 6) indicated that only the distribution for ITPS was significantly different from the normal distribution; therefore, a normal transformation using the ranking method was applied to create a normally distributed transform NITPS (Solomon, 2008).
Table 6: Tests of normality for ITPCx, ITPCn, ITPS, and NITPS (df = 235).
| Kolmogorov-Smirnova | Shapiro-Wilk | |||
| Construct | K-S | p | W | p |
| ITPCx | .053 | >.200 | .991 | .165 |
| ITPCn | .039 | >.200 | .993 | .348 |
| ITPS | .115 | .000 | .931 | .000 |
| NITPS | .042 | >.200 | .991 | .165 |
Note. aLilliefors significance correction.
The NITPS transform displayed graphical evidence of a normal distribution (see Figure 5), which was confirmed by the results of the K-S and S-W tests (see Table 6), indicating the distribution of NITPS was not significantly different from the normal distribution, with p > .200 and p = .165, respectively.
Figure 5: Distribution of NITPS with normal curve.
As further confirmation of the tests of normality, summary statistics were also determined for the distribution of the transform. The values for NITPS indicated minor left skewness = -0.09 and platykurtosis = -0.26, with both values lower than those for either ITPCx or ITPCn (see Table 7).
Table 7: Summary statistics for ITPCx, ITPCn, ITPS, and NITPS (n = 235).
| Construct | M (SD) | 95% CI | Skewness | Kurtosis |
| ITPCx | 2.91 (0.54) | [2.84, 2.98] | 0.15 | -0.30 |
| ITPCn | 3.04 (0.48) | [2.98, 3.10] | 0.24 | -0.27 |
| ITPS | 3.92 (0.78) | [3.82, 4.02] | -1.00 | 0.94 |
| NITPS | -0.01 (0.96) | [-0.13, 0.12] | -0.09 | -0.26 |
Construct Correlations
Substituting the NITPS normal transform for ITPS, Pearson's chi-square crosstab analyses were performed to confirm the existence of statistically significant relationships among the constructs. All three construct pairs demonstrated χ2 results withp < .001, with χ2 (6, N = 235) = 57.85 for ITPCx-ITPCn; χ2 (15, N = 235) = 61.71 for ITPCx-NITPS; and χ2 (10, N = 235) = 40.01 for ITPCn-NITPS.
The relationships among the constructs were then evaluated using both parametric and nonparametric correlation analyses. Pearson's correlation coefficients (r) were derived for the relationships between the independent variable constructs ITPCx and ITPCn, and between each independent variable construct and the dependent construct's NITPS normal transform. To confirm the validity of the parametric correlations with the normal transform, nonparametric Kendall's taub and Spearman's rho correlations were also calculated between the independent variable constructs and the dependent ITPS construct (see Table 8).
Table 8: Parametric and nonparametric correlations between paired constructs.
| Pearson's r | Kendall's taub | Spearman's rho | |||||
| Paired constructs | r | r2 | p | τ | p | rs | p |
| ITPCx-ITPCn | .530 | .281 | .000 | .338 | .000 | .483 | .000 |
| ITPCx-ITPSa | -.350 | .123 | .000 | -.256 | .000 | -.363 | .000 |
| ITPCn-ITPSa | -.228 | .052 | .000 | -.123 | .006 | -.181 | .005 |
Note. aParametric (Pearson's) correlations with ITPS were performed using the NITPS normal transform.
The correlation analysis identified statistically significant parametric correlations with p < .001 and nonparametric correlations with p < .01 between all construct pairs. Results were then evaluated for each individual research question.
Research Question 1: ITPCx and ITPCn
Results of the correlation analysis indicated the two independent variable constructs, ITPCx and ITPCn, were positively correlated. The Pearson's correlation (r = .530, r2 = .281, p < .001) and the nonparametric Kendall's (τ = .338, p < .001) and Spearman's (rs = .483, p < .001) coefficients all indicated positive correlations of medium to large effect size (Cohen, 1988, 1992). The null hypothesis H10 was therefore rejected; IT project complexity was positively correlated with IT project complication.
A scatter plot of the relationship between ITPCx and ITPCn with derived regression line (see Figure 6) illustrates the positive correlation between the variables but also highlights the high level of heteroscedasticity. Analysis of the response data for potential outliers identified none for ITPCx and one for ITPCn, but further review of the individual case justified the extreme value. As a result of the wide variability among the data points, no further regression analysis was performed.
Figure 6: Scatter plot of ITPCx vs. ITPCn with derived regression line.
Research Question 2: ITPCx and ITPS
Results of the correlation analysis also indicated that ITPCx and the dependent variable construct ITPS were negative correlated. For this relationship, the parametric (r = -.350, r2 = . 123, p < .001) and nonparametric (τ = -.256, p < .001 and rs = -.363, p < .001) correlation coefficients all indicated negative correlations of small to medium effect size (Cohen, 1988, 1992). The null hypothesis H20 was therefore also rejected; IT project complexity was negatively correlated with IT project success.
The scatter plot for the relationship between ITPCx and the normal transform NITPS with derived regression line (see Figure 7) shows the negative correlation between the constructs, but also reveals extensive heteroscedasticity. One potential outlier was identified for ITPS, but the data values were reviewed and confirmed as justified. No further regression analysis was performed.
Figure 7: Scatter plot of ITPCx vs. NITPS with derived regression line.
Research Question 3: ITPCn and ITPS
Results also confirmed ITPCn and ITPS were negatively correlated. Parametric (r = -.228, r2 = .052, p < .001) and nonparametric (τ = -.123,p < .01 and rs = -.181,p < .01) correlations were statistically significant but weak, exhibiting the smallest effect size among the construct relationships. The null hypothesis H30 was rejected; IT project complication was negatively correlated with IT project success.
The small effect size is graphically depicted in the scatter plot of the relationship between ITPCn and the normal transform, NITPS (see Figure 8). The derived regression line displays a smaller negative slope than that shown between ITPCx and ITPS. The relationship reflects a similar high degree of heteroscedasticity, however. One potential outlier each was identified for both ITPCn and ITPS, but data values were justified and no additional regression analysis was performed.
Figure 8: Scatter plot of ITPCn vs. NITPS with derived regression line.
Research Question 4: ITPCx and ITPS vs. ITPCn and ITPS
Finally, correlation analysis results indicated ITPCx and ITPS were more strongly negatively correlated than were ITPCn and ITPS. IT project complexity and IT project success had a greater negative Pearson's correlation (r = -.350, r2 = .123, p < .001) than did IT project complication and IT project success (r = -.228, r2 = .052, p < .001). In addition, IT project complexity and IT project success also had greater negative Kendall's (τ = -.256, p < .001) and Spearman's (rs = -.363, p < .001) nonparametric correlations than did IT project complication and IT project success (τ = -.123, p < .01 and rs = -.181, p < .01). The null hypothesis H40 was therefore rejected; IT project complexity had a greater negative correlation with IT project success than did IT project complication.
Although the positive correlation between ITPCx and ITPCn, and the negative correlations between ITPCx and ITPS and between ITPCn and ITPS were confirmed as hypothesized, the disparity in the strengths of the correlations was unexpected. Using both parametric and nonparametric methods, the correlation between the two independent variable constructs ITPCx and ITPCn was found to be stronger than the correlations between either independent variable construct ITPCx or ITPCn and the dependent variable construct ITPS.
Conclusions
The study results confirmed all of the original hypotheses of a positive correlation between IT project complexity and IT project complication, a negative correlation between IT project complexity and IT project success, and a negative correlation between IT project complication and IT project success. In addition, the results confirmed the hypothesis that the negative correlation between IT project complexity and IT project success was greater than that between IT project complication and IT project success.
The unexpected finding of a stronger relationship between the independent variable constructs than between either of the independent variable constructs and the dependent variable construct, has several potential explanations. The variable constructs, elements, and factors were synthesized from prior studies, but it is possible that factors and elements of complexity and complication were comingled, thus resulting in a stronger correlation than would have occurred if the factors and elements had been aggregated differently. There may also have been negative intra-scale correlations among some of the factors and elements, as well as other undetected moderating or confounding variables.
In addition, the variance or heteroscedasticity of the relationships indicated that IT project complexity and IT project complication were related but widely varying project characteristics. These results confirmed that IT projects can exhibit varying degrees of complexity and complication, depending on how they are initiated, structured, planned, constrained, staffed, and executed. Practical implications include the importance of recognizing and differentiating between complexity and complication and selecting appropriate project management methods and project success criteria.
Figure 9: Conceptual model with Pearson's correlation coefficients.
Recommendations for Further Study
The study confirmed a distinction between IT project complexity and IT project complication. Further research appears warranted to refine the elements and factors of both constructs and to extend the model to other types of projects. In addition, there are several opportunities for improving on the study methods and outcomes.
The study population was selected because it provided direct access to IT project management practitioners who had demonstrated reliable participation in similar prior studies. The self-selecting nature of a professional organization, however, may have limited generalizability to the target population. In addition, the study restriction to U.S.-based IT project managers complicated the sampling strategy, and comparison with prior studies did not yield useful insights beyond the variations in response rates. A future study with a more representative and global population could enhance generalizability of the findings.
There were also opportunities to improve some of the survey questions. Scales used to measure IT project success were based on prior studies but resulted in a non-normal distribution of responses. Although the scores were transformed, a more balanced set of ordinal category responses for positive and negative project outcomes could have yielded a more normally distributed range of responses, improving the reliability of the correlation analysis without requiring extensive data transformation.
Further factor analysis or structural equation modeling could also help reduce the heteroscedasticity evident in the construct relationships and identify a reduced set of elements and factors, enabling further regression analysis and analysis of variance to refine the study relationships. Identification of moderating and confounding variables could also help determine whether the relationship between project complexity and project success is primary or secondary.
In order to facilitate comparison with prior results, the project success criteria used in this study were similar to the triple constraint criteria used in the Standish Group studies. Future studies with a broader range of IT project success criteria could be beneficial in improving the understanding of the relationship between IT project complexity and IT project success. Broadening the definition of success to include criteria such as project quality, project contribution, business value, and customer satisfaction could help identify areas of interest for further research. There is also an opportunity to expand the usefulness of the study by investigating the effects of systems development and project management life cycles used to manage IT projects with varying degrees of complexity.
Even the relatively general findings of this study indicate it may be helpful for practitioners to incorporate an assessment of project complexity into the initiation, planning, and execution phases of all IT projects. Wherever possible, proactive steps to mitigate the likely effects of IT project complexity may help improve the likelihood of IT project success. Further research into the relationships among individual factors and elements will help clarify these relationships.
This study was based on the premise that there is a measurable difference between project complication and project complexity (Cilliers, 1998). Information technology projects are among the most complex projects typically encountered (Benbya & McKelvey, 2006; Boisot, 2006). Complicated projects can be managed effectively with traditional rational systems-based management approaches (Fayol, 1949; Taylor, 1919). Complex projects, however, can behave in unpredictable and uncontrollable ways (Gabriel, 1998; Lorenz, 1972). Most existing project management theories and methods are based on the rational systems view. A significant opportunity exists for further integration of complex adaptive systems theory into general project management practice (Jaafari, 2003).
