The impact of the project management office on Information Technology Project J
The rate of failed and challenged information technology (IT) projects is too high, according to the CHAOS Studies by the Standish Group and the literature on project management (Standish Group, 2008). The CHAOS Studies define project success as meeting the triple constraints of scope, time, and cost. Assessing critical success factors is another way of measuring project success (Dai, 2001). A proposed solution for improving the success rates of projects is implementing a project management office (PMO), sometimes referred to as a program management office.
The purpose of this study was to gather data to determine if the presence of a PMO improves IT project success based on the triple constraints and critical success factors. The study also considered what impact the type of PMO may have on IT project success.
Data were collected about IT projects, including success criteria and the types of PMO services. The data were collected via a web-based survey from members of the Project Management Institute's (PMI) Information Systems Special Interest Group (ISSIG) who work on IT projects. Having dedicated resources in an organization performing PMO functions was found to have a significant impact on IT project success.
The Background of PMOs
The Value of Project Management and Project Management Maturity
The CHAOS study from 2006 indicated that 35% of projects were successful, 19% of projects failed, and the remaining 46% were considered challenged (Standish Group, 2008). The high rate of failed or challenged projects could be mitigated by increasing the project management maturity level in an organization (Ibbs and Reginato, 2002). The Center for Business Practices defines project management maturity as “the progressive development of an enterprise-wide project management approach, methodology, strategy, and decision making process” (2007, p. 10). Project management maturity is achieved through enterprise-wide project management processes including training, support, and project prioritization in alignment with the organization's corporate strategy (Pennypacker & Grant, 2003). Organizational project management maturity has been found to correlate with improved project management success (Yazici, 2009; Wheatley, 2007; Ibbs & Reginato, 2002).
A Guide to the Project Management Body of Knowledge (PMBOK® Guide) defines a PMO as “an organizational body or entity assigned various responsibilities related to the centralized and coordinated management of those projects under its domain. The responsibilities of a PMO can range from providing project management support functions to actually being responsible for the direct management of a project” (2008, p.443).
A PMO can perform many functions, including creating a project management culture, improving project management maturity, implementing a project management methodology, project portfolio management, training, monitoring, and reporting project status (Block & Frame, 1998; Dai, 2001; Crawford, 2002; Charavat, 2003; Kerzner, 2003; Levine, 2005; J.K. Crawford, 2006; L. Crawford, 2006; Levatec, 2007). These functions can contribute to improved IT project success, leading to reduced costs, increased profits, and improved quality.
Hobbs and Aubry (2008) found that 54% of PMOs were created in the last two years, with only 16% being older than five years. Hobbs and Aubry (2007) found the most important functions of a PMO to be reporting project status to upper management and developing/implementing a standard methodology. Kerzner (2003) found the key benefits of a PMO to include standardization of operations, better resource allocation, quicker access to high-quality information, and reduction of company silos. A PMO with an enterprise project management system can provide early warnings for corrective action (Stewart & Kingsberry, 2003). PMOs can also provide project management training (McDonald, 2001) and a project management community of practice (Thamhain, 2004).
New Developments with PMOs
PMOs are supporting new trends in IT such as project portfolio management, agile methods, management of change, retrospectives and lessons learned, and virtual teams and collaboration. PMOs can support the use of an enterprise project management system for the reporting utilized in managing IT investments in projects as a portfolio (Levine, 2005). PMOs can help ensure that agile methods are being properly utilized (Augustine & Cuellar, 2006). PMOs can ensure the management change levers of communication, training, and incentive/rewards are understood and utilized to provide the organizational change required for an IT project to be successful. PMOs can support project retrospectives at the ends of projects as a way of gathering lessons learned and developing best practices (Kerth, 2001; Stewart, 2004).The impact of 9/11, increased globalization, mergers and acquisitions, outsourcing, and telecommuting have increased the need for virtual project teams (Stewart, 2006, 2008). PMOs can play a key role in supporting these virtual project teams (Curlee, 2008) with project management collaboration tools.
The primary question addressed by this study is:
To what extent does the existence of a PMO contribute to reported IT project success?
In addition, the following subsidiary questions are asked in support of the primary question:
Question 1: To what extent does PMP certification influence project success in organizations with a PMO and organizations without a PMO?
Question 2: To what extent does the project manager's education level influence project success in organizations with a PMO and organizations without a PMO?
Question 3: To what extent does project size influence project success in organizations with a PMO and organizations without a PMO?
Question 4: To what extent does the type of industry influence project success in organizations with a PMO and organizations without a PMO?
Methodology for Data Collection
A web-based survey was utilized to gather the data for this study. The types of questions included respondent demographic, the selected project information, success criteria, PMO information, and project environment factors for PMO functions. The questions were closed ended, with most rated on a Likert scale, ranging from strongly disagree (1) to strongly agree (7).
The survey instrument was initially validated by a pilot study of 20 respondents. An e-mail was sent to all 13,358 PMI Information Systems Special Interest Group (ISSIG) members for the actual survey. Of the 456 responses, 317 were completed and loaded into the Statistical Package for the Social Sciences (SPSS) for data analysis. Tests included sample t-test, one-way, ANOVA, MANOVA, and univariate F-tests.
Data Analysis and Results
Description of Participants
Of the 317 participants 76.0% (n = 241) were PMP certified and 95.6% were college educated (Bachelor's [n = 153, 48.3%], Masters [n = 139, 43.8%], and Doctorate [n = 11, 3.4%]). Eighty-five point two percent of the projects described by the participants had budgets less than or equal to $10 million, with $100,000 to $1 million (n = 108, 34.1%) and $1 million to $10 million (n = 124, 39.1%) the most frequent ranges. While all of the projects were IT projects, the most common industries represented were government (n = 55, 17.4%), health care related (n = 42, 13.3%), computers/information technology (n = 38, 12.0%), manufacturing (n = 35, 11.0%), and others (n = 122, 38.5%).
Project Performance Criteria
Project performance was measured in 14 different areas using the Likert scale, ranging from strongly disagree (1) to strongly agree (7). Exhibit 1 shows the statistical results for the 14 areas. Overall performance was used in the data analysis as the measurement for IT project success.
Exhibit 1: Project performance statistics.
PMO Descriptive Statistics
Exhibit 2 shows that 49.8% (n = 158) of the organizations had a formal PMO. PMO functions were performed by dedicated employees in 15.8% (n = 50) and by part-time resources in 16.1% (n = 51) of the organizations. Six point six percent (n = 21) of the organizations have plans to implement a PMO in the future, whereas in the remaining 11.7% (n = 37) of organizations, no one performs PMO functions and there are no plans to do so.
Exhibit 2: PMO level.
Project Environmental Factors
The project environmental factors are critical success factors for functions that are performed and supported by a PMO (Dai, 2001). The environmental factors were grouped into the six areas shown in Exhibit 3. Each of the six areas consisted of five specific functions, which were measured by questions on the Likert scale, ranging from strongly disagree (1) to strongly agree (7)
Exhibit 3: Project environmental factors statistics.
Responses to Research Questions
For the primary research question of “to what extent does the existence of a PMO contribute to reported IT project success,” Exhibit 4 shows the mean score of overall performance. Projects in which PMO functions are performed by dedicated employees (but not a formal PMO) had the highest mean at 6.20 (sd = 1.081), followed by a formal PMO at 6.13 (sd = 1.125). Projects in which there are no plans for a PMO had the lowest mean at 5.41 (sd = 1.343), followed by projects in which PMO functions were only performed by part-time resources at 5.96 (1.125). Exhibit 4 also shows the higher the PMO level, the lower the value for standard deviation.
Exhibit 4: PMO level and overall performance mean.
Exhibit 5 shows the means for overall performance between those organizations that have dedicated employees performing PMO functions and those that do not. When formal PMOs were combined with organizations that have dedicated employees performing PMO functions the mean was 6.14 (sd = 1.089) compared with 5.80 (sd = 1.339) for projects in which there are no employees dedicated to performing PMO functions.
Exhibit 5: PMO functions and overall performance mean (dedicated employees).
Exhibit 6 shows the results from one-way ANOVA tests for overall performance for analysis of variances for each of the variables of the primary and subsidiary research questions. PMO level (F = 3.206, sig. = .013) was significant in influencing overall performance, but having a formal PMO (F = 2.92, sig. = .131) compared with the other four PMO levels was not significant in influencing overall performance. When comparing the two levels with dedicated employees performing PMO functions (F = 6.143, sig. = .014) with the three levels without dedicated employees, the PMOs with dedicated employees were significant in influencing overall performance. For data analysis purposes, participants have been regrouped into two PMO types: those with dedicated employees performing PMO functions and those without dedicated employees performing PMO functions.
Exhibit 6: One-way ANOVA with overall performance.
Subsidiary question 1: The overall performance mean for those who were PMP certified was slightly lower than those who were not certified, with the value of 6.01 (sd = 1.135) to 6.07 (sd = 1.360). PMP certification was found to not be statistically significant (F = .116, sig. = .734) in influencing the overall performance.
Subsidiary question 2, addressing the influence of education level of the respondent, the mean did not show a consistent pattern (Exhibit 7). Only bachelor's (n = 153) and master's (n = 139) degrees had sufficient sample sizes, with means of 6.16 (sd =1.014) and 5.86 (sd =1.289), respectively, for overall performance. Education level was not significant (F = 1.301, sig. = .270) in influencing overall performance.
Subsidiary question 3, the influence of project budge size in US$; the mean also did not show a consistent pattern. Projects that were less than $100,000 (n = 38) had a mean of 5.79 (sd = 1.339), followed by $100,000 to $1 million (n = 108) with a mean of 6.14 (sd = 1.045), $1 million to $10 million (n = 124) with a mean of 6.05 (sd = 1.222), $10 million to $50 million (n = 25) with a mean of 6.24 (sd = 0.831), and greater than $50 million (n = 22) with a mean of 5.50 (sd = 1.596). Project size in US$ was not significant in influencing overall performance (F = 1.927, sig. = .106).
Subsidiary question 4, the influence of industry; the mean for overall performance ranged from 5.00 to 6.33. For the industries with a sufficient sample size of 20 or more, other industries (n = 122) with a mean of 6.17 (sd = 1.264) was the highest followed by manufacturing (n = 39) 6.09 (sd = 0.919), computers/IT (n = 38) 6.03 (sd = 1.1197), health care related (n = 42) 6.00 (sd = 1.269), and government (n = 55) 5.80 (sd = 1.458). Industry was not statistically significant in influencing overall performance (F = .982, sig. = .450).
Exhibit 7: Subsidiary question factor overall performance mean.
The results of a UNIANOVA of overall performance for the variables of the four subsidiary questions and the dedicated employees performing PMO functions of the primary research question are shown in Exhibit 8. For PMP-certified employees, the mean score was .20 higher for those with dedicated PMO employees. For non-PMP-certified employees, the gap was even larger, with .81 higher for those with dedicated PMO employees.
The same results on overall performance can be found for all of the other subsidiary questions in which there are sample sizes of 20 for organizations with dedicated employees performing PMO functions and those without dedicated employees performing PMO functions. For the education levels of bachelor's and master's, the overall performance means are .17 and .61 higher, respectively, for dedicated employees performing PMO functions. For project size, the mean increases .22 for projects in the $100,000 to $1 million range and .37 for the $1 million to $10 million range. For the government industry it increases .63, and .37 for those who chose “other” for their industry. Therefore, when there is a sufficient sample size for the values of the subsidiary question variables, those projects with dedicated employees performing PMO functions always had a higher mean for overall performance than those projects without dedicated employees performing PMO functions.
Exhibit 8: UNIANOVA of overall performance.
9 shows the results of a MANOVA for overall performance with the project environmental factors compared with dedicated employees performing PMO functions and those without. For project management standards and methods all five factors were found to be statistically significant, including: SM1 (assistance was provided in developing project proposal) (F = 6.068, sig. = .014), SM2 (methods of change requests were available) (F = 11.126, sig. = .001), SM3 (risk assessment procedures were established) (F = 4.114, sig. = .043), SM4 (documentation standards [progress/status reports, and time sheets, etc.] were used) (F = 5.861, sig. = .016), and SM5 (project closeout process was used) (F = 8.358, sig. = .004).
Project historical archives had four of the five factors that were found to be statistically significant, including: HA1 (information on changes to project plans from prior projects were readily available) (F = 4.510, sig = .034), HA2 (risk management documents from prior projects were readily available) (F = 6.337, sig = .012), HA3 (variance analysis [plan vs. actual] from prior projects were readily available) (F = .5.259, sig. = .022), and HA5 (a database of lessons learned was available) (F= 14.640, Sig. = .000).
Project administrative support had four of the five factors that were found to be statistically significant, including: AS1 (administrative staff meet regularly with project team members to ensure a project binder/website was kept up to date) (F = 4.510, sig. = .034), AS2 (assistance was provided to help document project results in standard formats as the project was carried out) (F = 6.337, sig. = .012), AS3 (a project “war room” was made available where participants could store working documents and conduct meetings) (F = 5.259, sig. = .022), and AS5 (project management software was made available for use) (F = 14.460, sig. = .000).
Human resources/staff assistance had three of the five factors that were found to be statistically significant, including: HR1 (assistance was received in identifying the proper person to manage the project) (F = 6.945, sig. = .009), HR4 (guidelines were received to conduct recruiting for the project staff outside the organization) (F = 4.848, sig. = .028), and HR5 (assistance was received to conduct recruiting for project staff outside the organization) (F = 6.754, sig. = .010).
Exhibit 9: MANOVA of overall performance with environmental factors.
Training had none of the five factors that were found to be statistically significant. For consulting and mentoring, two of the five factors were found to be statistically significant, including: CM1 (assistance to ensure the utilization for relevant project management methodologies was provided) (F = 4.114, sig. = .043) and CM5 (group sharing sessions were convened in person or electronically for project managers (F = 6.594, sig. = .011).
The data gathered support that dedicated resources performing PMO functions were statistically significant and positively impact IT project success. Interestingly, a formal PMO was not found to be significant as compared with the other PMO levels, which seems to indicate that the important factor is dedicated employees performing PMO functions, rather than being formally called a PMO. The work done and support services provided impact IT project success more than the title given to the resources performing those functions. Hobbs and Aubry's (2007, 2008) research has shown that PMOs are constantly evolving, and even the names of those performing PMO functions are varied. There appears to be more agreement in the literature on the project management best practices that would be performed by a PMO, but that could also be performed by other dedicated resources without the formal title of PMO.
As the PMO level increased, the standard deviation for the overall project success decreased. Therefore, in addition to the higher IT project success mean values for dedicated employees performing PMO functions, there was also less variance in success as an organization moved up the PMO levels. This is consistent with the various project management maturity models presented in the literature review. It is suggested that the more formally and consistently a methodology and other processes are followed, the more predictable the project results.
Smith (2005) found that PMP certified project managers had higher levels of project success than non-certified project managers. Godaydin (2007) found that PMP certified project managers outperformed non-certified project managers in the area of risk management. Based on the data of this study, PMP certification was found to not be as important for an IT project's success as having dedicated resources performing PMO functions to support the project. Interestingly, the mean for IT project success for PMP certified project managers at 6.01 (sd = 1.135) was slightly lower than for those not certified at 6.07 (sd = 1.360). Likewise, education level was not as important for an IT project's success as having dedicated resources performing PMO functions.
The 18 project environmental factors that were found statistically significant in influencing IT project success and previously listed in the results section should be considered as best practices. PMOs should confirm they are performing these functions to increase the rate of IT project success. There are 12 other project environmental factors, which were not found to be statistically significant in this study. Based on Dai's (2001) identification of these 12 factors as functions of the PMO presence, and other parts of the literature review, they can still be functions that PMOs need to perform.
Augustine, S., & Cuellar, R. (2006). The lean-agile PMO: Using lean thinking to accelerate project delivery. Cutter Consortium, 7(10), 1–24.
Block, T.R., & Frame, J.D. (1998). The project office. Menlo Park, CA: Crisp Publications.
Center for Business Practices. (2007). The state of the PMO 2007–2008: A benchmark of current business practices. Havertown, PA: Center for Business Practices.
Charavat, J. (2003). Project management methodologies: Selecting, implementing, and supporting methodologies and processes for projects, (1st ed.). Hoboken, NJ: John Wiley & Sons.
Crawford, J.K. (2006). The project management maturity model. Information Systems Management, 23(4), 50–58.
Crawford, J.K. (2002). The strategic project office: A guide to improving organizational performance (1st ed.). New York: Marcel Dekker
Crawford, L. (2006). Developing organizational project management capability: Theory and practice. Project Management Journal, 37(3), 74–86.
Curlee, W. (2008). Modern virtual project management: The effects of a centralized and decentralized project management office. Project Management Journal, 39(S), S83–S96.
Dai, X.C. (2001) The role of the project management office in achieving project success. Ph.D. dissertation, The George Washington University, United States—District of Columbia.
Gokaydin, T. (2009). Correlational study of projects managed by PMP certified and noncertified project managers. Ph.D. dissertation, Capella University, United States—Minnesota.
Hobbs, B., & Aubry, M. (2007). A multi-phase research program investigating project management offices (PMOs): The results of phase 1. Project Management Journal, 38(1), 74–86.
Hobbs, B., & Aubry, M. (2008). An empirically grounded search for a typology of project management offices. Project Management Journal, 39(S), S69–S82.
Ibbs, W., & Reginato, J. (2002). Quantifying the value of project management (1st ed.). Newtown Square, PA: Project Management Institute, Inc.
Kerth, N. (2001). Project retrospectives: A handbook for team reviews. New York: Dorset House Publishing.
Kerzner, H. (2003). Strategic planning for a project office. Project Management Journal, 34(2), 13–25.
Levatec, C.J. (2006). The program management office: Establishing, managing, and growing values of a PMO (1st ed.). Fort Lauderdale, FL: J. Ross Publishing, Inc.
Levine, H. (2005). Project portfolio management: A practical guide to selecting projects, managing portfolios, and maximizing profits (1st ed.). San Francisco: Jossey-Bass.
McDonald, J. (2001). Why is software project management difficult? And what that implies for teaching software project management. Computer Science Education, 11(1), 55.
Pennypacker, J.S. and Grant, K.P. (2003). Project management maturity: An industry benchmark. Project Management Journal, 34(1), 4–11
PMI Standard (2008). A guide to the project management body of knowledge (PMBOK® guide) (4th ed.). Newtown Square, PA: Project Management Institute, Inc.
Smith, D. (2005). PMP certification: Ensuring project success. Retrieved December 8, 2009 from http://www.certmag.com/print.php?in=1281.
Standish Group (2008), The Trends in IT Value, Boston: The Standish Group International, Inc.
Stewart, J.S. (2004), Promoting project management maturity with an enterprise project management methodology. Proceedings of the 2004 PMI North America Global Congress.
Stewart, J.S. (2006), Cross culture project management. Proceedings of the 2006 PMI North America Global Congress.
Stewart, J.S. (2008), Project management collaboration tools. Proceedings of the 2008 PMI North America Global Congress.
Stewart, J.S., & Kingsberry, D. (2003), Implementing a global program management office. Proceedings of the 2003 PMI North America Global Congress.
Thamhain, H.J. (2004). Team leadership effectiveness in technology-based project environments. Project Management Journal, 35(4), 35–46.
Wheatley, M. (2007). Maturity matters. PM Network, 21(7), 48–53.
Yazici, H.J. (2009), The role of project management and organizational culture in perceived performance. Project Management Journal, 40(3), 14–33.
©2010 Jake Stewart
Originally published as part of Proceedings PMI Global Congress 2010 – Washington D.C.