Contribution of individual project participant competencies to project success

Abstract

This paper examines the relationship of overall competency levels of individual project participants with project success. A web-based survey administered to members of the Project Management Institute created a database of 145 cases with eight independent variables: presence of a project management office, education of the project manager, certification level of the project manager, project duration, team size, competency of the most competent project participant, competency of the least competent project participant, and the calculated interaction between those competency levels.

Project managers responded to the survey by identifying the least competent project participant and the most competent project participant from the project they most recently managed to completion. Thirty-two individual competencies created an overall competency rating for each of these two participants. Twelve items created a summated scale with a rating for project success.

The findings show that project individual project participant competences contribute more to project success than traditional indicators of: presence of a project management office, education of the project manager, certification level of the project manager, project duration, and project team size.

While the overall competence level of the most competent project participant and the interaction between the most and least competent project participants are both positively correlated with project success, the overall competence level of the least competent project participant is the largest contributing factor to project success.

Introduction

Project success is generally defined as project completion on schedule, within budget, with delivery of features and functionality as promised to the satisfaction of key project stakeholders. Ask almost any information technology project professional about project success rates, and the answers are abysmal. The Standish Group has been studying project successes and failures since 1989 and has determined that in 2005 only 28% of information technology projects come in on time and on budget (Johnson, 2006, p. 129). Every 100 projects that start create 94 project restarts. In fact, a project may have multiple restarts (Standish Group, 1995, p. 1).

Standish and others (Kappelman, McKeeman, & Zhang 2006; Pinto & Slevin, 1987; Schmidt, Lyytinen, Keil, & Cule, 2001) list a number of factors that contribute to project success or failure; one of the more significant items is project staff competence.

General agreement exists that projects need smart, well-trained people who are motivated to do their jobs; however, limited research addresses what specific performance-based competencies contribute to project success (Skulmoski, 2005).

A search for “competency” in the knowledgebase of the Project Management Institute returned 79 articles. More than 80% of the research that addresses competence focuses on the project manager. Nothing appears to address the importance of competencies in project participants.

This study will demonstrate that project success is a function of the competence of the project participants. More important, the study will also demonstrate the competence level of the least competent participant best predicts project success.

Statement of the Problem

General agreement exists in the literature that individual skills are an important ingredient to overall project success; however, just how important relative to other factors is a matter of opinion. From 4-14% of project success is attributed to staff competency. It ranks from fifth to eleventh on lists of contributing success factors (Johnson, 2006; Kappelman, McKeeman, & Zhang, 2006; Pinto & Slevin, 1987; Schmidt, Lyytinen, Keil, & Cule, 2001). Exactly how individual competencies relate to project success is unknown. Similarly, whether or not the likelihood of project success could be improved by selecting project participants with above-average competencies is also unknown. Project teams are too often composed of “available individuals” rather than handpicked project participants. This research will assist project managers in selecting a project team with the right mix of competencies that will directly increase the likelihood of project success.

Purpose of the Study

The purpose of this study is to analyze the relationship between specific competency measures and specific success measures. Prior studies in this area of project participant competence have been qualitative, but Skulmoski (2005, p. 291) suggests that an analytical approach is warranted.

Although the theory of constraints has been applied to project management by managing the critical path and project plan, this analytical survey study will expand upon and clarify the theory of constraints by applying it to the project team formation. This research will examine the relationship between competency in individual project participants and project success. A web-based survey of members of the Project Management Institute (Minnesota chapter) regarding their project experience provides the data for analysis. The independent variables—competence of individual project participants—are generally defined as scores on a survey assessment instrument completed by the project manager. Least competent and most competent are subjective terms used by the project manager in selecting individuals about whom to complete the assessment instrument. The control variables of project manager experience and education are defined generally by education level and certification by the Project Management Institute.

This research will demonstrate that project success is influenced more by the competencies of the least competent project participant than by those of the most competent project participant. The study will also explore the offsetting effect of the most and least competent participants in relation to project success.

Importance of the Study

This study is the first quantitative study that compares the competencies of project participants with the success of the project itself. Previous research used an anecdotal or theoretical approach to address two related issues: identifying the key factors that contribute to project success and identifying the competencies that are important for project participants.

Competence research has identified those attributes, traits, skills, and characteristics that are unique to a highly competent project participant. By determining how those competencies relate to project success, the results of this study provide a potentially useful framework for project participant selection and team development. Future application of project participant competency include recruiting, selection, placement, assessment, retention, promotion, performance management, succession planning, and compensation of project personnel (Hartman & Skulmoski, 1999, p. 2; Skulmoski, Hartman, & DeMaere, 2000, p. 3; Spencer & Spencer, 1993, p. xii).

Improving the understanding of participant competence should improve project performance and the probability of project success (Skulmoski, Hartman & DeMaere, 2000, p. 10).

Scope of the Study

This study uses a quantitative survey method as the best approach to test the theory of constraints and to what extent it describes the relationship between project participant competence and project success.

Research Questions

The study addresses three primary areas of research. First, it will look at the difference between the most competent and least competent project participant. The survey instrument asked the project manager to identify these participants based on the manager's perception. The data analysis will determine if a statistical difference exists between the competency levels of the two participants identified by the project manager.

Next, the study will determine the best indicator of project success: (1) the competence level of the least competent project participant, (2) the competence level of the most competent project participant, and (3) the combined competence level of the participants.

Finally, the research will look at the offsetting effect of project participant competencies by determining if a highly competent participant compensates for a less competent participant.

Hypotheses

For purposes of answering the questions using analysis of variance and a stepwise regression statistical model, these three research questions translate into the following five null hypotheses:

H1o: There is no difference in the competency ratings of the most competent project participant and the least competent project participant.

H2o: There is no relationship between individual competencies of the least competent project participant and project success.

H3o: There is no relationship between individual competencies of the most competent project participant and project success.

H4o: Low individual competencies are related more closely with project failure than high individual competences are related to project success.

H5o: There is no relationship between the interaction of participant competencies and project success.

Rationale of the Study

Popularized by Eli Goldratt (1990), the theory of constraints indicates that a chain is only as strong as its weakest link.

As applied to this study, the theory of constraints implies that the independent variables, competence of individual project participants, will influence or explain the dependent variable, project success. It also implies that the competence level of the least competent project participant (the weakest link) will have the greatest impact upon project success (the system output). This quantitative study will show that project success is constrained by the competency level of the least competent project participant. A project is only as successful as the overall competence level of its least competent project participant.

Definition of Terms

Competency—One of a set of requirements for an individual to perform a specific job properly. A competence may reflect knowledge, skills, and behavior, or a combination of any or all three. Differentiating competencies are the factors that distinguish superior performers from average performers (Spencer & Spencer, 1993, p. 15).

Managerial Role—A role responsible for creating a project environment that includes goals, incentives, clear relationships within the team, and a clear issue management process. In addition, the managerial role needs to make sure that other participants have the competencies required to thrive in the project environment (Skulmoski, 2005, p. 229).

Mixed Role—The liaison between the project manager and the technical participants. Participants in a mixed role require most of the same competencies as managerial and technical roles, except that they are not as highly honed as either of the other roles (Skulmoski, 2005, p. 228).

Participant Competency Interaction—A term used in this study to describe the combined effect of the most competent project participant working with the least competent project participant. Previous research indicates that the collaborative effort of the entire team counterbalances individual deficiencies (DeFranco-Tommarello, 2002, p. 8).

Project Success—The favorable outcome of a project. In this study, project success includes the combined measures of client satisfaction, product quality, budget, schedule, and team satisfaction (Standish Group, 1995, p. 2).

Technical Role—Project participants who are leaders in their professional discipline. Participants in a technical role focus on the details of the project that are delegated by a project manager. This role is tasked with providing solutions to specific problems (Skulmoski, 2005, p. 227).

Theory of Constraints—A theory publicized by Eli Goldratt based on the premise that the rate of system throughput is limited by at least one constraining process (i.e., a bottleneck). Only by increasing throughput (flow) at the bottleneck process can overall throughput be increased. In order to manage the performance of the system, the limiting constraint must be identified and managed correctly (Goldratt, 1990, 1997; Leach, 2005).

Literature Search

This literature search had three objectives: first, to understand the definition of project success and the factors that contribute to it; second, to understand the competencies required for technical professionals and project participants; and finally, to examine the theory of constraints as a possible theoretical construct to link project participant competence and project success.

Three bodies of knowledge were examined: (1) project success and failure; (2) individual competency in business, management, and information systems; and (3) theoretical constructs linking project participant competency and project success. Two topics that are frequently cited as important to project success were included in the literature search, but are not part of this review—competencies related to project management and the structure of project teams.

Project Success

Success Measures

The most common criteria of project success are completion measurements—on time, within budget, and delivering the specific results (Barner, 2000, p. 193; Johnson, 2006, p. 115; Jorgensen & Molokken-Ostvold, 2006, p. 2; Martin & Webster, 1986, p. 165). These criteria are also known as the triple constraints of project management: cost, schedule, and quality.

Based on the existing literature and interviews with project managers, Pinto and Mantel (1990) isolated three aspects of project outcomes that could be used to assess project failure. These aspects were the implementation processes (including cost and schedule measures), the project value as perceived by the project team, and the client satisfaction with the final deliverable.

The 12 individual criteria from Pinto and Mantel's work formed the basis for the success measurement on the survey instrument in the current study.

Table 1: Pinto and Mantel's Project Failure Criteria

Client Satisfaction Perceived Quality Implementation Processes
The project works.
The project will be used.
The project will benefit its users. The project will benefit its users.
Important clients will use this project.
Start-up problems will be minimal.
This project solves the problems for which it was created. This project solves the problems for which it was created.
This project will lead to improved performance.
This project will have a positive impact.
This project is a definite improvement.
The project was completed on schedule.
The project was completed within budget.
The team was satisfied with the project process.
Source: Pinto & Mantel, 1990, p. 271

Success Factors

The Standish Group, who has been studying project successes and failures since 1989, identified 10 project success factors (Johnson, 2006). These factors are listed in Table 2.

Pinto and Slevin developed another model of project success factors in 1986. Their “Project Implementation Profile” (PIP) includes 10 success factors (Slevin & Pinto, 1986) that closely resemble the factors identified by the Standish Group. The result of this research by Pinto and Slevin was an empirical validation of the conceptual success factors developed by previous authors. Working with Mantel, Pinto later expanded on that research by investigating the relationship between the 10 critical factors, and project failure (Pinto & Mantel, 1990).

Other researchers have compiled similar lists of factors that contribute to project success and failure (Kappelman, McKeeman, & Zhang, 2006; Schmidt et al., 2001). Fowler and Horan (2007, p. 17) identified six key risk areas: sponsorship/ownership, funding and scheduling, personnel and staffing, scope, requirements and relationship management. In the area of personnel and staffing, Tesch, Kloppenborg, and Frolick identified the top rated risk was a lack of “enough staff or those with the right skills” (2007, p. 64).

Table 2: Literature Comparison of Project Success and Failure Factors

Johnson, 2006 Pinto & Slevin, 1987 Kappelman, McKeeman, & Zhang, 2006 Schmidt, Lyytinen, Keil, & Cule, 2001
1. User Involvement 4. Client Consultation—Communication, consultation, and active listening to all impacted parties.
7. Client Acceptance—The act of “selling” the final project to its ultimate users.
5. Project stakeholders have not been interviewed for project requirements.
10. Key project stakeholders do not participate in major review meetings.
2. Failure to gain user commitment
4. Lack of adequate user involvement
2. Executive Support 2. Top Management Support—Willingness of top management to provide necessary resources and authority/power for project success. 1. Lack of top management support or commitment to the project. 1. Lack of top Management commitment to the project.
3. Clear Business Objectives 1. Project Mission—Initial clearly defined goals and general directions. 7. Undefined project success criteria. 7. Changing scope/objectives.
4. Scope Optimization 2. Functional, performance, and reliability requirements and scope are not documented. 3. Misunderstanding the requirements.
5. Agile Processes 8. Monitoring and Feedback—Timely provision of comprehensive control information at each stage of the implementation process. 6. Lack of frozen requirements.
6. Project Management Expertise 9. Communication
10. Trouble-shooting—Ability to handle unexpected crises and deviations from plan.
11. Conflict between user departments.
7. Financial Management
8. Skilled Resources 5. Personnel—Recruitment, selection, and training of the necessary personnel for the project team. 8. Project team members have weak commitment to the project scope and schedule.
11. Project team members do not have required knowledge/skills.
5. Lack of required knowledge/skills in the project personnel.
8. Introduction of new technology.
10. Insufficient or inappropriate staffing.
9. Formal Methodology 3. Project Schedule/Plan—A detailed specification of the individual action steps for project implementation. 4. No change control process.
6. No documented milestone deliverables and due dates.
10. Tools 6. Technical Tasks—Availability of the required technology and expertise to accomplish the specific technical action steps. 9. Communication breakdown among project stakeholders. 7. Failure to manage end user expectations.
Note: Numbers reflect the factor ranking in the original literature.
Sources: Johnson (2006), Pinto & Slevin (1987), Kappelman et al. (2006), Schmidt et al. (2001)

Competencies

Kelly

Through 10 years of research and training in organizations like Bell Labs, 3M, and Hewlett-Packard, Robert E. Kelley proved that being a top performer has little correlation with intelligence, creativity, social skills, or aspirations. “Once you have passed the cognitive hurdle, having more cognitive ability than the requisite amount does not seem to yield star performer benefits” (Kelley, 1998, p. 28).

According to Kelley, “about 10-15% of all people will outperform their peers by a wide margin and rise into the star ranks” (Kelley, 1998, p. xvii). These represent the employees whose productivity is about 10 times the average worker (Kelley, 1998, p. xviii). What distinguishes star performers are the methods they use to complete their own work and to work successfully with others. These methods allow top performers to double their productivity without increasing their work effort (Kelley, 1998, p. 15).

As a result of the observations and interviews, Kelley's team compiled a list of nine work strategies used by top performers to increase productivity.

Spencer and Spencer

Spencer and Spencer's Competence at Work summarizes 286 job competency studies using the job competency assessment methodology in a variety of roles: entrepreneurial, technical and professional, sales, human service, and managerial. These studies represent more than 650 jobs in a variety of industries: government, military, healthcare, education, and religious organizations (Spencer & Spencer, 1993, p. ix).

The criteria used most frequently in competency studies are:

Superior Performance: “Defined statistically as one standard deviation above average performance, roughly the level achieved by the top one person out of 10 in a given working situation” (Spencer & Spencer, 1993, p. 13).

Effective Performance: “Usually really means a ‘minimally acceptable’ level of work, the lower cutoff point below which an employee would not be considered competent to do the job” (Spencer & Spencer, 1993, p. 13).

Spencer and Spencer created generic models for a number of positions, including technical professionals—“individual contributors whose work involved the use of technical (as opposed to human services) knowledge. Jobs studied include software developers, engineers, applied research assistants, and a technical job in a bank trust department” (1993, p. 161). Twelve competencies were identified that distinguish between superior and average technical professionals (Spencer & Spencer, 1993, p. 163).

Skulmoski

In the past, emphasis of project competence research has been on project management skills and effectiveness. Soft competencies, like traits and behavior are studied less frequently. Likewise, most project competence research has focused on the project manager, and neglected the project team.

In Project Participant Competence, Skulmoski studied the importance of competencies as they relate to roles on the project. His premise was that research into understanding the competencies of project personnel by different job functions might provide useful insight into the competencies required for successful projects (Skulmoski, 2005, p. 39).

A key element of Skulmoski's research question was the assumption that project participant competencies changed by project role. For a project to be successful, a project manager requires a different set of competencies than other project participants.

Skulmoski's Delphi participants identified 59 different project roles (2005, p. 140) that formed a continuum from purely technical to purely managerial roles (Skulmoski, 2005, p. 7). Instead of researching by job function, Skulmoski structured his competency survey by role—technical, managerial, or hybrid—because job functions may vary significantly between organizations and projects (Skulmoski, 2005, p. 136). How traits, motives, attitudes, and behaviors are related to project performance or project success was unknown for any of the project roles.

Skulmoski's Delphi survey resulted in the following list of competencies for technical and hybrid project roles (see Table 3).

Table 3: Competency Ratings for Hybrid Role

Competency Group Competency Importance to Hybrid Role Importance to Technical Role
Communication Effective Questioning/Generate Feedback 7% 2%
Communication Open Communication 5% 4%
General/Project Management Skills Project Management Skills and Knowledge 5%
Professional Conduct Technical Skills/Theoretical Knowledge 4% 7%
Personal Attributes Flexibility/Deal With Ambiguity 4% 5%
Leadership Leadership/Decisiveness 4%
Negotiation Skills Consensus Building 4%
Professional Conduct Pride in Workmanship/Quality/Craftsmanship 3% 5%
Personal Attributes Analytical/Eye for Details 3% 4%
Leadership Motivate Self and Others 3% 3%
Professional Conduct Professional Conduct 3% 3%
Leadership Create an Effective Environment 3% 2%
Negotiation Skills Conflict/Dispute Resolution 3% 1%
Leadership Objectivity 3%
Personal Attributes Problem Solving/Solution Oriented 2% 3%
Social Skills Truthful/Honest 2% 3%
Personal Attributes Energetic/Committed/Focused 2% 2%
General/Project Management Skills Manage Expectations 2%
Personal Attributes High Level Perspective 1%
Personal Attributes Judgment 1%
Communication Listening Skills 7%
Leadership Ownership of Tasks 5%
Personal Attributes Confident/Realistic 3%
Social Skills Ability to Get Along/Team Player 3%
Communication Verbal Skills 2%
Professional Conduct Responsible/Results Oriented 2%
Professional Conduct Participate and Contribute Fully to the Project-Self and Facilitate Others 2%
General/Project Management Skills Issue Formulation 2%
Personal Attributes Creativity/Innovative/Resourceful 2%
Personal Attributes Self-organization/Self-directed 2%
Personal Attributes Ability to Learn/Self-Evaluation 2%
Negotiation Skills Compromise 2%
Other Other (40 competencies) 36%
Other Other (35 competencies) 22%
Source: Skulmoski, 2005, p. 197-200

The rank ordered importance of the competencies changes when examining the individual competencies instead of the competency categories, and when considering the project role. For a technical role, both Personal Attributes and Professional Conduct contribute more to project success than Communication or Leadership. In a hybrid role, Personal Attributes and Leadership contribute more to project success than Communication.

While Skulmoski groups the competencies into groups differently from Spencer and Spencer or Kelley, the underlying behaviors remain the same, and the models bear a strong resemblance to each other. Of the 32 named competencies identified as important to project success for technical and hybrid roles, only nine do not match exactly to behavior described by Spencer and Spencer. Twenty-six competencies match across the two studies, identified by both Skulmoski (2005) and Spencer and Spencer (1993).

Table 4: Literature Comparison of Project Participant Skills

Harper & Harper, 1992, p. 45-46 Spencer & Spencer, 1993 Kelley, 1998 Fisher & Fisher, 1998, p. 227 Kerzner, 2003, p. 333 Skulmoski, 2005 Relkin, 2006
Supports team goals Achievement Orientation Initiative Customer Advocate Self-starter ability Communication skills Understands new technologies
Puts team goals ahead of individual goals Impact and Influence Knowing who knows Trainer Work without supervision Leadership Skills Designing technical architecture
Listens to everyone on the team Conceptual Thinking Self Management Resource – expanding personal knowledge Good communication skills Professional Conduct Integrating systems
Both task and team focused Analytical Thinking Perspective Skilled Worker – demonstrate technical skills Cooperative Project Management Skills Understands business practices, approaches, organization, politics and culture
Recognizes conflict as useful and necessary Initiative Followership Team player Technical understanding Personal Attributes Project Management Skills
Trusts other team members Self Confidence Teamwork Decision maker – assimilate and use information Willing to learn backup skills Negotiation Skills Communicating and listening
Communicates openly and honestly Inter-personal Understanding Small-l Leadership Problem Solver Ability to perform feasibility studies and cost/benefit analyses Social Skills Focuses on results
Values diversity Concern for Order Organizational Savvy Ability to perform or assist in market research studies Thinks strategically
Works toward consensus Information Seeking Show and tell Able to evaluate asset usage Influencing and persuading
Uses others as a resource Teamwork and Cooperation Decision maker Adaptable
Expertise Risk management
Customer Service Orientation Understands the need for continuous validation
Sources: Harper & Harper (1992, p. 45-46), Spencer & Spencer (1993), Fisher & Fisher (1998, p. 227), Kerzner (2003, p. 333), Skulmoski (2005), Relkin (2006)

Application of Participant Competency to Project Success

Three general relationships between competency and success can be examined.

Highest level of participant competence contributes the most to project success:

In a study of 103 marketing and management students divided into study teams, Miles and Mangold (2001, p. 38-39) found the best predictor of team performance is the individuals' prior performance. Miles and Mangold concluded that organizations would obviously want to recruit and hire individuals with a record of prior success and accomplishment. Another study of college athletes confirms the findings and that “the team product will be better, the more capable the average member” (Wright, McMahan, Smart, & McCormick, 1997, p. 4).

In the first study to examine personality factors and team performance, Kichuk and Wiesner noted that “a team's probability of success is contingent upon having members who are each capable of contributing to the task at hand and who work well together” (1998, p. 1-4).

Bass (1980, p. 433) stated that the abilities of group members should influence team performance. He offered as self evident proof that “the team product will be better, the more capable the average member,” and “if the team's performance depends on discovering the right answer, then team performance will be as good as its best member.” Bass' model of team performance notes that the abilities of team members (as measured in part by intelligence tests) affect the task performance of team members, and he argued that this accounts for 50% of the variance in team performance (Wright, McMahan, Smart, & McCormick, 1997, p. 4).

Lowest level of participant competence contributes the most to project success:

According to Avery (2001, p. 96):

The least committed member of your team is the most powerful because his lack of commitment establishes a low baseline to which other team members may fall. The success—or mediocrity—of your team likely will be determined by him.

While studying three-man military crews, Tziner and Eden (1985, p. 85) found significant interaction affects when combining teams of individuals with high ability. Performance by crews where all the members were of high ability far exceeded the level of performance predicted by individual ability. Replacing a crewmember of high ability with one of low ability diminished the crew performance disproportionately (Tziner & Eden, 1985, p. 91).

A combined level of participant competence contributes the most to project success:

Wilson, Hoskin, and Nosek conducted a study to determine if experience with collaboration could benefit beginning programmers when performing problem-solving tasks. The study found evidence that an individual's ability had little overall effect on team performance. The researchers claimed that the collaborative effort of the entire team counterbalanced individual deficiencies (DeFranco-Tommarello, 2002, p. 8).

“A team can't function well unless the members individually function well, and the performance of each person acts as a catalyst to the others. It goes back to the old cliché about the whole being greater than the sum of the parts” (Commander Tom Schibler, US Navy SEALS, quoted in Ankarlo, 1995, p. 16).

Sarah Fister Gale (2007, p. 60) wrote that while some leaders think that recruiting as many experts as they can find or afford is the best approach to creating a project team, the approach is expensive and segregates the lesser competent employees from the knowledge base they need to develop. Gale recommends a tiered system combining an expert with three or four employees of less experience and competence. “That way, each employee can turn to the next person up in the hierarchy before taking the problem all the way to the top” (2007, p. 65).

Theory of Constraints

The theory of constraints was defined and explored by Eli Goldratt in his famous business novel in 1990. According to Goldratt (1990, p. 123): “The strength of a chain is only as strong as its weakest link. There are no two ‘weakest’: the number of constraints is very limited.” Based on the theory of constraints, improving the system performance requires improving the weakest link. An investment in anything other than the weakest link will increase overall costs, but will not increase the output of the entire system. The benefit of the theory is its focus on the output of the entire system, instead of a single task or resource (Dettmer, n.d., p. 3). “Strengthening the already strong links will not increase success unless the weak links are also strengthened” (Kostrich, 2005).

The theory of constraints has special application in the area of project management, called Critical Chain. Critical Chain management urges managers to identify and manage the project system's bottleneck resource (Lechler, Boaz, & Stohr, 2005, p. 48).

Siha (1999, p. 7) bridged the theory of constraints to organizational performance: She asserted, “The general principles of TOC can be applied to improve the performance of service organizations. Since system constraint is at the heart of TOC, the recognition of the nature of organization constraint is the first step toward continuous improvement” (1999, p. 7). This view is supported by Cox Mabin, and Davies (2005, p. 64), who stated, “… this truism applies to human systems as well as it does to technical or mathematical systems.”

Methodology

The purpose of this study is to analyze the correlation between specific competency measures and specific success measures. Skulmoski (2005, p. 291) suggested this approach. “Perhaps with rich data and a substantial period of time a positivist approach might yield insight into the many relationships between project participant competency and project performance/success.”

The 12 individual criteria from Pinto and Mantel's (1990, p. 271) work formed the basis for the success measurement on the survey in the current study. The 32 competencies included in the survey instrument are the individual competencies identified by Skulmoski's (2005) Delphi study as important for project participants in the technical or hybrid role.

The unit of analysis for this research is an individual project. However, a project manager represents each project. The survey instrument specifically requests that the project manager excludes himself or herself from the project participant population when selecting the project participant with the highest and lowest competence level in the attempt to remove bias regarding the project manager as the most competent member of the team.

This study includes three independent variables: the competence profile of the least competent project participant and competence profile of the most competent project participant as measured as the sum of the scales for the participant specific questions in the survey instrument, and the interaction of the participant competence profiles, is measured by the product of the two competency scores. One independent variable, project success, is measured by the sum of the project success scales in the survey instrument.

A t-test was conducted comparing the average competency rating of the high competency project participants and the low competency project participants. Spencer and Spencer (1993, p. 13) noted that superior performers had a competency rating at least one standard deviation higher than that of average performers.

Next, a stepwise regression model was constructed to measure the relationship between the competence measures and the success of the project. Stepwise regression introduces the variables in order of their contribution to the change in the dependent variable. Stepwise regression will continue to introduce the three independent variables (competency of the least competent project participant, competency of the most competent project participant, and competency of the most and least competent participants combined) into the equation until a variable no longer contributes to the variation in project success.

Data Gathering Method

The survey instrument was developed with 86 questions. Eight questions address the project size, team formation, project management experience, and industry. Twelve questions address the success of the project. Thirty-two questions address the competency of the project participants—first regarding the most competent project participant, then regarding the least competent project participant.

The instrument used a 5-point Likert scale to assess each project success and competency question. Zero values were added to the scales to indicate “Not Applicable” so that responses for all items could be required. Summative scales were chosen in the absence of data that shows that certain competencies or success metrics are more meaningful than others.

The survey was self-administered through web-based delivery. Given the technological experience and comfort level of the survey audience, web-delivery was appropriate for this sample group.

Sample Selection

The PMI Minnesota Chapter formed the database for the study. The chapter's Board of Directors directed its communication services organization to email the entire membership (3,088 members) with a request from the chapter president to complete the survey, and a link to the web-based survey instrument, creating anonymity. There were 145 usable cases in the respondent set.

For standard levels of alpha (0.05) and power (0.80), a sample size of 783 cases is needed to detect a small effect, 85 cases to detect a medium effect, and 28 cases to detect a large effect (Field, 2005, p. 34). Statistics researcher and professor, Peter Green suggested a sample size of 104 plus the number of predictors in a model to have sufficient data to test the individual predictors (Field, 2005, p. 173). With 145 cases, the sample size in this study was adequate for testing eight independent variables in the regression equation, and detecting a medium to large effect.

Validity of Data

The individual competencies ranked by Skulmoski (2005) gave a different perspective than the competency groupings in his work. The individual competencies were used in this research to give a more accurate perspective, in spite of the additional length to the survey instrument.

Recall or memory bias can be a problem because the items measured require that subjects recall past events. Often a person recalls positive events with more clarity than negative events (Nardi, 2003, p. 75). Because all respondents answered the same survey questions, and because the survey was self-administered without prompting from an interviewer, the effect of recall or memory bias in individual responses was offset by the response pool overall. In addition, each participant is a professional project manager who has managed multiple projects on a regular and ongoing basis. Every participant was asked to review the last project completed, so that recall or memory bias was minimal.

In the original work on project success factors by Pinto and Mantel (1990, p. 272), a Cronbach alpha score was used to measure the reliability for the 10 critical success factors used in this research as a measure of project success or failure. Scores ranged from .79 to .90, well above an acceptable level (Pinto & Mantel).

Similarly, a Cronbach's alpha was run on the survey instrument for this study to assess internal reliability.

Originality and Limitation of Data

The research instrument in this study is based on the research of Pinto and Slevin (1987), Johnson (2006), and Skulmoski (2005). The Standish Group (Johnson, 2006) research provided a basis for additional variables such as project size, project management expertise, and project length. Pinto and Slevin's (1987) findings were incorporated into the questions regarding project success. Skulmoski's (2005) research provided the foundation regarding critical project participant competencies. Although this foundational research provided a basis for the survey instrument and variables, no other study has surfaced that empirically relate the variables to each other.

The findings of this research are based on data from trained project managers who demonstrate a high level of commitment (PMI-MN members). Characteristics specific to highly committed project management mean caution should be exercised before applying these findings to projects in other disciplines or with less skilled or committed managers.

The survey instrument provided quick and inexpensive data collection from a large number of respondents, facilitating statistical analysis. However, the use of surveys did not allow participants to identify new competencies, interact with the researcher, or explain the nuances of definition (Spencer & Spencer, 1993, p. 101).

Results

Research Findings Related to Literature

The distinguishing characteristic between project participant roles is the amount of technical and managerial responsibilities held by the participant in the subject project. The responsibilities form a continuum, ranging from technical roles to managerial roles.

Project Participant Role Continuum

Figure 1: Project Participant Role Continuum

Source: Skulmoski, 2005, p. 87

Skulmoski (2005, p. 233) indicated that the management role needs a higher level of competency than the technical project participant. The current study indicates that participants in a mixed role demonstrate a greater overall competency level than participants in either technical or managerial roles. However, a competency assessment of the project manager was specifically excluded from this research. “Managerial role” in this context refers to project participants who fulfilled a managerial function on the project, but did not have the title or responsibility of project manager.

Skulmoski (2005, p. 199) also concluded that the mixed role participant needed all of the same competencies as the managerial and technical roles, but to a lesser degree. In this study, the overall competency of participants in a mixed role is greater than the competency level of either the managerial and technical participants, whose competence level is statistically equal. Results indicate that the mean competency of participants in a mixed role is greater than the competency of participants in technical or managerial roles, while the competency levels of technical and managerial roles is statistically the same.

Looking across the roles (technical, managerial, and mixed) Skulmoski (2004) found that the top competency was Project Management Skills and Knowledge, contributing to both the mixed and managerial roles (4% and 7% respectively). Following this is Technical Skills/Theoretical Knowledge, contributing 7% to the effectiveness of the technical role, and 4% to the effectiveness of the mixed role. The findings in the current study support these findings—mixed role participants demonstrate the same level of Project Management Skills and Knowledge as their counterparts in managerial roles, which is greater than this competence level for participants in technical roles. Technical Skills/Theoretical Knowledge appears equal for participants in technical roles and mixed roles, which is greater than this competence in managerial roles. Participants with a mixed role demonstrated a higher rating in both competencies than did their technical or managerial counterparts.

Overall, mixed role participants demonstrated higher technical skills than project management skills F (1,580) = 95.87, p = 0.000. This is to be expected in this study—because the participants in a mixed role were not managing the projects in which their competencies were evaluated. It is not surprising that most participants scored low in project management skills, regardless of the role they fulfilled (managerial, mixed, technical)—their role on the subject project was not project manager, so those skills might not have been present, or if present, they were not demonstrated or needed.

Spencer and Spencer (1993, p. 13) measured superior performance as one standard deviation above-average performance. In this study, the most competent participant group has a mean competency of 163.85, while one standard deviation above the mean is 171.13. Likewise, the least competent participant group has a mean competency rating of 117.84. The most competent project participants in this study do not meet Spencer and Spencer's definition of superior performers. However, the participant rating for this study was subjective and relative. No absolute competency rating is inferred by the opinion of the survey respondent, only that the participants evaluated were considered most competent and least competent relative to their peers for the subject project.

Skulmoski found that business know-how and problem analysis skills distinguish average from superior project participants (Skulmoski, Hartman, & DeMaere, 2000, p. 9). This study supports those findings. There is a statistically significant difference between the most and least competent participant for those competencies.

Statistical t-tests in this study demonstrate significant differences between the most competent participant and the least competent participant on all 32 characteristics used to measure overall competency. This study does not illustrate the concept of threshold or superior competencies as defined by Spencer and Spencer (1993) and Skulmoski et al. (2000).

The significant positive correlation between most, least and interaction, and success illustrates theories that are supported by a number of authors (Ankarlo, 1995; DeFranco-Tommarello, 2002; Doering, 1973; Ellis, 2008; Kichuk & Wiesner, 1998). Kichuk and Wiesner (1998, p. 6) found that “low scores of some individuals can be compensated for by high scores by other team members. DeFranco-Tommarello's (2002, p. 8) research found evidence that “collaborative effort counterbalances individual deficiencies.” IAG Consulting concluded that the competencies of the project team (as opposed to individuals) strongly influence the likelihood that the project will be considered a success by the stakeholders (Ellis, 2008, p. 19).

The data in this study indicates a positive correlation between the competence of the individual participants and the interaction of individual participant competencies and project success. However, with values less than r = 0.25 for all the correlations, the effect size is smaller than typical to typical.

The final outcome from this study was to select the single best predictor of project success from the eight independent variables in the study. The final and best regression model includes only one predictor—the competence level of the least competent project participant. The variable least is the best predicting independent variable. It explains 3.82% of the variance in project success.

These findings corroborate findings in other studies (Avery, 2001; Gale, 2007; Kerzner, 2003).

Kerzner's (2003, p. 631) findings were the same, in that “unacceptable performance by one individual may quickly undermine the performance of the entire team.” Gale (2007, p. 65) explained this phenomenon with the observation, “… more experienced people end up devoting more time answering questions and coaching new employees than they do focusing on project goals.”

Research Findings

Five competencies are significantly correlated with project success for both the most and least competent project participant: Open Communication, Effective Questioning/Generation of Feedback, Participates and Fully Contributes to the Project/Encourages Others, Objectivity, and Technical Skills/Theoretical Knowledge (see Figure 2).

Significantly Correlated Competencies for Most and Least Competent Participants

Figure 2: Significantly Correlated Competencies for Most and Least Competent Participants

Four competencies are significantly correlated with project success for the most competent project participant, but not for the least competent participant: Formulates Issues, Self-Organization/Self-Directed, Analytical/Eye for Details, and Verbal Skills (see Figure 3).

Significantly Correlated Competencies for Most Competent Participants

Figure 3: Significantly Correlated Competencies for Most Competent Participants

Thirteen competencies are significantly correlated with project success for the least competent project participant, but not for the most competent participant (see Figure 4).

Significantly Correlated Competencies for the Least Competent Participants

Figure 4: Significantly Correlated Competencies for the Least Competent Participants

Project Management Skills and Knowledge are not highly correlated with project success in this study; however, the survey specifically excluded Project Managers from the participant pool, so the importance of these skills may be undetected.

Research Findings Related to Research Questions

The study addressed three primary areas of research. For purposes of answering the questions using analysis of variance and a stepwise regression statistical model, the three research questions translated into five null hypotheses.

First, the difference between the most competent and least competent project participant was investigated. The survey instrument asked the project manager to identify these participants based on the manager's perception. Statistical data analysis was used to determine if a statistical difference exists between the competency levels of the two participants identified by the project manager.

H1o: There is no difference in the competency ratings of the most competent project participant and the least competent project participant.

This hypothesis is false. A paired t-test of the most and least competent project participant from each subject project demonstrates a significant statistical difference in the overall competency level of these two participants. Further paired t-tests indicate the difference exists for all 32 individual competencies used to measure overall competency.

Next, the study determined the best indicator of project success: (1) the competence level of the least competent project participant, (2) the competence level of the most competent project participant, or (3) the combined competence level of the participants.

H2o: There is no relationship between individual competencies of the least competent project participant and project success.

This hypothesis is false. There is a significant positive correlation between the competence level of the least competent project participant and project success. While the effect size is smaller than typical to typical, the greater the overall competence of the least competent project participant, the greater the success of the project.

H3o: There is no relationship between individual competencies of the most competent project participant and project success.

This hypothesis is also false. There is a significant positive correlation between the competence level of the most competent project participant and project success. Although the effect size is smaller than typical to typical, the greater the overall competence of the most competent project participant, the greater the success of the project.

The overall competency level of both the most competent project participant and the least competent project participant are highly correlated with project success. As either of these competency levels increase, project success is more likely. However, the correlation between the competence of the least competent project participant and project success is stronger than the correlation between the competence of the most competent project participant and project success.

H4o: Low individual competencies are related more closely with project failure than high individual competences are related to project success.

This hypothesis is true. Using stepwise regression to develop a regression equation with a single predicting variable indicates the competence level of the least competent project participant is the best independent variable to predict project success when selecting from PMO, certification, system, duration, team size, most, least, and interaction.

Finally, the research looked at the offsetting effect of project participant competencies by determining if a highly competent participant compensates for a less competent participant.

H5o: There is no relationship between the interaction of participant competencies and project success.

The hypothesis is false. There is a significant positive correlation between the interaction of the competence level of the most competent project participant and project success with the competence level of the least competent project participant. Although the effect size is smaller than typical to typical, as this interaction increases, the success of the project also increases.

In addition, the interaction variable was orthogonalized to separate the true interaction from the correlation with the most and least variables. This was necessary to allow the interaction term to be used in regression analysis, if it was among the variables selected in best subset or stepwise regression. The interaction variable was not selected as a predicting variable in the stepwise regression analysis.

Conclusions

Discovering the competencies that distinguish superior from average project participants is valuable to the project organization. “When competence is understood by those in the firm, they can use this understanding to guide recruitment, skill assessment and development activities” (Hartman & Skulmoski, 1999, p. 2).

The exact relationship between individual competencies and project success has been variously presented as: (a) success is mostly dependent on the competency of the most competent project participant, (b) success is mostly dependent on the competency of the least competent project participant, or (c) success is dependent upon the combination of participant competencies. The distinguishing characteristic between project participant roles is the amount of technical and managerial responsibilities held by the participant in the subject project.

Five competencies are significantly correlated with project success for both the most and least competent project participant. Four competencies are significantly correlated with project success for the most competent project participant, but the not for the least competent participant. Thirteen competencies are significantly correlated with project success for the least competent project participant but not for the most competent participant.

There is a significant positive correlation between the competence level of the least competent project participant and project success (r = 0.212, p = 0.01). There is a significant positive correlation between the competence level of the most competent project participant and project success (r = 0.193, p = 0.02). The overall competency level of both the most competent project participant and the least competent project participant are highly correlated with project success.

There is a significant positive correlation between the interaction of the competence level of the most competent project participant and the competence level of the least competent project participant with project success (r = 0.252, p = 0.002).

The competency level of the least competent project participant is the best predicting variable for project success; it is the weakest link in the project chain. Although it explains only 3.82% of the variance in project success, improving any other variable represented in this study will only increase the project cost, it will not lead to greater success.

The findings of this research indicated no difference in project success as related to industry, size of the sponsoring organization, presence of a PMO, project manager expertise, project type, project management toolset, project duration, or project team size. Although the competence level of the least competent project participant is the best predictor of project success, there clearly are other variables or combinations of variables that contribute to the variation in project success.

This research focuses on the project participant, not the project manager. The study was about individual project participant competencies and the impact of those competencies on project success. By demonstrating the correlation between the least competent project participant and the success of the projects where those participants are assigned, the research objectives have been achieved. The link between project participant competency and success has been established, and the focus of improvement needs to be on improving the overall competency level of the least competent project participant.

References

Ankarlo, L. (1995). Nine traits of highly successful work teams. Boulder, CO: CareerTrack Publications.

Avery, C. (2001). Teamwork is an individual skill: Getting your work done when sharing responsibility. San Francisco, CA: Berrett-Koehler Publishers, Inc.

Barner, R. W. (2000). Team troubleshooter: How to find and fix team problems. Palo Alto, CA: Davies-Black Publishing.

Bass, B. (1980). Team productivity and individual member competence. Small Group Behavior, 11, 431-504.

Cox, J. F. III, Mabin, V. J., & Davies, J. (2005). A case of personal productivity: Illustrating methodological developments in TOC. Human Resources Management, 24, 39-65.

DeFranco-Tommarello, J. (2002). Literature review of collaborative problem solving and groupware in the software development domain. PhD dissertation, New Jersey Institute of Technology, United States – Newark, NJ. Retrieved November 25, 2007 from NJIT eTD: The New Jersey Institute of Technology's electronic Theses & Dissertations.

Dettmer, H. W. (n.d.). Goldratt's theory of constraints: A system-level approach to continuous improvement. Retrieved January 15, 2007 from http://www.saigon.com/~nguyent/dettmer2.html

Doering, Robert D. (1973, February). An approach toward improving the creative output of scientific teams. IEEE Transactions on Engineering Management, p. 29-31.

Ellis, K. (2008). Business analysis benchmark. Toronto, Ontario: IAG Consulting.

Field, A. (2005). Discovering statistics using SPSS (and sex, drugs and rock'n ’roll) (2nd ed.). London: Sage Publications.

Fisher, K., & Fisher, M. (1998). The distributed mind: Achieving high performance through the collective intelligence of knowledge work teams. New York: AMACOM.

Fowler, J. J., & Horan, P. (2007). Are information systems' success and failure factors related? An exploratory study. Journal of Organization and End User Computing, 19, 1-22.

Gale, S. (2007, January). Mix and match: Combining the talents of more experienced team members with the “newbies” brings out the best in both groups. PM Network, 58-65.

Goldratt, E. (1990). What is this thing called theory of constraints and how should it be implemented? Great Barrington, MA: North River Press.

Goldratt, E. (1997). Critical chain. Great Barrington, MA: North River Press.

Harper, A., & Harper, R. (1992). Skill-building for self-directed team members. New York: MW Corporation.

Hartman, F., & Skulmoski, G. (1999). Quest for team competence. Project Management, 5, 10-15.

Johnson, J. H. (2006). My life is failure: 100 things you should know to be a better project leader. Boston: Standish Group International.

Jorgensen, M., & Molokken-Ostvold, K. J. (2006). How large are software cost overruns? Critical comments on the Standish Group's CHAOS reports. Information and Software Technology, 48(4), 297-301.

Kappelman, L., McKeeman, R., & Zhang, L. (2006). Early warning signs of IT project failure: The dominant dozen. Information Systems Management, 23, 31-36.

Kelley, R. E. (1998). How to be a star at work: Nine breakthrough strategies you need to succeed. New York: Random House.

Kerzner, H. (2003). Project management: A systems approach to planning, scheduling, and controlling (8th ed.). Hoboken, NJ: John Wiley & Sons, Inc.

Kichuk, S., & Wiesner, W. H. (1998). Work teams: Selecting members for optimal performance. Canadian Psychology, 39, 23.

Kostrich, J. M. (2005). An unconventional reef. Retrieved March 21, 2007 from http://www.bestfish.com/unreef.html

Leach, L. (2005). Critical chain project management (2nd ed.). Boston: Artech House.

Lechler, T. G., Boaz, R., & Stohr, E. (2005). Critical chain: A new project management paradigm or old wine in new bottles? Engineering Management Journal, 17(4), 45-58.

Martin, M. D., & Webster, F. (1986). Contract type and the measurement of project success. Project Management Institute Seminar/Symposium (pp. 165-172). Montreal: Project Management Institute.

Miles, S. J., & Mangold, G. (Fall, 2001). An examination of the impact of individual team member characteristics on team performance and member satisfaction. Journal of Business and Public Affairs, 28(2), 34-40.

Nardi, P. M. (2003). Doing survey research: A guide to quantitative methods. (2nd ed.) Boston: Pearson Education, Inc.

Pinto, J. K., & Mantel, S. J. (1990). The causes of project failure. IEEE Transactions on Engineering Management, 37, 269-276.

Pinto, J. K., & Slevin, D. P (1987). Critical factors in successful project implementation. IEEE Transactions on Engineering Management, 34, 22-27.

Relkin, J. (2006, May 10). 10 essential competencies for IT pros. Tech Republic, Retrieved December 13, 2006, from http://articles.techrepublic.com.com/5100-10878_11-6070668.htm

Schmidt, R., Lyytinen, K., Keil, M., & Cule, P. (2001). Identifying software project risks: An international Delphi Study. Journal of Management Information Systems, 17, 5-36.

Siha, S. (1999). A classified model for applying the theory of constraints to service organizations. Managing Service Quality, 9(4), 255-265. Retrieved February 6, 2007 from ProQuest database.

Skulmoski, G. J. (2005). Project participant competence. PhD dissertation, University of Calgary, Canada – Alberta. Retrieved December 15, 2006 from ProQuest Digital Dissertations database. (Publication No. AAT NR05653).

Skulmoski, G., Hartman, F., & DeMaere, R. (2000). Superior and threshold project competencies. Project Management, 6(1), 10-15.

Slevin, D. P., & Pinto, J. K. (1986). The project implementation profile: New tool for project managers. Project Management Journal, XVIII, 57-71.

Spencer, L., & Spencer, S. (1993). Competence at work: Models for superior performance. New York: John Wiley & Sons, Inc.

Standish Group. (1995). The CHAOS Report (1994). Retrieved January 4, 2007, from The Standish Group Web site: http://www.standishgroup.com/sample_research/chaos_1994_1.php

Tesch, D., Kloppenborg, T., & Frolick, M. (2007). IT project risk factors: The project management professional's perspective. The Journal of Computer Information Systems, 47, 61-69.

Tziner, A., & Eden, D. (1985). Effects of crew composition on crew performance: Does the whole equal the sum of its parts? Journal of Applied Psychology, 70, 85-93.

Wright, P. M., McMahan, G. C., Smart, D., & McCormick, B. (1997). Team cognitive ability as a predictor of team performance. Ithaca, NY: Cornell University Center for Advanced Human Resource Studies.

This material has been reproduced with the permission of the copyright owner. Unauthorized reproduction of this material is strictly prohibited. For permission to reproduce this material, please contact PMI or any listed author.

© 2010 Project Management Institute

Advertisement

Advertisement

Related Content

  • PM Network

    Strategic Kick-Start member content locked

    PM Network queries the project management community on ensuring team members have a firm grasp on strategic alignment and key objectives at kickoff.

  • Project Management Journal

    The Dilemmas of Design Thinking in Innovation Projects member content locked

    By Hölzle, Katharina | Rhinow, Holger Organizations have discovered Design Thinking as a promising framework or language for innovation-focused project teamwork. The goal is to develop new products and services by being customer-centric…

  • PM Network

    Get to the Root of the Problem member content locked

    By Smits, Karen Design thinking is all the rage in the business world. An iterative problem-solving approach for designing products and services, it focuses on end-user needs and challenging assumptions. But oddly,…

  • PM Network

    Bridge Builders registered user content locked

    PM Network queried the project management community about keeping virtual teams engaged

Advertisement

Publishing or acceptance of an advertisement is neither a guarantee nor endorsement of the advertiser's product or service. View advertising policy.