Competency assessment

a process for identifying, developing, and assigning high performing project managers

Michael Plumeri, Ph.D.

Introduction

Over the past several years, studies have revealed that a significant number of projects are in a troubled state. They are over budget and behind schedule; they have lost control of the scope and lost focus on deliverables; or they are simply failing to meet the need of the stakeholder. In their annual report of the state of project management, The Standish group reports a persistent level of poor to terrible project performance that costs companies hundreds of millions of dollars.

One of the main reasons for troubled or unsuccessful projects is the lack of qualified, committed project management professionals. In many organizations, employees have very little incentive to assume the position of project manager, largely due to a disconnect surrounding what the role entails. Organizations typically recognize the technical capabilities of individuals, and assume these skills can be translated into project management expertise. Because of this, professionals who have worked for years to earn the title of senior engineer, technical specialist, or technical consultant are unwilling to exchange their current jobs for the role of project manager. The role is added to their regular job description, instead of being viewed as a legitimate function to be valued by the organization, and that requires a special set of skills. Therefore, many organizations still haven’t connected the value of the project manager to the success of the organization.

Why Project Managers Fail: The Case for Professionalization

Research reveals that while more organizations recognize the need to improve their project management initiatives, they have the difficult task of creating a corporate culture that will support the support mechanisms to effectively deploy a project management process.

The performance of project managers varies from project to project; and within any given organization today, there is a wide range of experience in project managers. At one end of the spectrum are project managers who are new to the practice, or who fall into the category of the “accidental project manager”. New project managers are those individuals who are just beginning to practice the art of managing projects. Accidental project managers are appointed to the role because they were seen as having the time to take on a project along with their other work. Both of these types of project managers must struggle to be effective because they lack familiarity with the basic project management concepts. They lose focus on the project’s big picture, and instead get drawn into the minutiae, slowing down progress and damaging team health. At the other end of the spectrum are the experienced project managers. These individuals have either learned to manage projects by the “seat of their pants” or through an ad hoc training program, or through some other means. The challenge is getting a sense of consistency between projects and project managers.

Because there are varying degrees of capability, competence, and confidence among project managers, organizations are experiencing an inconsistency in the quality of how projects are managed. This inconsistency can be traced to a number of factors. One obvious factor is that many organizations are recruiting project managers from other organizations. These project managers have been exposed to a variety of approaches and without fault, apply them to their current effort. Because each manager’s approach is different, it is difficult for organizations to create and maintain a consistent approach to project implementation.

Another factor is the tendency to start every project as if it were brand-new. New project managers are not given time to review previous organizational projects to see the lessons learned, to review possible templates for project planning, or to identify possible areas of reuse in a new project. As a result, the cost of projects escalates, the time to complete them increases, and customer satisfaction is negatively affected.

In terms of resources, most organizations lack a pool of individuals who are trained in the art of managing projects. Instead, they adopt a “just-in-time” mentality when it comes to developing the skills of the project managers. As a result, there is and absence of an approach to identifying and delivering the right training to the right people. Even if employees are given classroom training, they are not given appropriate job assignments to practice those skills. Then they are expected to utilize their training without the benefit of a mentor to guide them once their training is completed.

It’s a recipe for disaster.

With all of that said, what do project managers really mean to the organization? The impact of this level of project performance can be seen in terms of people, clients, and the enterprise. What is needed to counteract these trends or conditions is a bold approach to the professionalization of the role of project manager. Not only in terms of providing them with the means to acquire the needed skills and competencies, but also providing them with the opportunity to achieve senior level positions in an organization. This professionalization should be based on the foundation that there is a basic level of skills and potential that can be identified, planned for, and delivered within and organization.

Change the Approach, Change the Outcome: Avoid the Insanity

It is often said that the definition of insanity is trying to do the same thing the same way over and over again and expecting different results each time. The project management practices that organizations exhibit today fit this definition in many ways.

In order to realize the benefits of a project management practice, organizations need to completely change their approach in how they hire and train project managers. They also need to more aggressively develop the knowledge, skills, and competencies of their project management staff. To accomplish this kind of cultural change, the organization must focus on two factors: the individual and the organizational component based on individual skills. The three areas that organizations need to target for assessment are:

  • Knowledge of project management concepts, terminology, and theories;
  • Assessment of an individual’s behavior in the workplace; and
  • Assessment of project manager potential.

Knowledge Assessment

There are number of ways to assess an individual’s knowledge. The international standard for knowledge in the project management field is the Project Management Institute’s A Guide to the Project Management Body of Knowledge (PMBOK® Guide), affectionately known as the PMBOK® Guide. The advantage of using this standard is that the PMBOK® Guide has created a uniform language of project management that can be used across all industries. The language and concepts help build a consistency of communication that allows for the use of tools and techniques across projects within an organization. Knowledge assessment should address both the five process areas of the PMBOK® Guide as well as the nine knowledge areas.

Knowledge Assessment

A second area of assessment is in behaviors exhibited in the workplace. This approach requires the use of a multi-rater tool that allows the acquisition of feedback on the project managers’ behavior from a variety of sources, typically a peer, subordinate, supervisor, or even a client. This type of assessment should focus on the desired behaviors that effective project manager’s exhibit in the execution of their jobs. For example, it might examine the creation of a stakeholder communication plan, the development and distribution of a team charter, or the execution of a risk management plan.

Potential Assessment

The third area of assessment is potential. The role of project manager carries with it a number of critical factors that, when used together, can identify the potential of the project manager candidate to be successful in managing projects. Additionally, these factors can indicate whether the candidate would be best used in small, non-complex projects or large, highly sophisticated and complex projects. Factors to consider include the abilities to handle stress, be flexible, and manage personal time; organization skills; and conflict management. The results of this assessment indicate an individual’s potential to survive and thrive in the role of a project manager.

The analysis of this information is done through gap analysis. The knowledge gaps are determined by examining the differences between the demonstrated level of knowledge and the level of knowledge that is required. The behavioral gaps are identified by examining the differences between the self-rating of the project manager candidate and the rater’s score. The gaps in both knowledge and behavior, based on the size of the gap, are targeted as developmental opportunities. The results of this integrated assessment are used to create professional development plans for project manager candidates.

While this individual assessment is being conducted, the organization should be determining what roles they will need to ensure an improved level of project performance. Possible roles include team leaders, multiple levels of project managers, program managers, project portfolio managers, project executives, project office directors, and chief project officers. With each of these roles, the organization will need to create effective job/role descriptions that define performance/competency expectations, experiential requirements, and prerequisites of whatever technical skills are required.

Bridging the Gap

With these role/position descriptions in place, the organization can conduct a gap analysis between their desired state and the existing state. This analysis is accomplished by aggregating the individual results of the knowledge assessment with the multi-rater assessments. Once this gap analysis is completed, the next step is to examine the results of the assessments of potential to identify the project managers who display the most potential to succeed in the project manager role. The key to examining this data is to identify the project managers who would be successful in either the small, non-complex projects or the larger, complex projects.

This collective information is used to structure both a developmental program and an organizational project management career path. A developmental program is a coordinated track that combines educational, training, and professional experiences for each role in the project management area. The developmental program should be aligned with a career path that clearly demonstrates how an individual can progress from a team leader to a project executive within the organization. Additionally, it can be used to assist in recruiting candidates to the organization.

The ultimate goal in using this approach is the creation of sustainable performance in managing projects. This can be accomplished by creating a well-trained, effectively positioned workforce that is capable of maximizing their potential in the various roles required in project management. The results will be manifested in higher job satisfaction, higher customer satisfaction, and higher corporate morale.

“Begin With the End in Mind”

This well-known statement by Steven Covey (1990) effectively illustrates how an organization should initiate a program designed to professionalize the role of the project manager. From the beginning, your organization needs to identify the roles and positions that are necessary to support the growth and sustainment of project management practices throughout the organization. In a progressively responsible position approach, start with team members, then team leaders, project managers, program managers, project office directors, and chief project officers.

Team Members

The team member position is where the actual, day-to-day work of the project is done. This is where the programmers write code, the installers install equipment; subcontractors provide their deliverables, and so on. Within this level, more definitive project roles include planners, controllers, schedulers, business analysts, risk managers, and others. The primary differentiator in these roles is a basic awareness of the common project management practices.

Team Leaders

The team lead position is an important transition between team member and project manager. This is where people begin to obtain their initial experience with tackling the responsibility of managing a small portion of a larger project. The team leader requires more than an awareness of the project management practices; he or she needs to be able to execute them with a level of effectiveness. They may be learning how to create the work breakdown structure, develop a rudimentary schedule, or participate in the development of a risk management plan. The key is to have them supported or directed by a knowledgeable project manager who can demonstrate or model the desired behaviors. Essentially, they would be in the learning mode while acquiring solid project management skills.

Project Managers

The project manager level can be divided into levels depending on the needs of the organizations. These levels can include assistant, associate, senior, or expert. An assistant project manager is an individual who has successfully led three or more small teams and is now ready to lead a small, non-complex project under the supervision of a more senior project or program manager. An associate project manager is an individual who has led three or more small projects and is ready to assume the responsibility of leading a medium size project of moderate complexity. At this level, the associate project manager may be asked to lead multiple moderately sized projects. A senior project manager is an individual who has demonstrated competency by successfully leading a moderately sized, moderately complex project or has managed multiple projects simultaneously. The expert project manager is an individual who has successfully led several projects of varying size and complexity. This individual would be responsible for managing a portfolio of projects with the responsibility of managing several project managers of different levels.

Program Managers

Program managers rise from the ranks of the expert project managers. These individuals have a track record of successfully leading complex projects. Additionally, they have the business acumen to understand the relationship of projects to the organizations strategic business objectives.

Along with this responsibility comes resource planning, profit management, and tactical decision-making authority.

Project Executives and Project Office Directors

Project executives and project office directors also come from the ranks of expert project managers, and use the same set of skills to effectively manage an enterprise project system.

Establishing Performance Standards

For each of these levels, a set of expected performance standards, which relate to each level, needs to be established. Exhibit 1 provides a framework for identifying the desired roles and associated competencies for a proposed career path in project management. The competencies are rated on a scale from one through five. One indicates an awareness of the project management concepts and practices. Two is indicative of basic execution; the individual is in a learning mode. Three is effective execution; the individual demonstrates solid skills in the use of project management tools and techniques. Four indicates the individual models the skills of project management at a superior level. Five indicates the individual has mastered the skills and is capable of teaching, coaching, or mentoring others effectively.

This Project Management Performance Expectations Worksheet

Legend: 1 = Awareness, 2 = Basic execution, 3 = Effective execution, 4 = Models, 5 = Coach, Teach, Mentor

Exhibit 1: This Project Management Performance Expectations Worksheet

This table is a representative sample of the approach to take when identifying core skills. Each of the rows could be further dissected to identify specific skills that are considered key result areas or key performance indicators for the organization. Additionally, the level of competency can also be defined according to the needs of the organization.

The first area of assessment is knowledge. Because the PMBOK® Guide is recognized as the international standard on project management knowledge, any assessment should be geared towards this standard. The following table (Exhibit 2) is an example of a completed knowledge assessment. On an individual basis, the candidate can see how they scored on each knowledge area, how they compared to the highest score, their percentile ranking, and how many areas they passed. For the organization, an aggregate table provides insight into the areas of strengthened areas that need improvement for their entire population. This information is used to begin developing a targeted education and training program designed to meet those needs.

Example of a completed Knowledge Assessment report

Exhibit 2: Example of a completed Knowledge Assessment report.

The second assessment area is behaviors in the workplace. As mentioned earlier, the ideal assessment involves the candidate, as well as someone who has first-hand knowledge of the candidate’s behavior in the workplace. This person can be a peer, subordinate, supervisor, or a client. The assessment has the candidate and the rater apply scores to a number of key performance indicators across the project management process areas. There is a dual focus to this assessment: confirmation of behaviors and analysis of competency. The analysis of this assessment looks for significant gaps between the candidate and rater. These gaps are created either by the candidate rating themselves higher than the rater or the candidate rating themselves lower than the rater. When there is little or no gap found, the important thing to review is the level of agreement. Both the candidate and rater could agree on the level of performance but in reality, that level may be less than the desired level. In that case, the organization has an opportunity for developmental actions if required. The following table is an example of a behavioral assessment.

Example of a Behavioral Assessment report

Exhibit 3: Example of a Behavioral Assessment report.

The final area of assessment is the most difficult and sensitive to determine: the potential to be successful in the stressful role of project manager. Potential is an elusive quality that can be difficult to ascertain. Nonetheless, there are identifiable traits that contribute to the success of an individual in this role. Traits such as flexibility, adaptability, organizational skills, personal time management, and conflict management can be measured in order to determine the strength of an individual. Because projects come in many sizes and complexities, some individuals may thrive in an environment where they have the autonomy to run small, standard projects yet struggle when they are given a larger, more complex project. For an organization to effectively manage their projects, they need to know which project managers thrive under which conditions.

The real value of these assessments is learned by aggregating the results of all three assessment areas (knowledge, behavior, and potential), and using the output reports to develop a comprehensive view of their project manager population. A possible output could be that the candidate has adequate knowledge, poor execution behaviors, and solid potential. Using the combined information, the organization can determine where the gap really exists. It may be a matter of education, adding a mentoring relationship, or providing more directed experiences to improve performance. Another possible scenario is that the candidate has high knowledge, poor performance, and low potential. Analysis of this situation may determine that the candidate may be best suited to a specific role in the project management area, such as a planner, scheduler, or business analyst rather than a project manager. Using these assessments together allows the organization to more effectively develop and deliver targeted professional development interventions for their project management population.

Summary

Project management as a practice offers an impressive set of tools for an organization to more effectively manage their resources, both human and capital. A professional development program that incorporates the best practices of using these tools effectively contributes to improved competency. Organizations that recognize and align their resources to thrive in different types of project environments will see improvements in overall project performance. Focusing on building project manager competencies means first identifying what needs to be improved. To do this requires a comprehensive project manager competency assessment program that analyzes knowledge, behaviors, and potential. The results of this type of individual assessment will help organizations begin to harness the power of their project managers’ skills and abilities and enable them to focus training where it’s most needed. Matching project manager skills with the types of projects they are prepared to handle will result in more effective project execution, and subsequently better organizational performance.

References

Covey, Steven, (1997) Habits of Highly Effective People, NY, New York: Simon and Schuster, ,

Pennypacker, Jim, (2004) Value of Project Management Training, Havertown, PA: Center for Business Practices,

PMI (2004) A Guide to the Project Management Body of Knowledge (PMBOK® Guide). Newtown Square, PA: Project Management Institute

This material has been reproduced with the permission of the copyright owner. Unauthorized reproduction of this material is strictly prohibited. For permission to reproduce this material, please contact PMI or any listed author.

©2007 Jimmie L. West, Ph.D., Michael Plumeri, Ph.D.
Originally published as a part of 2007 PMI Global Congress Proceedings - Seattle

Advertisement

Advertisement

Related Content

Advertisement

Publishing or acceptance of an advertisement is neither a guarantee nor endorsement of the advertiser's product or service. View advertising policy.