Abstract
We, as team members, want to do the best we can to ensure the project is delivered successfully. No one on a project likes to admit they are short in knowledge or skill areas. It does not matter if you are a participant, team lead or the program/project manager. A Project Management Skills Gap Assessment (PM SGA) is a tool that can provide insight on how and where one can improve their Project Management knowledge and skills.
This paper and presentation will depict a case study stating the “who, what, where, when, why and how” of Project Management Skill Gap Assessments and their effectiveness at developing plans for individual and organizational improvement. The PM SGA also measures the difference of “expected” knowledge versus “actual” knowledge for each participant and drives the future organizational baseline for competency.
Recently, the authors successfully administered SGAs and organizational improvement plans for a large public utility company. These plans assisted the client in the identification of new training objectives and supporting career path development. Additionally, other customer organizational data was collected to facilitate assessment on a team and organizational level. Patterns found in the individual SGAs depicted many common project management and organizational behaviors, some of which include:
Project Management Behaviors:
- White Knights
- Fireman's Syndrome
Organizational Behaviors
- The Blind Leading the Blind
- Working Towards Best in Class
- Seat of the Pants Project Management
- Management by Memo Organization
Once the PM SGA data for the client was collected, it was then mapped to the key indicators of Infotech Management's Skill Gap Assessment Best Of Class Benchmark, Kerzner's Project Management Maturity Model (PMMM) (Kerzner, 2001) as well as PMI's Organizational Project Management Maturity Model (OPM3™) (Project Management Institute, 2003).
Once knowledge gaps are identified, the following action plans have demonstrated success:
- Development/Enhancement of Standardized Processes
- Initiation and Implementation of a PMO Function (real or virtual)
- Customized, Focused “Boot Camp” Style Training
- Achieving PMP Certification for Those Who Aspire
- Mentoring and Coaching
Introduction
There is an old commercial for deodorant that states “Never let them see you sweat”. Similarly, “Never show you have a gap” in your project management knowledge is something most project managers believe. No project manager likes to share their knowledge gaps. It is akin to showing the chink in your armor, a weakness. These knowledge gaps should be identified and closed as best as possible so that the other strengths are enhanced. No project manager is perfect; we all are supposed to learn and grow and change. This is the underlying premise behind Project Management Skills Gap Assessments.
Skills Gap Assessment – What Is It?
A Skills Gap Assessment (SGA) is a tool used to assess and measure an individual's project management knowledge. This is a snapshot of your current actual knowledge level which correlates to the skills you currently possess. When used by an individual, it can support his/her performance improvement plan to identify the gaps and define action plans for resolution. When SGAs are collected from multiple members of an organization, a composite view is created trends emerge. The Skill Gap Assessment can be used in conjunction with an Organizational Assessment to help assess the current state of Organizational Project Management Maturity.
The PM SGA data establishes a measurable baseline prior to the implementation of any project management focused improvement initiative, such as:
- Training
- Process Definition/Enhancement and Implementation
- Tool Implementation
- Coaching and Mentoring
Participants are given the Skill Gap Assessment upon completion of training, etc. to measure and validate the effectiveness of the improvement initiative. While it is very common for improvement initiatives to be identified and rolled out, there is often no “apples to apples” comparison to measure the improvement on an individual or group basis on the back end.
The PM SGA data can be rolled-up to a project team, department and/or organizational level which allows for many different slices and views. Generally, individual scores are kept confidential with only organizational rollups being provided for analysis. SGA participants should be reassured that their scores will not be negatively reflected in any future performance evaluation.
It is important to compare the rolled up organization data to industry and best-in-class benchmarks. Infotech Management has collected over 10,000 resultant profiles across a variety of industries and participant roles. The benchmarks represent the desired project management knowledge and competency goals. Once the improvement initiatives have been completed, ideally the benchmark should be realized and the organization can reap the benefits of their initiative(s).
Focusing the improvement initiatives on the “gaps” yields the greatest value – both for individuals and organizations.
The PM SGA measures five knowledge areas within two skill groupings:
Technical Skills
- Planning
- Scheduling
- Implementation
Soft Skills
- Leadership
- Organization
Skills Gap Assessment – Who Participates and Why?
Skills Gap Assessments are typically conducted for three levels of project management competency:
- Team Leaders
- Project Managers
- Program Managers (including organizational managers where appropriate)
Recently, the authors successfully administered SGAs and organizational improvement plans for a large public utility company. For two key operational areas of this client, they had observed (on their own) client problems and project issues relating to status and completion of projects. Essentially, project completion dates were being missed more often than what was deemed acceptable and this trend appeared to be increasing. Additionally, due to significant schedule and budget variances, many of their projects required “recovery”. The resource utilization issues identified were due to “outages” of project management knowledge and skills. This client had confirmed that their organizational project management maturity level was low.
The client desired improvement in project management knowledge, skills and processes, but was unsure of how to proceed. They had solicited proposals on how to address this issue, but felt general-purpose project management training was too costly and too time consuming. The proposition of a Project Management Skills Gap Assessment appealed to the company because it provided a measurable baseline from which to assess and enact improvement initiatives. Options for improvement initiatives were evaluated against the gaps present. These options included targeted project management training in an intensive “boot camp” format followed by mentoring and coaching. These options suited their existing organization behavior. At the completion of an improvement initiative, a post-SGA was conducted to demonstrate measurable improvements in project management knowledge and maturity.
In order for any improvement initiative to be successful, management commitment was key. This client's upper level commitment to the program participants was a critical success factor and had a stabilizing effect on the business and project performance of the entire organization.
Skills Gap Assessment – How Does It Work?
The PM SGA consists of an online (web-based) assessment of fifty multiple choice questions across the three technical and two soft skill areas. It was followed up with participant interviews conducted at client worksites. Organizational demographic data was gathered to identify interview candidates and used for gap modeling. The individual and organizational scores in each of the five tested areas were reviewed and combined with the qualitative data from the interviews for validation and confirmation of accuracy. There are some key organizational behaviors that can be inferred from raw scores so those inferences were validated during the interview process. In addition, data was used to analyze the organization's project management maturity using both the Kerzner and OPM3 models.
What Does An SGA Show?
Using bar charts and kiviat diagrams, the five tested areas of technical and soft skill knowledge are documented and measured against expected and best-in-class benchmarks for each competency level and industry. Sample diagrams are shown below in Exhibit 1.
Exhibit 1: Sample PM SGA Diagrams
From these diagrams and charts, we have identified some common pattern in individuals and organizations:
White Knights
This behavior is typified when the work is being accomplished through sheer individual efforts, or brute force, rather than through a defined, standardized process. While this is sometimes necessary when an emergency arises, it should be unacceptable as a “business as usual” approach for the other 90% of projects. The SGA composite profile deviates with high spikes in both the Leadership and Implementation score values. The other three tested areas usually are significantly below anticipated benchmarks which exacerbates the problem.
Fireman's Syndrome
This profile is similar to a White Knight, with a disconnect in Leadership and Implementation as well as Planning. This profile suggests that the participants know what they are supposed to do (high Planning scores) but elect not to do it. They brute force their way to success, only after making the environment ripe for chaos and reactive behavior. Further analysis often reveals an immature project management environment that lacks infrastructure to support projects or project teams.
In addition to comparing individuals to the competency level that is most appropriate to their current project role, a composite profile can be created that compares all three competency levels to each other. This analysis will tend to reveal organizational issues as common threads are identified across all levels. For this particular client organization, it was useful to identify “star” potential and helped the organization select candidates for the training boot camp.
The Four Types Of Organizations
When groups of PM SGA scores are assessed, four types of organizations can be depicted from the rolled-up data, based on either low or high individual competency mapped against organizational capability.
The Blind Leading The Blind – Low Competency and Capability
When an individual and the organization are weak in their efforts, they may have the passion but not the direction, much like the lesson of Alice in Wonderland when she was trying to leave:
“Would you tell me, please, which way I ought to go from here?”
“That depends a good deal on where you want to get to,” said the Cheshire Cat.
“I don't much care where ---“ said Alice.
“Then is doesn't matter which way you go,” said the cat.
– Lewis Carroll (Carroll and Tonniel 1997)
Working Towards Best in Class – High Competency and Capability
A coordinated effort of individuals possessing strong skills and a supportive organization yields a path which drives towards continuous improvement and monitored change. This is the goal to reach for when evaluating the improvement initiative effectiveness. An organization whose scores represent this score have fully assimilated, engaged and demonstrated their skills and knowledge in Project Management. They not only walk the walk and talk the talk – they teach it to others and share it with their peers.
Seat of the Pants Project Management – High Competency and Low Capability
A purely reactionary organization and their members function in the “Just get it done” mantra. Skillful and knowledgeable, but hamstrung by the capability and understanding of the true “what it takes” by management. This organization is typified by “white knights” who have their own individual processes, but no standards. Organizations “throw” money at the problem by massive amounts of training without ever addressing the underlying infrastructure necessary to make changes last. Attrition and burn out are key indicators of an organization in this maturity level.
Management by Memo Organization – Low Competency and High Capability
Dictatorial and functionally independent, the organization drives all commands and edicts down to the masses since they “know” better. The individual members have basic or minimum project competency, but not enough to question why or how it can be improved. Change is instituted on small scales with minimum impact to the de facto way of doing business. It is the Queen Mary being steered with canoe oars. Emphasis is on filling out the forms and not on doing the work.
Action Plans for Improvement Initiatives
Once knowledge gaps are identified, action plans (or improvement initiatives) can be enacted. The following action plans have shown demonstrated success:
- Development (and/or enhancement) of standardized processes has been one of the key foundational efforts to promote increased project management maturity.
- Implementation of a PMO Function, either real or virtual, can be extremely effective in evolving organizations with low capacity. The centralization of knowledge, templates, lessons learned and the other collaborative aspects support the improvement of the technical skill categories.
- Focused, customized “boot camp” style training that specifically addresses the knowledge gaps is extremely effective. This results in the development of a common project management and establishes the foundation. From this, all participants receive common grounding in terms and practices so that they can assimilate and improve their own skills.
- PMP® certification aspirations confirms the commitment of the individual and the organization to standardized project management practices and improvement. The new PMP's become agents of continuous process improvement and growth in project management maturity.
- Mentoring and coaching for reinforcement of project management best practices in the context of the business direction and vision. A proper balance of “guideline” and actual applicability on a case-by-case basis is a very cost effective approach for organizations who possess high capability but low competency.
- Development of a standardized Project Management Learning Network provides the educational framework for standards and repeatable knowledge improvement. It may also provide the basis for a defined project management career path and promotability of project management organizationally.
Conclusion
A Project Management Skills Gap Assessment is a tool that can provide insight on how and where an individual or organization can improve their project management knowledge and skills. The PM SGAs establish a measurable baseline prior to the implementation of improvement initiatives. Following the implementation of an improvement initiative, post-SGAs help measure and validate the effectiveness of the improvement initiative. Experience shows that customized, focused initiatives are a cost-effective means of improving individual and organizational knowledge, skills and maturity. Additionally PM SGA's support all organizational models evaluations – CMMi, Six Sigma, OPM3 (Project Management Institute 2003) and Kerzner PMMM (Kerzner, 2001) – to document maturity and initiatives in support of their enhancement.
