Projects and people
a marriage made in heaven
David Winsborough, Organisational Psychologist, Winsborough Ltd.
Bev Marshall, Organisational Psychologist, Winsborough Ltd.
Iain Fraser, PMP, Project Management Consultant, Project Plus Ltd.
This large project emerged from a series of meetings with New Zealand's largest telecommunication company, Telecom New Zealand. Telecom New Zealand outlined a need for a formalized assessment and development planning process for Project Managers across all business units as part of a major process development initiative. The company undertakes a large number (over 1,000) of concurrent projects, including some complex, large value, high-risk projects. Many of these projects involve the development of products and services in a highly competitive industry and there is considerable pressure to be first to market. The company had identified highly effective project management as a source of competitive advantage, and wanted to identify potential leaders among their project managers.
We were brought in to prepare development plans for each project manager in order to progressively improve the competency level and to assess each candidate against the various standards. This was done with the express purpose of matching the demands of each project with the caliber of the individual project managers. The assessment also provided an overall view of the “bench strength” of project managers within the organization, and identified the top 10 performers. In the absence of an existing methodology to assess the competence of project managers working in different streams of the business, these judgments could not be made objectively.
The expertise of two firms was employed: Winsborough Limited are specialist organizational psychologists and had previously developed the Telecom Project Manager competencies; Project Plus Limited are expert project managers and had already provided specialist training and consultancy advice to Telecom New Zealand.
The methodology the two organizations developed was innovative, based around a “multiple hurdles” approach: participants had to achieve a certain standard at the end of stage one to qualify for the second stage of the assessment, which involved psychometric assessment and an in-depth behaviorally based interview.
The project began with the development of the assessment methodology. The Telecom competencies developed by Winsborough Limited were:
- Planning and monitoring—Doing the business
- Maintaining Effective Relations
- Managing the team
- Achievement orientation
- Understanding the business context
- Compiling and deploying resources
Project Plus Limited had a project manager assessment questionnaire that tapped into most of the competency areas. The organizational psychologists and the project experts sat down together and adapted and augmented the existing questionnaire so that it became a behaviorally based assessment against the competencies. Two forms of the questionnaire were developed: a self-assessment for project managers and a manager's assessment for each line manager to complete on each of their project managers.
As noted above, a multiple hurdles approach was taken. Project managers completed the self-assessment against the competencies, and their manager completed the same assessment on them. Those project managers who reached the required standard (as described below) in this first stage of the assessment were invited to participate in stage two.
The standards selected for use through the process were the National Competency Standards for Project Management sponsored by the Australian Institute of Project Management. These are generic standards that can be used in any industry or enterprise. They were developed with input from Australian industry (and in the absence of New Zealand standards, most appropriate to use here), and by using Body Of Knowledge work from the Project Management Institute in the USA, and The Association of Project Managers in the UK. The choice of these standards enabled the client to benchmark itself against external standards. There are up to eight levels within the project management standards. The standards for competent project managers begin at Level 4. However, the Australian Standards Framework commences at Level 1 for any type of work or activity.
Levels 1 to 6 were selected as being most representative of requirements for projects managers at Telecom New Zealand. (Levels 7 and 8 are most appropriate for more senior managers or project directors.) A full description of the levels can be found in the appendix to this paper.
Competence can be developed through a variety of means, including training, coaching, observing in the workplace, or learning on the job. Level 4 is considered competent in any given project management area.
Exhibit 1. Comparison of Ratings—All in Sample vs.Top 8
At this level, work is likely to be without supervision with general guidance on progress and outcomes sought. The work of others may be supervised or teams guided and facilitated. Responsibility for and limited organization of the work of others may be involved. Competency at this level involves the application of knowledge with depth in some areas and a broad range of skills. There is a wide range of tasks and roles in a variety of contexts, with complexity in the range and choice of actions required. Competencies are normally used within routines, methods and procedures where some discretion and judgment is required in planning selection of equipment, work organization, services, actions and achieving outcomes within time constraints.
Internationally, people management, staff resource deployment and performance management are increasingly seen as important aspects of the competency requirement.
Phase One Results
Eighty-seven project managers (and their line managers) completed the competency based questionnaire Following analysis by the consultants, the output from this phase was:
• An individual profile of each participant against the competencies.
• A series of graphs showing Telecom New Zealand's overall project management capability, including particular areas of strength and weakness across the group.
• Identification of any managerial inconsistency in the assessments. These flagged managers who were extraordinarily harsh or lenient in their ratings.
• Gradation of the 87 project managers into three categories— those who did not meet the Level 4 standard, those competent at the standard, and those who clearly exceeded the standard. NB Although the standard was set at Level 4, there were a number of participants who were selected to continue to Phase 2 based on a number of reasons (e.g., their manager's decision regarding potential or a large discrepancy between manager and self-assessment ratings).
This phase enabled the broad assessment of project management strengths and weaknesses across the whole organization and has proven an extremely useful workforce planning and development tool.
Phase Two Results
Fifty-three people were invited to participate in Phase two and underwent an in-depth competency based interview (60–90 minutes) with an organizational psychologist from Winsborough Ltd. and a Project Management expert from Project Plus Ltd. Participants also completed a psychometric test of managerial judgment. The output from this stage two was:
• A development report for each person highlighting areas of strength or weakness, pointing to development needs and providing some suggestions for action (provided to individuals).
• Reports for line managers summarizing group and individual strengths, development needs and any special considerations.
• An outline of Telecom's project management competency including areas of strength across the organization and also identifying any areas where there was widespread need to development.
• Identification of a group of eight people with senior project manager/project director potential.
It is important to note here that the initial competency development project identified superior project manager competencies as well as the standard project manager competencies used in this project. The competencies for superior project managers identified three specific areas of importance:
• Ability in dealing with and relating to people
• Ability to navigate through competing interests
• A “will do” attitude that translates into an unflagging commitment to the project coupled with a “can do” attitude that translates into unshakeable faith in their ability to achieve a successful outcome.
The stage two work substantially confirmed the view that these three factors differentiate the top eight from the rest. Significantly, scores on technical competencies failed to differentiate project managers, leading directly to the hypothesis that “nuts and bolts” skills are useful but may not be essential or sufficient for superior project leader performance. A longitudinal study is being proposed to examine this view.
Exhibit 1 displays the average of all project managers in the sample, as rated by their managers and the consultants, against the average scores of the Top 8. The Doing the Business competency (which covered the nut and bolts aspects of project management) failed to differentiate superior performance. It was, however, a reliable indicator of poor performance.
Managing the Team, Compiling and Deploying Resource, Understanding the Business Context, Maintaining Business Relationships and Communication were, however, factors that differentiated the very good from the average. These factors should guide future selection, and development planning for all potential project managers and leaders.
Usefully, the line managers’ and consultants’ ratings were similar and co-varied, which validates the use analytical tools against the managers’ use of their actual experience and observations of the performance of each individual.
This assessment project is unique in its application of a psychometric measure of managerial judgment to the sample. This instrument was utilized to test the hypothesis that superior project managers would have similar profiles to superior line managers. That is, their judgment and decision-making about situations involving the use of resources, managing relationships both with stakeholders and their team members, and pursuing organizational goals would be highly developed and robust. The test assessed a person's ability to weigh up “real life” management situations and decide on appropriate and effective ways of handling them. The Top 8 were significantly higher on overall managerial judgment but this difference was entirely made up by their skill in people management. This pattern was consistent across all scales.
Exhibit 2 shows the relative comparisons for the overall project manager sample against the Top 8 on the dimensions of managerial judgment.
As scores on this test did not correlate with project manager ratings, we are confident it taps a different, and useful, dimension contributing to project leadership. Importantly, this enabled us to develop a project leader potential matrix (see Exhibit 3).
This matrix maps the overall level of project management against the overall score on managerial judgment. If our purpose is in identifying potential project leaders (our Top 8), then our focus is on those who fit into the top right hand corner: they are high in managerial judgment and exceed the standard level of competence set at Level 4.
There is a cluster of project managers above level 4, but who do not show well-developed judgment—their development needs are likely to be in developing their resource management and people management skills.
There is increased interest in understanding the factors that underpin good project management performance. For example Hauschildt and his colleagues have attempted to identify a cluster of factors that distinguish Project Stars from other project management types and that correlate with project success. Significantly, his high performers shared a number of characteristics with the Top 8 identified in this project. In our analysis this lends substantial weight to the idea that superior performers are more readily distinguished by people management skills and good decision-making, rather than pure project management technical competence.
The current study extended the application of Project Management Standards in a unique and useful way. By developing a behaviorally based questionnaire we were able to successfully identify a sample of superior project managers. With the addition of a rigorous, competency based interview we were able to effectively identify high performing project managers and further, highlight a very small subset with the potential to become project leaders. The factors that differentiated these project managers from their colleagues were easily identified. On this basis, significant, mission critical organizational deliverables have been entrusted to our Top 8, although one of them has moved on to another position, and 1 has resigned. Further work examining the success of these selections will be performed over time.
Hauschildt Jurgen et al. 2000, Sept. Realistic Criteria for Project Manager Selection and Development. Project Management Journal, pp 23–32.
Appendix: AIPM Standards and Levels
These pages give more detail regarding the rating levels and how these correlate with the standards that were used for the assessments.
The standards selected for use in this assessment were the National Competency Standards for Project Management sponsored by the AIPM. These are generic standards that can be used in any industry or enterprise. They were developed with input from Australian industry (and in the absence of New Zealand standards, most appropriate to use here). The standards also used Body of Knowledge work from the Project Management Institute in the USA, and The Association of Project Managers in the UK.
There are up to eight levels within the project management standards. The standards for competent project managers begin at Level 4. However, the Australian Standards Framework commences at Level 1 for any type of work or activity.
The Levels between 1 and 6 were selected for this assessment as being most representative of requirements for project managers (Levels 7 and 8 are most appropriate for more senior managers or project directors). As a guide, Level 1 indicates a very low degree of competence in project management currently. Level 6 indicates a very high degree of competence, with Level 4 being the required level in order to be considered “competent” in any given project management area.
This “competence” can be developed through a variety of means, including training, coaching, observing in the workplace, or learning on the job.
AIPM Level 1
Work is likely to be under direct supervision with regular checking, but may take the form of less direct guidance and some autonomy where working in teams is required.
Competency at this level involves the application of knowledge and skills to a limited range of tasks and roles. There is a specified range and variety of contexts where the choice of actions required is clear.
Competencies are normally used within established routines, methods and procedures which are predictable, and within which judgment against established criteria is also established.
AIPM Level 2
Work is likely to be under routine supervision with intermittent checking, but may take the form of general guidance and considerable autonomy where working in teams is required.
Competency at this level involves the application of knowledge and skills to a range of tasks and roles. There is a defined range and variety of contexts where the choice of actions required is usually clear, with limited complexity in the choice.
Competencies are normally used within established routines, methods and procedures, in some cases involving discretion and judgment about possible actions.
AIPM Level 3
Work is likely to be under limited supervision with checking related to overall progress, but may take the form of broad guidance and autonomy where working in teams is required. Responsibility for the work of others may be involved, and team coordination may be required.
Competency at this level involves the application of knowledge with depth in some areas and a broad range of skills. There is a range of tasks and roles in a variety of contexts, with some complexity in the extent and choice of actions required. Competencies are normally used within routines, methods and procedures where some discretion and judgment is required in selection of equipment, work organization, services, actions and achieving outcomes within time constraints.
At this level people management skills become more important as performance management of others is involved.
AIPM Level 4
Work is likely to be without supervision with general guidance on progress and outcomes sought. The work of others may be supervised or teams guided and facilitated. Responsibility for and limited organization of the work of others may be involved.
Competency at this level involves the application of knowledge with depth in some areas and a broad range of skills. There is a wide range of tasks and roles in a variety of contexts, with complexity in the range and choice of actions required.
Competencies are normally used within routines, methods and procedures where some discretion and judgment is required in planning selection of equipment, work organization, services, actions and achieving outcomes within time constraints.
People management, staff resource deployment and performance management are increasingly important aspects of the competency requirement.
AIPM Level 5
Work is likely to be under broad guidance. The work of others may be supervised or teams guided. Responsibility for the planning and management of the work of others may be involved.
Competency at this level involves knowledge and a range of technical and other skills being applied to tasks, roles and functions in both varied and highly specific contexts. Such knowledge will need to be of substantial depth in certain areas.
Competencies are normally used independently and both routinely and non-routinely. Judgment is required in planning and selecting appropriate equipment, services, techniques and work organization for self and others.
AIPM Level 6
Work is likely to be under limited guidance in line with a broad plan, budget or strategy. Responsibility and defined accountability for the management and output of the work of others and for a defined function or functions may be involved. Competency at this level involves the self-directed development of knowledge with substantial depth across a number of areas and/or mastery of a specialized area with a range of skills. Application is to major functions in either varied or highly specific contexts.
Competencies are normally displayed independently and are substantially non-routine. Significant judgment is required in planning, design, technical or supervisory functions related to products, services, operations or processes.
Above AIPM Level 6
Above Level 6 the person is likely to have full responsibility and accountability for all the aspects of the work of others, and functions including planning, budgeting and strategy. They will engage in self-directed development and will demonstrate high levels of complex judgment in technical and/or managerial functions.
Proceedings of the Project Management Institute Annual Seminars & Symposium
November 1–10, 2001 • Nashville, Tenn., USA