Abstract
This paper draws on the extensive research and application of project management capability assessment at the project, program, and organisational levels. The paper will explore the components of assessment that can be applied to the world of projects including, what they are for, how they work, and why you'd use them. By the end of the paper you will have learned how to build a tailored roadmap of project management capability assessments designed to improve your specific organisation's project management delivery capability. This roadmap will form the basis for understanding where you are today to ensure you know how to build a better tomorrow.
Introduction
Every organization wants to put its best foot forward and this generally means being seen to have a good strategy and, more importantly, being able to deliver it. Most organizations have a strategy and many of their strategies look similar. It is the ability to implement the strategy that makes the difference. Key vehicles for implementation of corporate change and strategy are portfolios of projects and programs. When an organization recognises that business change and corporate strategy can be delivered through projects, there is realisation that project management capability is a key to competitive advantage (Aitken & Crawford, 2008a; KPMG, 1997; Nieto-Rodriguez & Evrard, 2004) and that good governance requires assurance of capability. These are two sound reasons for investing in development and continuous improvement of project management capability.
To develop and improve project management capability you need to know the status of your current capability, and have a plan for where you want to be. In fact, just maintaining current capability means that you have to be involved in some form of continuous improvement or evidence suggests it will go backward (Thomas & Mullaly, 2008). All of this requires an understanding of the components of organizational project management capability, ways of assessing or measuring those components, and standards and benchmarks against which to judge how well you are performing as a basis for planning improvement. Assessing your capability, setting a baseline and measuring your progress toward a desired future state are proven ways of improving personal and corporate performance.
This paper presents guidelines developing a coherent plan for assessment of organizational project management capability that will drive improvement. It starts by looking at the components of project management capability and what needs to be assessed with indications from research of key areas to focus attention for both assessment and improvement to get best return on investment. A framework for different aspects of organizational project management capability and how to assess them is provided.
What to Assess: The Components of Project Management Capability
If we think about project management capability as encompassing everything within an organization that relates to delivery through projects we realize that it involves the whole of the organization and not just the project management community and the projects themselves. It involves the capability of the board of directors and senior management who set the corporate strategy, decide on the portfolio of projects and programs they will undertake, and how they will resource them. The executive team also provide the vital link between corporate, program, and project governance, between the permanent and temporary or project-based parts of the organization, through their role as sponsors (Crawford et al., 2008). The permanent organization or business as usual (BAU) needs to have the capability to accept the fruits of projects and programs and exploit them to achieve the planned benefits.
Historically, there has been a tendency to think of projects in isolation and focus on the development of methodologies, techniques, and tools required to manage individual projects, typified by A Guide to the Project Management Body of Knowledge (PMBOK® Guide). But there is growing awareness of the importance of capability to manage multiple projects, recognized by PMI's standards for portfolio and program management (Project Management Institute, 2006a, 2006b) and of the need for corporate infrastructure and environment to support projects that inspired the Organizational Project Management Maturity Model (OPM3®) Knowledge Foundation.
To really understand our corporate project management capability we can't just look at how each project is managed. We need to assess at various levels. The executive level needs sufficient agility to articulate strategy, respond to the changing external environment and make decisions about deployment of resources, and provide the leadership and governance to monitor and control overall performance. The next level of governance and program capability requires effectiveness. The executive team and other members of senior management need to be able to effectively provide sponsorship to projects and programs, as the governance conduit between strategy and its implementation. They must have the leadership ability, credibility, communication, interpersonal, and self-management skills necessary to govern the project, take accountability for the business case, give direction and make decisions, critically review progress, manage internal and external interfaces, as well as providing support as required to the project and program managers and their teams (Crawford et al., 2008). And they usually need to do this for more than one project or program at a time, on a part-time basis while continuing to fulfil their challenging senior management role. Program managers need to be effective in orchestrating the resourcing and coordination of multiple projects, mentoring project managers and their teams, managing stakeholders, and monitoring and ensuring the delivery of benefits. At the project level the aim is to define clear goals and performance expectations and to emphasise efficiency.
This hierarchy of organizational project management capability can be summarized in the following table (Exhibit 1), which illustrates the results of analysis of three decades of research into the desired outcomes from project-related endeavours and the capabilities associated with achieving them (Teague & Cooke-Davies, 2007).
Exhibit 1: Capabilities and Desired Outcomes
Using the framework presented in Exhibit 1, the Human Systems team conducted a survey probing the presence or absence of specific capability areas at each level, and the degree of success accomplished for each criterion at each level. Results are based on a total of 339 responses from senior managers, sponsors project managers, and project team members from 44 organizations in 27 countries and 23 industry sectors. Of a total of 43 capabilities identified (10 at the Strategic Capability [Agility] level, 20 at the Governance and Program Capability [Effectiveness] level, and 13 at the Delivery Capability [Efficiency] level), 13 were shown to have the significant impact on successful achievement of desired outcomes and therefore may be considered the critical capabilities:
Strategy Capability - Agility
1. Capacity to resource the whole portfolio
Governance and Program Capability—Effectiveness
2. Approval of projects only on the basis of a solid business case
3. Fully resourcing each project
4. Considering all strategic options for the delivery of a project
5. Assuring that all benefits promised by the project have committed “owners”
6. Ensuring that each project is fully aligned with the organizational strategy
Delivery Capability—Efficiency
7. Having a competent project manager
8. Having proven methods and systems for planning
9. Ensuring clarity about the technical performance expected from the product of the project
10. Using accurate information as the basis for decisions
11. Effective teamwork
12. Project team having the necessary authority to deliver the project
13. Effective risk management
Portfolio, program, project, and organizational standards give us a detailed picture of what constitutes project management capability. Research such as that conducted by Human Systems and previously outlined starts to give us an idea of where to focus our attention for investment in assessment and development of capability for maximum impact. We can further refine our understanding of what to assess by looking at the different types of capability that are required. From the aspects of capability previously indicated we can see that capability encompasses portfolio, project, and program management processes, systems, and practices that are used, as well as the competencies and skills of the people who use them. There are also differences in the capabilities required to manage a single project and to manage multiple projects, both at the program and portfolio level.
Having reviewed what constitutes project management capability in organizations, the next step is to think about how we can assess the different types of capability.
How to Assess Project Management Capability
The categorization of project management capability previously introduced provides a good starting point for determining how to assess project management capability, and Exhibit 2 provides a summary of the types of assessments that can be used for processes and practices, competencies and skills, single and multiple projects.
Exhibit 2: Categorization of Assessments
Single Project—Processes and Practices
This form of assessment is primarily concerned with the deployment of practices and processes on a particular project and will often be conducted at various times throughout the life of the project. The assessment may be required by clients to provide assurance that the project will be delivered as required in terms of time, cost, quality, scope, and safety. Earned value, stage gate, and peer and external consultant reviews may all be used for this purpose; however, stage gate reviews tend to be concerned with governance while earned value reviews focus more on time, cost, and scope performance. Project health checks may provide the opportunity to benchmark the use of processes and practices on the project with other projects within the organization or industry sector. These forms of assessment are clearly intended to both monitor and drive performance at the project level. Benchmarked project health checks are generally designed to encourage improvement in the way projects are managed.
Multiple Projects—Processes and Practices
Processes and practices for management of multiple projects encompass governance processes and structures, methodologies and tools, IT infrastructure, resource management, knowledge management, communication and reporting, risk management, change management, stakeholder management, competency development and procurement, and contracting processes. When thinking about the assessment of an organization's overall project management capability there is a tendency to think in terms of models for assessment. A number of models are available for this purpose and they tend to be in the form of “excellence” or “maturity” models.
General management is familiar with “business excellence” models such as the European Foundation for Quality Management (EFQM) Excellence Model and Baldrige Award, which they use to identify, describe, and then improve business processes. Similar models, focusing on project and program management, can be used to assess the current status of organizational project management capability to guide development and for periodic reassessment to determine and demonstrate progress. The IPMA International Project Excellence Awards and the Human Systems Corporate Practices Assessment (CPA), which has been continuously developed over the last 15 years by the companies that are members of the Human Systems knowledge networks are “excellence” models developed on principles similar to those driving the EFQM and Baldrige awards. The focus here is on identifying and assessing those working practices that lead to improved performance (Cooke-Davies, 2004).
The large number of available “maturity” models have been strongly influenced by the Capability Maturity Model (CMM) developed by the Software Engineering Institute (SEI) of Carnegie Mellon University for the software development process (Humphrey & Sweet, 1987; Paulk, Weber, Curtis, & Chrissis, 1995). Many of these have combined the concept of maturity as established by the SEI CMM with the view of project management presented in the Project Management Institute's PMBOK® Guide, which tends to present a specifically project centric or tactical view. Others such as Gareis (2002) and Kerzner (2001) take different views of organizational project management maturity.
In 2003 the Project Management Institute released its Organizational Project Management Maturity Model (OPM3®) Knowledge Foundation, which identifies more than 600 “best practices,” more than 3,000 “capabilities,” and more than 4,000 relationships between capabilities. They have since developed an OPM3® Product Suite and a process for certification of assessors. The OGC has also developed a Project Management Maturity Model (PMMM) in recognition that maturity questionnaires provide a simple tool for identifying areas where an organization's processes may need improvement (Cooke-Davies, 2004).
Different organizations in different industries will place emphasis on different aspects or components of project management capability. Each organization should therefore choose the model and process best suited to its need as a basis for establishing its current baseline, formulating and driving an improvement plan, and conducting periodic re-assessment.
Project, program, and portfolio management offices (PMOs) of various types and sizes have become an important aspect of organizational project management capability. Although research (Hobbs & Aubry, 2007) clearly indicates that there is no simple categorization or formula for PMOs; but there is a growing interest in assessing their performance. The available research ( Crawford, 2004; Hobbs & Aubry, 2007) and benchmarking initiatives such as that of Human Systems provide a basis for comparison and baselining and this is often combined with assessment of stakeholder satisfaction through questionnaires and interviews.
Of course, programs comprise multiple projects. Program assessments tend to focus on assessment of component projects and utilize the assessment methods available for single projects. At the program level, there is an opportunity for comparison and aggregation of results across the program and assessment of features that are more significant for programs than for projects such as resource utilization, program risk, change management, stakeholder satisfaction, and benefits realization.
Single Project—Competencies and Skills
The results of research presented earlier in this paper highlight the importance of a competent sponsor, project manager, and project team to effective and efficient project performance. Assessment and development of competence of the people on projects is therefore a vital aspect of overall organizational project management capability.
Assessment of competence requires standards or guides against which measurement can be made. Knowledge and demonstrable performance (use of project management practices) are the two aspects of competence that are most widely addressed in project management standards or guides produced by project management professional associations or standards setting bodies. Knowledge is the easiest to assess and least controversial aspect of competence and is generally assessed using multiple-choice questionnaires such as those used for the Project Management Professional (PMP®) exam.
Demonstrable performance or use of project management practices can be assessed against performance-based competency standards such as those of the Australian, British, and South African governments and the Global Alliance for Project Performance Standards (GAPPS). Assessment against performance-based competency standards is generally undertaken by trained workplace assessors. Candidates are required to gather evidence of the use of practices in accordance with performance criteria specified in the standards. The workplace assessor works with candidates, advises, and assists them in achieving recognition of competence. Candidates are assessed either as competent at a particular level, or “not yet competent.” If assessed as competent, a candidate may be awarded a qualification recognized within a government endorsed framework. The GAPPS (www.globalpmstandards.org) does not award qualifications. It audits and endorses assessment processes and standards as a basis for mutual recognition and transferability of project management and related qualifications.
Behaviors are the most difficult, expensive, and controversial aspects of competence to assess, although it is behaviors that differentiate between threshold and superior performance. A number of organizations have developed corporate competency models, identifying the behaviors that are considered desirable and associated with superior performance within a specific corporate context. However, these are usually applicable across the organization and not project management specific. Assessment Centers are sometimes used to assess behavioral competencies and can be extremely effective but they are also very expensive.
When dealing with individuals, assessment and development of competence can often be addressed through the certification and qualification processes of professional associations and academic institutions. Professional certification will generally ensure a consistent level of basic knowledge, shared language, and threshold performance across a project team. Superior performance may emerge as a result of the reflective processes encouraged in some academic programs.
NASA has found that team assessments provide them with the best returns in terms of performance and capability improvement on projects (information about this is available on the NASA APPEL website).
Multiple Projects—Competencies and Skills
A sensible approach to assessment of project management competence, from an organizational perspective, is to use knowledge tests and self assessment (Aitken & Crawford, 2008b) as a first-cut assessment across the entire resource pool; performance-based assessments for a subset of more senior practitioners; and then behavioral assessments for those people seen to have potential as superior performers. Results can be aggregated to baseline competency profiles across the resource pool, and provide guidance for training needs.
A number of organizations encourage development of individual project management capability through internal accreditation processes and some of the larger organizations have established relationships with academic institutions to provide a range of development and assessment programs for more senior project personnel. There is growing recognition of the role of the program manager. The Project Management Institute has produced standards for both program and portfolio management (Project Management Institute, 2006a, 2006b) and offers a program manager qualification, the Program Management Professional (PgMP®). It is important to recognise that program management is not just a higher level of project manager. Many program managers come to the role as part of a general management rather than from a project management career path. The Office of Government Commerce in the UK was one of the first organisations to recognise the program manager role and produced a guide titled Managing Successful Programmes (Office of Government Commerce (OGC), 2007).
Using Assessment to Drive Project Management Capability Improvement
As the saying goes: “What gets measured gets done.” To maintain and improve capability, you first need to assess or measure your current capability. You need to know where you are. Using the guidelines previously provided you can determine which aspects of capability you will assess and how to carry out that assessment. Once you have a baseline you can develop a plan for improvement, initiate, and resource an improvement program. Benchmarking is important to ensure that your improvement targets are realistic, achievable, and relevant. They are also important to ensure that you aren't fooling yourself about your capability. Networking with other organizations can help you to identify improvements that have worked for others.
Assessing once is not enough. You need to have a regular program of assessment whether this is for single projects or programs, for the PMO, for individuals and teams or for overall project management capability. These assessments drive project management capability improvement providing evidence of success and targets for moving forward. As research into the value of project management has demonstrated, organizations that stop focussing on value, or believe that they are “done” stop demonstrating value, and the act of not enhancing value appears to destroy value (Thomas & Mullaly, 2008). Assessment demonstrates both value and improvement. A regular program of assessment keeps everyone on their toes and ensures that you keep your best foot forward.
