Certified Success
Would you ask your plumber to function as the prime contractor for your new office building? Would you ask the loans officer at the local bank to head your finance department? Not likely … yet for years the first place many organizations looked for an information management (IM) project manager was in the group responsible for the technical solution. Three years ago the success rate for large information management (IM) projects in the Department of National Defence (DND) was the same as the rest of government and industry—very low. Since that time we have improved our success rate substantially. One of the major contributors to this improvement is the development and certification program for project professionals. The program has proven that skill in the technical solution combined with training and experience in project management will greatly increase the opportunity for successful delivery. This paper describes the design, implementation, performance measurement and extension of the development and certification program for managers of IM projects valued at over $3M.
Project Management Problems
Problems in the delivery of IM projects came under scrutiny as a result of the publication of the first CHAOS report (The Standish Group International Inc., 1994). Public and private sector organizations realized their problems were not unique nor were they alone in the project challenges being encountered. Before 1999 as many as 25% of the large IM projects within DND were experiencing serious issues that were not apparent to senior management and thus were not being addressed by them.
The need for development and certification of project managers was evident in literature from both the public and private sectors. The Treasury Board of Canada Secretariat is responsible for establishing policies related to project management within the Canadian federal government. In establishing the Enhanced Management Framework (EMF), Treasury Board defined a requirement for project managers to have demonstrated the knowledge, skills, and experience commensurate with the project size and risk (Treasury Board of Canada Secretariat, 28 May 1996, 8). This is consistent with the various Standish Group CHAOS reports; in particular their identification of qualified and experienced project managers as a key contributor to project success (The Standish Group International Inc., 1999, 4). The Treasury Board requirement is also supported by the contention from academia that one of the four main factors contributing to project failure is the assignment of an untrained project manager (University of Portsmouth, Nov 2001).
ADM(IM)'s Solution
Within DND, the Assistant Deputy Minister, Information Management [ADM(IM)] is responsible for implementation of all IM projects valued over $3M. Presently there are 35 concurrent IM projects with a combined value of over $1.6B, making ADM(IM) the largest IM project management organization in the Canadian federal government.
In 1998 an ADM(IM) directed report identified deficiencies in IM project management in the areas of project management practices, human resource management, and portfolio management. In response, the director of information systems project delivery began the development of project delivery management (PDM). It is defined (DND/DISPD, 26 Oct 1999, 3) as a comprehensive discipline and is based on the Project Management Institute's (PMI®) A Guide to the Project Management Body of Knowledge (PMBOK® Guide). PDM is designed to increase the probability of successful delivery of individual IM projects as well as the overall portfolio of IM projects and is composed of the following business areas:
- Project oversight and portfolio management
- Interdependency management
- Knowledge management
- Professional development and certification.
The Certification Framework
We recognized early in the design phase of our certification program a need for a common project management methodology in order to ensure that practitioners would have a shared vocabulary and vision of the “how-to” of project management. PMI's PMBOK® Guide was the selected methodology as it was identified by Treasury Board as the basis for their Project Managers Handbook (Treasury Board of Canada Secretariat, Mar 1998, 30).
Exhibit 1. Project Manager Certification Framework
Having decided on a methodology DND's next step was defining a project manager certification framework. If the PMBOK® Guide was the chosen methodology, why not simply adopt the project management professional (PMP®) certification? The PMP certification requires demonstration of knowledge and experience or proof that the individual “has been there, done that and got the t-shirt.” PMP is a one-size-fits-all certification whereas Treasury Board defined three proficiency levels for project managers linking the proficiency level to project size. These levels were defined in terms of skills, knowledge and experience.
Our certification framework began with the Treasury Board tri-level model. The levels are defined by project dollar value and are based on education, project management training and experience. Using a tri-level framework permits early, formal recognition of individuals who have attained the basic necessary knowledge and experience requirements while also ensuring that the most costly projects are assigned to the most qualified project managers. Three levels of certification provide clear career progression opportunities within the organization. Once an individual completes the minimum level requirements he or she qualifies for both certification by the ADM(IM) as an IM project manager and progression to the advanced project manager development program. The certification framework and the two related development programs were introduced in 1999 and, effective 1 April 2002, all IM project managers must be certified as detailed in Exhibit 1.
The dollar value breakdown is a general guidance principle. High-risk projects, for example, are dealt with on a case-by-case basis. A high-risk project in the $3M to $10M range may require the project manager to meet the certification requirements normally associated with a $50M project.
Basic Program
The basic development program is targeted at those individuals currently employed in projects who have expressed an interest in a career as project managers. Funds for enrolment in the basic development program may be provided by the applicant's organization/project, by the basic development program budget or through a combination of the two.
Each year the manager of the basic program issues a call for nominations for individuals seeking full or partial sponsorship and each application must be endorsed at the director/project manager level. Applications are screened based on the individual's education, experience, present project role and future career plans. The development program manager's budget allows for sponsorship of up to 10 participants per year, which addresses growth and turnover.
The Training
The development program provides basic project management knowledge through formal training and project-related experience. Development of a custom program was never considered given the number of existing vendor offerings. The public offerings have the added benefit of exposing DND participants to the experiences of other private and public sector students. A request for proposal for training was developed with the following mandatory requirements:
- The program of courses must result in a certificate in project management approved by the senate of a Canadian university
- The training program must be PMBOK® Guide based
- The vendor must be a PMI® registered education provider
- The individual courses within the program must be recognized at a post-graduate level by a Canadian university
- The individual courses must be available to the public (i.e., not custom developed for DND)
- The individual courses must be delivered at a vendor site.
Exhibit 2. Training Satisfaction
A formal evaluation of all vendor responses resulted in contract award to CDI Education Corporation who deliver a Master's Certificate in IT Project Management. CDI is the Canadian partner of ESI International, who developed the program in cooperation with George Washington University.
The vendor requires completion of the program within three years. Individual courses vary from three to five days in length and each course is offered at various times during the year, permitting scheduling flexibility. Participants are encouraged to complete the program within 18 months, which requires an average of one course every two and one-half months.
Advanced Program
The advanced development program provides an opportunity for individuals who have completed the basic program to obtain PMP certification. Applicants must have demonstrated to PMI their eligibility to write the PMP certification exam. Funding for a PMP exam preparation workshop is provided by DND and the workshops are available from several vendors. Upon successful completion of the exam, the program reimburses the applicant's examination fees. Applications to the advanced program may be submitted at anytime and up to six sponsorships are available annually.
Results to Date
As of 1 April 2002 there are 34 certified project managers with 14 having completed the basic program and three having completed the advanced program. Of the remaining 17, 13 had previously obtained their PMP certification and four completed a course of study deemed equivalent to the basic program
Program Feedback
During the development of PDM the need to measure success was recognized and a Balanced Scorecard™ approach was developed. Feedback from the participants in the development and certification program was identified as one of the factors in the scorecard's organizational health measurement. As each person completes the basic or advanced program he or she is required to complete a short survey based on his or her level of agreement with three questions. The rating scale ranges from 1 (strongly disagree) to 5 (strongly agree)—see Exhibit 2. The questions and their cumulative average ratings follow:
- I would recommend the training to others (4.6)
- The training improved my job preparedness (4.1)
- The courses are worthwhile (4.5).
In addition, individuals earning ADM(IM) certification as an IM project manager are required to complete a comparable survey using the same rating scale; see Exhibit 3:
- I would recommend certification to others (4.5)
- I appreciate the recognition (3.9)
- The certification program is worthwhile (4.2).
Interviews with the Director General of IM Project Delivery and his four directors were conducted to determine their views on the effectiveness of the program. There was full agreement that project delivery success has increased as a result of the introduction of PDM in 1999. As previously mentioned, 25% of the projects experienced serious issues that were not apparent to senior management and thus were not being addressed by them. Currently we show no projects with serious issues and only two projects (less than 10%) as having a potential slippage in the baseline. Senior management intervention is not required but they are well aware of the situation and of the mitigation action plan. Development and certification is seen as a major contributor to this success.
Exhibit 3. Certification Satisfaction
The development and certification program has resulted in the following:
- Common understanding of project management terminology and processes
- Recognition, both by the client and the project management staff, of the group's determination to treat project management as a career and discipline thus improving credibility and morale
- Improved effectiveness in filling key project roles
- Increased level of self-confidence among project management staff
- Potential problems detected much earlier allowing proactive rather then reactive measures to be taken.
Lessons Learned
Four major lessons learned have emerged since 1999:
- While other organizations may hesitate to invest in training, selling the program to senior management within the ADM(IM) was not difficult. Senior management was already familiar with the various reports on causes of project failure and recognized the areas addressed by the development and certification program. Expressing the annual program cost as a percentage (2.5%) of the total annual budget for all IM projects reinforced the low cost.
- The intended audience must be provided with a number of opportunities to be briefed on the programs prior to the annual call for nominations.
- The manager of the program must become a course advisor. Simply providing documentation and website links for all the necessary information does not meet the needs of some individuals.
- Course selections and progression to program completion must be closely monitored. Some people must be reminded to measure the long-term gain associated with the training against a project's short-term requirements.
- Departmental recognition of an individual's efforts is important. The formal presentation of the ADM(IM) certificate provides this opportunity and was very much appreciated by recipients.
Building on Success
Following the establishment of the project manager development and certification program attention turned to providing career progression into the project manager field. The project control officer (PCO) role is seen as the primary stepping-stone on the path to becoming a project manager. PCOs develop, implement and maintain a control system to manage the project using an approved project performance management plan (PPMP). Responsibilities include:
- Preparation of a proposed PPMP for approval by the PM and the portfolio manager
- Development and management of the cost and schedule baselines
- Providing support and tools to the technical and functional teams in the accomplishment of their objectives including:
- Planning and budgeting all work
- Tracking actual work progress
- Tracking cost and schedule performance data
- Assisting in development of corrective action plans
- Reporting project cost and schedule status in relation to the approved baselines.
Exhibit 4. Project Control Officer Certification Framework
A development and tri-level certification framework was designed for PCOs using the same methods as the project manager certification framework however no existing training program was found. Consultations were held with existing DND and contractor PCOs and the set of skills required by a PCO was defined. These skills were then mapped to existing courses from diverse sources and the selected set of training courses formed the PCO development program. The selection of participants follows the same process as that for project manager development and funding is available for eight sponsorships per year. The PCO program began in April of 2001 and as of 1 April 2002 there are two certified PCOs and a further seven in training. Exhibit 4 details the PCO certification framework.
Conclusion
The CHAOS reports, Treasury Board, DND and academia referenced in this paper have identified the need for trained, experienced project managers. Over the course of the past three years we have:
- Implemented a certification framework for project managers based on experience, education and training
- Implemented a basic project manager development program and an advanced project manager development program
- Implemented a certification framework for PCOs based on experience, education and training.
- Designed and implemented a PCO development program.
There has been, over this same three-year period, a significant improvement in the success rate of IM projects. This improvement has been brought about through a disciplined approach to project delivery and the development of a cadre of project professionals who share a common methodology and training background—when you all sing from the same sheet the music is much sweeter.
The recognition of this training and certification program as a “best-practice” by other Canadian government departments, the implementation council for Treasury Board's EMF as well as the city of Ottawa's information technology project management department has served to reinforce the success of the program. The next challenge is working with DND's requirements definition organization to assist them in establishing similar training and certification programs.
As a final thought, whenever I am asked about the training costs and retention of employees, I answer with the following saying (source unknown): “you can train your staff and risk losing them or you can decline to train your staff and risk keeping them.”
