Are there better ways to teach project management?

Abstract

As instructional approaches continue to mature and modern technologies effect change in the educational arena, those who design and deliver project management training are faced with the challenge of incorporating new approaches and technologies into an effective training strategy. This paper provides a case study on how the Johns Hopkins University Applied Physics Laboratory integrated several complementary methodologies into its PM training strategy. The strategy attempts to achieve an optimal blend that capitalizes on the relative strengths of each approach or technology to overcome the relative weaknesses of others, with due consideration to maintenance of the training and learning resources, as well as future developments that are likely to occur.

Introduction

Founded in 1942, the Johns Hopkins University Applied Physics Laboratory (APL) is a not-for-profit applied research and development organization dedicated to solving a wide range of complex problems that present critical challenges to the United States. Using advanced technologies, world-class facilities, hands-on operational knowledge, and a systems engineering approach, APL's 3800 employees achieve affordable, effective solutions for its customers on more than 400 projects of various sizes and duration (approximately $600 million total annually).

Given the variety and complexity of APL's projects, sound project management (PM) processes are essential to achieving the goals of APL's customers. One of the factors that play a critical role in ensuring the development and promulgation of sound PM processes throughout the organization is the effectiveness of the training and performance support provided to the Laboratory's project managers.

While APL has a generally excellent reputation for developing its project managers, the ever-increasing pace of change—particularly over the past decade—has increased the variety and complexity of APL's projects, while also demanding that project managers build and maintain their knowledge and skills more quickly. Mentoring and informal on-the-job training are no longer sufficient, and off-the-shelf courses don't provide the necessary knowledge and skills in organization-specific processes.

To provide improved learning and performance resources to the laboratory's project managers, APL's Training and Development (T&D) Office and Project Masters, Inc. (Ellicott City, MD), have collaborated during the past two years to develop an efficient and effective PM training strategy, taking advantage of lessons learned from previous training efforts at APL, insights gained through an organization-wide training needs assessment, and a solid foundation of Project Masters' educational resources.

The resultant APL Project/Program Managers (P/PM) Training Program is designed to provide an optimal blend of off-the-shelf and customized content in a flexible, multi-modal methodology aimed at meeting the learning needs of a wide variety of participants. This paper examines the design process from the initial assessment stages to the evaluation process, providing a road map for other organizations desiring to integrate optimally the latest instructional approaches and technologies into their PM training strategies.

Survey of Past and Current PM Training Approaches

In considering whether a better way of teaching PM exists, it is useful to first examine past and current PM training approaches and their relative strengths and weaknesses. This can help explain why these approaches—particularly when used alone—often prove less effective than expected in building the competencies that project managers need to be successful.

Most project managers learn the knowledge and skills they need through one (or more) of the following means:

1. Self-study resources (e.g., websites, textbooks, professional journals, policy and process documentation, etc.)
2. Project assignments
3. Mentoring by experienced managers and colleagues (either within one's organization or through one's professional contacts)
4. Off-the-shelf PM training courses—either instructor-led or online—designed to meet the learning needs of relatively broad audiences
5. Customized PM training programs developed by organizations (often, with the assistance of outside vendors) to meet their specific organizational needs

None of these methods are inherently better or worse than the others; they all possess particular strengths and weaknesses that should be—but all too frequently are not—accounted for in the design process.

1. Self-study resources can be vague and difficult to understand without mentoring or training.
2. Knowledge gained through one's project assignments may initially cost less than formal training, but when mistakes are made, experience can also prove an expensive and unforgiving teacher.
3. Mentoring and peer interactions can provide “real-time” information that more structured methods often cannot. However, such interactions can prove inadequate, particularly when “experts” are asked to mentor in areas where they lack expertise. Also, project managers may learn unconventional ways of getting things done, which may be undesirable when such methods conflict with what the organization requires.
4. Off-the-shelf PM training provides some degree of structure that mentoring does not and is usually less expensive than customized training. However, the process knowledge and skills provided by standardized training often does not reflect the organization's specific processes accurately and completely.
5. Training customized to meet the specific requirements of the organization may provide the ideal content, but can also be expensive to create and maintain.

Accordingly, multiple methods should be—and often are—employed, especially over the course of a project manager's career. An organization, for example, may rely on project assignments and mentoring to provide managers with certain knowledge and skills early in their career. Then, as the managers' learning needs evolve, off-the-shelf courses may serve to provide a more structured framework for general PM knowledge and skills. Finally, customized training may be used to meet the needs of more experienced managers for organization-specific process knowledge and skills.

APL P/PM Training Program Design Considerations

Before addressing the various elements of the APL P/PM Training Program, it is useful to detail some of the theoretical and practical design considerations that influenced the program's content and format.

Theoretical Design Considerations

On a theoretical level, the design of the APL P/PM Training Program was shaped by two primary influences: constructivist learning theories and the models-based approach to scientific theory.

Ernst von Glasersfeld articulated the rationale for constructivist architectures in learning, saying “knowledge is not passively received either through the senses or by way of communication; rather knowledge is actively built up by the cognizing subject” (von Glasersfeld, 1995, 51). Because participants have different levels of PM competency coming into the course, and because they play various roles within their projects, they bring to the program diverse needs. As a result, although some behaviorist strategies were utilized in the program (most particularly in the ways that basic PM terminology and concepts were taught and drilled), constructivist strategies had a stronger influence on the program's content and format. Lev S. Vygotsky took constructivism one step further by proposing that knowledge is not constructed alone, but is “social in nature and is constructed through a process of collaboration, interaction, and communication among learners in social settings” (Vygotsky, 1978, 35). This suggested that participants would be likely to learn more in a group than they would on their own.

The design of the program was also influenced by Ronald N. Giere's models-based approach to scientific theory (Giere, 2004), and, in particular, his use of the idea of a theoretical model's “fitness” to reality as being similar to the “fitness” of a map to its intended purposes (Giere, 1999, 25-26, 81-82, 214-215). Giere's emphasis on the use of “maps” was adopted in discussing models of projects (e.g., real-world examples and case study simulations) and their “fitness” to the real-world projects to be carried out. The level of concreteness that these approaches provide make learning PM tools and methods—and transferring what is discussed and learned back to the workplace—much easier.

Practical Design Considerations

In addition to the theoretical design considerations discussed above, there were a number of practical considerations that needed to be taken into account during the design process. These practical considerations came to light through (a) lessons learned from previous PM training and (b) a formal assessment conducted to determine the specific needs of the APL audience.

The APL P/PM Training Program is not APL's first attempt to develop project managers, and the lessons learned from previous attempts had a significant influence on both the current program's content and format. Incorporating these lessons learned helped to ensure that past mistakes were not repeated and encourage feedback from participants as to whether “fixes” successfully addressed the issues.

APL has been developing project managers since its founding in 1942, generally relying on project assignments, mentoring, university education, and ad hoc short courses conducted by outside and in-house experts. As a division of the Johns Hopkins University, APL has easy access to the university's engineering programs, which has offered a Project Management concentration in its Technical Management Program for many years.

Starting in the early 1990s, recognizing that a more structured program was essential to meet the needs of APL project managers, APL contracted outside vendors to teach multi-day PM courses. Course materials underwent minor revisions to reflect APL terminology and processes, and instructors taught the course multiple times, which meant that they were able to gain an understanding of APL terminology and processes and provide consistent instruction to each set of participants. In the end, these two experiences demonstrated that off-the-shelf PM training did not generally fulfill the need for APL-specific content and that it was highly desirable to have the same person or persons teach the course each time.

In the mid-1990s, APL also developed a highly customized PM training course, designed to meet the needs of more senior project managers. In spite of a great deal of effort being put into the course's development, it suffered from a number of fatal flaws and, for a number of reasons, was terminated after the first half was piloted. First, because of APL's highly decentralized structure at the time, it was nearly impossible to provide enough detail on APL-specific PM processes to be useful, while remaining general enough to be taught to a group of managers from across the Laboratory. Also, the course was taught in two-hour sessions, one session each week over fourteen weeks. This proved too long an interval for most participants to remain focused on the learning process, and most participants missed one or more sessions. Finally, the discussion of PM processes tended to be descriptive, rather than prescriptive, leading to repeated questions about how processes ought to work, all too often without useful answers being provided.

In addition to these lessons learned, practical design considerations were also discerned through an organization-wide training needs assessment conducted by the T&D Office using a focus-group methodology. Two separate focus group meetings were held, one to address the needs of new and prospective project managers, and a second to address the needs of more senior project managers. While it was initially thought that these two groups had divergent learning needs, the focus groups seemed to indicate that this was not the case.

In sum, the needs assessment and lessons learned provided input on or validation of a number of practical design considerations that indicated that the program should:

1. Address the needs of project managers at all levels of responsibility and managing all types of projects (i.e., large or small, long or short term, products or services, hardware or software, etc.). The reasons for this included (a) the need to generate adequate “throughput” to support holding the program several times each year, and (b) the fact that managers were often promoted on short notice to different kinds of projects or to positions with greater responsibility. (It was understood, however, that certain audiences—such as very senior managers, those managing certain types of projects, etc.—had distinct learning needs that, at least for the time being, would remain outside the scope of the program being developed.
2. Promote and encourage standardized processes and tools with an optimal blend of description and prescription, as well as best practices from other organizations. For this reason, designers started to focus on the Project Management Institute (PMI^®) and A Guide to the Project Management Body of Knowledge (PMBOK^® Guide).
3. Provide detailed coverage of assessing and managing risk, as well as how to communicate risk considerations to customers, due to the nature of much of APL's work (i.e., applied research and development, critical outcomes, ambiguous scope descriptions, and short timelines).
4. Incorporate APL-specific content, particularly on organizational-specific business systems and processes.
5. Modularize the instruction to provide sufficient flexibility as to when participants attend a topic, thereby better fitting their busy schedules.
6. Cultivate a wider PM community to support a continuous learning process.
7. Draw on a consistent instruction team comprising outside experts, particularly for general topics, and in-house experts, particularly for APL-specific content.

Program Elements

In its current configuration, the APL P/PM Training Program comprises six elements, designed to provide a comprehensive solution to meet the needs of APL project managers. Before starting the program, participants attend a brief orientation that describes the content, format, and purpose of the program elements. They also are asked to reflect on and discuss their own learning needs, so that the course can be adjusted to accommodate those needs to the extent possible.

Online PMBOK^® Guide -aligned PM courses (SkillSoft Corp.)

APL licenses online technical and business courses from SkillSoft Corporation (Nashua, NH), including their Third Edition PMBOK^® Guide -aligned curriculum. These online PM courses support the APL P/PM Training Program in several ways. First, they are used to help prospective participants assess their readiness for the program. Secondly, they are used as pre-work to ensure that all participants begin the program with a basic understanding of PM concepts, processes, and tools, since some basic material is covered only briefly in the instructor-led portions of the program to preserve time in the classroom for more advanced content. Finally, the online courses contain a substantial body of content that is outside the scope of the instructor-led segments and consequently play a key role in follow-on training. While many participants fail to fully utilize this resource, it does play an important role in “leveling the playing field.”

Instructor-led PMBOK^® Guide -aligned PM course (Project Masters, Inc.)

The program's foundation is the four-day APL Project Management Fundamentals course, which was developed from Project Masters' PMBOK^® Guide -aligned instructional materials and customized over time to meet the needs of the APL audience (e.g., APL-specific processes and terminology). While all nine PMBOK^® Guide knowledge areas are covered to some degree in the course, the primary focus is on managing project scope, time, cost, and risk. This design choice incorporates the traditional primary constraints of scope, time, and cost management, but also an organizational need—uncovered during the needs assessment—to provide substantial coverage of risk management.

The Fundamentals course is taught over a period of two weeks by one of Project Masters' highly experienced instructors. Holding it over two weeks provides a compromise solution that balances the needs of those who believe that the course would be most effectively taught in four consecutive days—thereby reducing the opportunity to forget material between sessions—and those who would prefer for it to be taught one day each week over four consecutive weeks—thereby reducing the impact of the course on their work schedules. It also allows the participants some time between sessions to reflect on how what they are learning in class applies to their work, as well as to confer with their mentors.

During the course, a case study simulation provides a framework for applying new knowledge and skills, as well as for demonstrating how MS Project can be used to manage projects effectively. Those who want to delve more deeply into MS Project and its features can take a supplemental three-day MS Project course, participation in which is restricted to those who have completed the Fundamentals course.

APL-Specific Content Sessions

As discussed above, APL's previous experience with PM training demonstrated the critical role APL-specific content plays in bringing about a successful learning outcome. Therefore, after attending the APL PM Fundamentals course, participants are expected to attend several APL-specific content sessions taught by APL subject matter experts in half-day sessions. Some of these sessions elaborate on PMBOK^® Guide knowledge areas:

1. Project Human Resource Management
2. Project Procurement Management
3. Project Risk/Quality Management (which includes Environmental Health and Safety)

Other sessions cover PM-related processes and tools:

1. APL Program Financial Management, which covers the financial management of project at APL
2. Security (including classified material handling)
3. Business Development and Contracts with the Federal Government
4. APL Resource Management Information System for Project Managers
5. Export Control

This modularized approach allows participants to attend the APL-specific content sessions with the other members of their Fundamentals course, as is preferred, or during a future offering, if necessary due to scheduling conflicts. Participants can even attend sessions more than once, if they so desire.

Online APL PM Community (Ensemble Collaboration, Inc.)

A theme that emerged during the needs assessment was that project managers generally recognized that, to a large degree, their effectiveness was due to how extensive their professional network was and how capably they tapped into it. In addition, they understood that their learning process was a continuous one, and not one that could be dealt with in a single course. These perspectives are supported by Etienne Wenger's research on cultivating of “communities of practice.” “Although our experience of knowing is individual, knowledge is not” (Wenger, 2002, 10).

To support these requirements, APL licenses Collaboration On Demand from Ensemble Collaboration (Fredericton, NB) to help project managers connect online to ask questions and share resources within the context of the training program or their work. While there are a growing number of applications available to support collaborative learning, one of the benefits of Ensemble's approach is that it allows proprietary or other sensitive information to reside inside APL's firewall, while the hosted collaboration space resides outside the firewall. This gives APL added flexibility by allowing project managers to access the full set of resources—both open and proprietary—when operating inside APL's firewall, and yet still access many of the resources—albeit not those that are sensitive—when operating outside the firewall (e.g., while on travel).

While APL has a large community of project managers knowledgeable in using technology to communicate, it is also a decentralized community with little experience in engaging the entire PM community on a large scale. Therefore, starting an online community has taken longer than expected, and the community is still in its infancy.

Mentoring

Due to the number of different customers and projects that APL has, a critical part of any PM training approach is mentoring, which helps participants apply what they are learning to their particular projects and customers. For example, in learning to manage project communications, project managers consider the specific steps to be taken to establish sound project communication plans with their particular customers.

Because of the important role that mentoring plays, participants are asked to assess what areas they need mentoring in and then determine who can provide mentoring. In some cases, they are able to do this on their own; in other cases, they request the assistance of a supervisor or peer. The mentoring process is expected to start during the Fundamentals course, but may continue through the APL-specific content sessions and beyond.

Textbooks

Each participant receives a copy of A Guide to the Project Management Body of Knowledge (PMBOK^® Guide), Third Edition, which serves three purposes:

1. A “read-ahead” resource to assess readiness and prepare for the four-day course
2. An instructional resource during the class
3. A reference resource after the class

In addition, some participants have learning needs that fall outside the program's design learning objectives. When these needs can be effectively addressed through textbook reading (and perhaps some mentoring), a participant may request that the T&D Office procure an appropriate textbook. In some instances, the participant retains the textbook, while in others, the textbooks are retained in the Laboratory's Staff Development Resource Library (thus made available to other employees). In all instances, though, the participant is asked to provide a review of the textbook, which is posted on the online PM Community for future participants to use in determining the textbook's appropriateness as a resource for them.

Program Evaluation

The T&D Office conducts numerous evaluations of the APL P/PM Training Program and its elements. The evaluation strategy draws on Donald Kirkpatrick's classical model for evaluating training, which suggests four progressive levels of evaluation: reaction, learning, behavior (i.e., performance), and results (i.e., ability to meet business objectives) (Kirkpatrick, 1987, 302). The use of multiple evaluations is designed to obtain immediate feedback about the program (i.e., reaction), while also seeking information about the program's long-term effectiveness, both in meeting the program's learning objectives (i.e., learning), as well as improving the performance of APL's project managers over the long-term (i.e., behavior). Participant feedback has also resulted in changes to the program's content and format. For example, participant feedback led to a re-design of the course to incorporate small group exercises using MS Project.

During the Fundamentals course, each participant receives an evaluation form requesting feedback on the content and format of the course (including the instructor and training materials) and on how well individual learning needs have been met. In addition to the information gathered through the evaluation forms, short verbal quizzes and practical exercises (including the case study simulation) are conducted during the course to confirm that learning objectives are being met. Participants are asked to complete similar evaluation forms for each APL-specific content session they attend.

One year after they begin the program, participants are asked to complete an overall evaluation, including feedback on each element of the program. By that time, they are supposed to have completed all of the program elements, so this feedback is designed to provide a longer-term perspective on the program's effectiveness.

Evaluations for the first eight sessions have been encouraging. Most participants say that they have learned a lot—both on conceptual and practical levels—from the program. Several program elements and features have received special note:

1. The prescriptive nature of a PMBOK^® Guide -aligned Fundamentals course has proven helpful in outlining how projects ought to be organized and managed. In addition, the incorporation of MS Project in the practical exercises has been viewed positively.
2. The concern about participants having different levels of PM competency coming into the program has turned out to be less of a problem than anticipated. Even senior managers with many years of experience appear to have benefited from the more formal and structured program of learning.
3. Participants have not attended the APL-specific content sessions as widely as expected. It appears that one reason may be that the flexible schedule is too open and accommodating, thus encouraging participants to put off attendance.
4. The Online APL PM Community remains a little used resource. This is not unexpected, however, due to the nature of online community building. Some suggestions offered by participants (for example, using the collaborative workspace for the program orientation to familiarize participants with its operation and features) may prove useful for increasing participation in the future.

Summary

Has APL's P/PM Training Program been a success? Perhaps the most accurate gauge of its effectiveness is how quickly each offering fills up, how many new participants relate that previous participants recommended the program to them, and how quickly the PMBOK^® Guide is starting to provide a common language and structure for discussing PM practices at APL.

What does the future have in store? Without a doubt, the program is not a finished product—and probably never will be. Certain program elements—such as the online PM Community, the APL-specific content sessions, and mentoring—will require more work to help them attain their appropriate level of effectiveness. But, as the program continues to mature, course content—particularly in the APL-specific topics—will continue to grow and evolve. And, project managers are beginning to create and share resources, as well as develop training in topics in which they have expertise.

And, there is reason to be excited by future developments in technology and organizational development, which promise to keep change a central feature of the program. Mobile business technologies—such as cell phones, wireless networking, and tablet notebook computers—will certainly affect how project managers choose to gain access to learning and performance support. And, trends in knowledge management and the emergence of more powerful enterprise-wide applications will influence how organizations maintain their enterprise knowledge base.