Obtaining a project management education
IN A SOCIETY CHARACTERIZED by the need for retooling the workforce and establishing continuous learning programs, it is important that companies offer their employees training and development opportunities. It also is important that they focus on selecting training opportunities that provide the highest return in terms of employee learning and satisfaction.
There are a variety of methods to teach introductory project management topics; however, results vary regarding the type of instruction received. I recently conducted a study comparing three training delivery modes—face-to-face instruction, self-directed learning using textbooks, and interactive multimedia software applications. The results show that face-to-face interaction with the instructor is the most effective. Learner satisfaction also is higher when there is proximity to the instructor. In terms of satisfaction with content, CD-ROM instruction presents engagement advantages and learner involvement that increase satisfaction.
Project Management Educational Delivery Modes and Rankings
A study conducted by Itzhak Wirth and Scott J. Amos [“Distance Learning for Project Management,” 1996 AACE International Transactions] listed over 20 distinct delivery methods of project management training programs. From their analysis, project management topics emerge as most teachable through face-to-face instruction, as opposed to mediated instruction.
Wirth and Amos rank the training methods on an intensiveness scale (from 1=low to 5=high), which they define as the set of delivery circumstances of the project management program. The items in their scale—instructor location, instructor proximity, instruction period, interaction frequency, interaction duration, and interaction type—rank the delivery mechanisms based on the type and amount of interaction, considered here as important learning and satisfaction elements.
From: Wirth and Amos 
Wirth and Amos create an intensity hierarchy that summarizes, within the same groupings, what features are most relevant for achieving program intensiveness (Exhibit 1). A five-day face-to-face training delivered by an instructor at the project premises ranks as the most “intensive.” Multimedia applications used at the company premises (that is, a corporate intranet) follow the face-to-face instruction by offering automated interactivity. Computer-mediated training programs within an academic degree program have a moderate intensiveness, particularly due to the one-way interaction with the instructional materials (one-way video or audio). Self-paced learning ranks as the least intensive.
Based on the Wirth and Amos classification, the highest rank order corresponds to interactive seminars (formal seminars with company-specific emphasis, manual/case study discussion; generally two-day duration), and experiential seminars (seminars involving discussion of training materials; topical emphasis on reviews and evaluation of project cycles; typical duration three hours).
Classroom instruction (lecture/discussion; individual project reports and case studies; topical emphasis on project management competencies; 16-week duration) ranks in the sixth position; closely followed by multimedia training programs (as many as 10 30-minute TV-episodes/textbook, with exercises consisting of computer-mediated multiple-choice tests; topical emphasis on generic project management) in the seventh place. Computer-based training (the automated version of programmed instruction by an interactive workbook for in-depth exploration of general project management topics) ranks 10th, the lowest position on the intensiveness scale.
The Wirth and Amos findings, although dealing specifically with program intensiveness and not with program effectiveness, are particularly useful to guide expectations on the comparative performance of the three delivery modes examined in this study.
Background for This Study
Earlier studies addressed the comparative learning effectiveness of computer-mediated vs. more traditional forms of instruction. However, several of these earlier studies suffered limitations mainly related to the noncomparability of the learning materials; for example, different instructors presenting different content.
My study provides a unique scenario that resolves many of the comparability issues. The study focuses on the teaching of identical content (scheduling tools in project management) authored and presented by the same instructor. J. Davidson Frame, the author of the textbook, was, in fact, the face-to-face instructor, as well as the video narrator in the multimedia CD-ROM.
I conducted this study on the basis of both a personal and a generally shared business need (an Educational Services Institute grant provided partial support). The study's goal was to identify how the recent educational trends in computer-mediated instruction compared with more traditional classroom systems and whether the investment in technology-based training offers comparable returns to the learner. The results presented here also are part of my dissertation work and constitute a portion of a study that is replicated on different learner populations. My goal is to build a framework for technology-supported education by identifying factors that influence learning and performance.
Methods of Instruction
The instructional inputs used in this study are comparable in content. They differ on the types of media used for conveying information, as well as the number of instructional objectives accomplished.
Textbooks. Frame's textbooks, on which the multimedia CD-ROM and the in-class instruction are based, are Managing Projects in Organizations: How to Make the Best Use of Time, Techniques, and People [Jossey-Bass Publishers, 1995] and The New Project Management: Tools for an Age of Rapid Change, Corporate Reengineering, and Other Business Realities [Jossey-Bass Publishers, 1994].
Both publications rank highly (based on number of sales and reviewers’ comments) in project management literature. They present similar design and layout features, address complementary topics, use diagrams and drawings to reinforce understanding, offer several examples, and occasionally include mini-case studies to foster reflection and application. The case studies present problems and offer solutions, to emulate a mechanism for feedback provision. The textbooks provide learning guidance and present stimuli material through examples, cases, and anecdotes.
Multimedia CD-ROM. Managing Projects in Organizations—The CD-ROM [ESI International, 1997] by Frame, with Carl L. Pritchard, is the multimedia version of Frame's two project management books and provides a stand-alone tool for learning project management introductory topics. The CD-ROM's units mirror Frame's chapter titles and structure. The software teaches project management fundamentals to aspiring project managers, project team members, executives, or other stakeholders, and is distributed as a single unit or with special licenses for corporate purchases. It is used as a support tool for distance learning, and users are expected to work with the software independently and at their own pace. The software includes practice tests and feedback features to self-assess the level of competency reached by using the application and uses video clips and audio to capture attention, simulate real-world scenarios, and provide feedback. Simulations reproduce officelike settings, while navigational aids accompany the user in each step. Contextual instructions appear consistently at the bottom of each screen. A notepad and calculator shortcuts offer easy access to supporting tools.
In-Class Instruction. The introductory project management course module (“scheduling tools”) used as a working focus in this study is developed from Frame's 1995 and 1994 books.
The face-to-face presentation is supported by the use of overhead transparencies, which display the graphical images of the textbook and develop additional graphical representations. The instructor primarily uses markers to write on transparencies during the presentation. Occasionally, display boards are used to support explanations. The instructor offers examples, asks frequent questions of the audience, provides feedback to responses, and encourages participation through in-class discussion of course problems. Frequently, cases and other exercises are completed in class, with the instructor's support.
Summary of Comparative Evaluations of Instructional Materials. Each media used in the study presents different advantages (as presented in Exhibit 2) typical of the nature of the medium.
A comparative “content and media quality” evaluation shows that the multimedia CD-ROM and in-class instruction are the two environments meeting the highest number of pedagogical and technical requirements, and, thus, are favored over self-directed learning using textbooks. The evaluation was conducted on the instructional materials used in this experimental study and was based on their compliance with established instructional objectives [Robert M. Gagné and M. David Merrill, “Integrative Goals for Instructional Design,” Educational Technology Research and Development, Vol. 38, No. 1, 1990]. The multimedia CD-ROM offers the variety of media and content representations. The in-class instruction is supported by visuals, communication skills, interaction strategies, and a delivery structure that keeps the learners engaged with the content.
However, learning using the self-directed textbooks is ranked as the least suitable to meet established instructional objectives. The texts are well organized and well written, but lack the interactivity and feedback features that characterize in-class instruction and multimedia CD-ROMs.
Subjects and Setting
The study's subjects are full-time master's of business administration students at George Washington University in Washington, D.C. All are learning scheduling tools such as PERT/CPM analysis; all belong to a cohort, and therefore are enrolled in the same section (for a total of three cohorts in three sections, with 40 students in each cohort). Each section used one of the three methods: face-to-face instruction, self-directed textbooks, or interactive multimedia software applications. The participants were assigned to learn scheduling-tools concepts, either as substitutes for the instructor's presentation of the topic or as an introduction to it (the method was assigned before the students learned about this topic in their regular coursework). Although the methods (which I call treatments) were randomly assigned, the subjects already were enrolled in a comparable class section, and, therefore, the study framework was quasi-experimental.
Each student took a pretest before the treatment and a post-test after the treatment. The tests were based on standard questions from various PMP® exam practice textbooks. Satisfaction and demographic surveys also were included in the experimental procedure to help answer two main research questions:
Which learning methods are more effective for learning project management topics (multimedia, textbook, or in-class instruction)?
Which learning methods (multimedia, textbook, or in-class instruction) are more appealing learning environments in learning project management topics?
To answer these questions, the study examines the three modes of learning (independent variables) and their effects on students’ overall test performance and satisfaction (dependent variables). Factors that can mediate the relationship and therefore influence the final results, such as student characteristics (gender, age, prior knowledge, computer abilities, and learning preferences) and subject characteristics (complexity of the topic) also are accounted for.
Learning Comparisons. There is indeed a statistically significant difference in learning and satisfaction about the instructional delivery methods, and there is a statistically significant difference in the learning improvements of each group.
The comparison of pre- and post-test scores used to measure content learning on three comparable groups, with an average of 40 respondents per group, indicates the face-to-face group obtained the highest post-test score, and at a 95-percent confidence level the difference is statistically significant. However, with a 90 percent confidence level, it can be said that the multimedia group had a bigger improvement than the textbook group.
Satisfaction Comparisons. The analysis of satisfaction results (based on a set of questions about satisfaction with content of instruction, organization of information material, learning experience, pace of instruction, and presenter) shows that the face-to-face group is still the most satisfied with the delivery method. Satisfaction with the instructional method is highest in the face-to-face group, when the learners actually meet the instructor. It is the lowest with the textbook instruction, when the learners have a lower level of personal contact with the instructor.
In terms of satisfaction with the pace of instruction, learners are less satisfied when they have more control of the pace, rather than following the presentation deployment of the instructor. While the face-to-face group was the most satisfied with their learning experience, the other groups reported comparable satisfaction.
Implications for Project Management Education
The preliminary results from this study show that the effectiveness of the delivery method is mostly related to factors such as proximity of instructor and the interaction opportunities of a face-to-face environment. In terms of learner satisfaction, the proximity of the instructor is also a factor of overall higher satisfaction with the presenter, the pace of instruction, and the organization. However, the study demonstrated that satisfaction with content is not necessarily higher in the face-to-face instruction.
ALTHOUGH ALL THREE METHODS were successful in providing a significant learning experience and satisfying the participants, face-to-face instruction remains the most effective means of delivery. The expansion of project management education using computer-mediated instruction (such as multimedia and the Internet) therefore needs to take into account that interaction and proximity are key elements for learners’ engagement. Interaction and proximity should be included in all project management training programs either directly, by resorting to traditional in-class workshops and seminars, or indirectly, by creating a mediated-delivery environment in which the instructor actively participates by using, for example, conference call, video broadcast, and synchronous communication. ■
Reader Service Number 030
Katia Passerini is a consultant in the Information Technology Group of Booz Allen and Hamilton. She specializes in multimedia applications and delivery over fixed and mobile networks; knowledge management; and information technology strategy. She is completing her doctoral work in information and decision systems at The George Washington University, Washington, D.C., USA.
October 2000 PM Network