Education and training of program management personnel in DOD

INTRODUCTION

Legislation, DOD directives and customer desires are promoting dramatic changes in the education and training needs for program management in the Department of Defense. The Defense System Management College (DSMC) response to this change initiative is described. The first section reports on the new career development program for acquisition personnel with its strong emphasis on education and training. The second section discusses the most extensive course in DOD on project management, the Program Management Course. The third section is an analysis of a recent survey of acquisition personnel regarding the most important competencies in program management education.

DEFENSE CAREER DEVELOPMENT PROGRAM FOR ACQUISITION PERSONNEL

Responding to the needs of the Department of Defense (DOD) and the requirements of congressional legislation, the DOD has implemented a directive [1] and manual [3] which establish a career development program for acquisition personnel. The program establishes specific acquisition workforce position categories and career fields with specific standards for education, training, and experience in each of these fields. Proficiency of DOD acquisition personnel is ensured through a certification program. The program provides career paths for the acquisition workforce in terms of education, training, experience and assignments necessary for career progression of civilians and military members of DOD.

The structure of the program consists of twelve career fields under the cognizance of seven functional boards as shown in Figure 1. The functional boards develop the career field requirements in education, training, and experience. The program requires basic, intermediate, and senior levels for each of these career field categories. The functional boards establish mandatory and desired education and training for each level. All mandatory courses will be centrally funded and the funds administered by the Undersecretary of Defense for Acquisition. Beginning in FY 93, the newly formed Defense Acquisition University (DAU) will allocate the funds for mandatory courses in acquisition management to the service colleges. DAU will provide coordination of acquisition management education and training at all DOD Schools and Colleges.

In addition to the formation of DAU, the DOD has instituted the concept of an acquisition corps with its members intended to compete for critical acquisition positions. The agencies within DOD establish a process to determine whether an individual has met all the standards established for the position in an acquisition corps. There will be four acquisition corps, one for each service and a fourth to include the Office of the Secretary of Defense (OSD) and all other defense agencies. Each of these acquisition corps are in the formative stages but by 1 October 1993 all critical acquisition positions, both military and civilian, must be filled by members of an acquisition corps. The selection criteria for acceptance to each acquisition corps involves the grade level, a minimum number of years of acquisition experience, and the training and education of the candidates. Each acquisition position will have a certification standard established and personnel assigned must meet the mandatory requirements, thus placing a critical need for career planning that will provide qualified candidates for acquisition positions.

Career pathways for progression are provided both vertically to higher levels and horizontally between career fields. The program encourages individuals to be certified in more than one career field and provides typical career paths for both military and civilian personnel. For civilians, the career planning is done between the individual and supervisor in an individual development plan (IDP) that looks at both short-range (one-to-two years) and long-range (three-to-five years) goals and provides developmental objectives and activities to meet these goals. For the military, a similar planning process exists but is not documented in an IDP.

One of the 12 career fields in acquisition is program management. The program management field requires experience and training in more than one discipline and at more than one level as it involves the integration of the various acquisition disciplines. Program management is not limited to program manager, deputy program manager or program executive officer positions, but includes acquisition positions that are cross-functional and are designed to provide an integrative system perspective. The important aspect in defining these program management positions is the cross-functional integration of information. The requirements for the three levels of the program management career field are summarized in Figure 2. A Program Management Functional Board acts in an advisory capacity to the Undersecretary of Defense for Acquisition and recommends mandatory training for the career path, including required course content to meet the mission need. The chairman of the Program Management Functional Board is the Commandant of the Defense Systems Management College.

Figure 1. DOD Acquisition Career Fields by Functional Board

Figure 2. Career Path Program Management

The program management career path also has statutory requirements for program managers and deputy program managers of major and significant non-major acquisition programs and for all program executive officers. This is required for implementation of the Defense Acquisition Workforce Improvement Act (DAWIA). For program managers and deputy PMs of major defense acquisition programs, three major requirements have been put in place by DAWIA requiring (1) completion of the Program Management Course at DSMC, (2) eight years acquisition experience of which two were in a systems program office, (3) and a signed written agreement to stay on the job through completion of the milestone closest to four years. With the implementation of DAWIA, program managers and deputy PMs of significant non-major programs are required to have a minimum of six years of acquisition experience and to attend the Program Management Course (PMC) at DSMC. Thus, while PMC is only shown as desirable for level III of the program management career field, a pool of candidates for PM and deputy PM jobs must have taken PMC.

PROGRAM MANAGEMENT COURSE

The Program Management Course is the premier acquisition course in the Department of Defense and has been in existence since 1971. This course is given at the Defense Systems Management College at Fort Belvoir, Via, and is a five-month course given twice a year. Having more than doubled in size during the last decade, there are currently 420 students per class.

The PMC is designed for middle managers from all of the acquisition disciplines and is intended to increase their ability to successfully manage activities of a defense systems acquisition program. The course is taught from the program manager's point of view with the emphasis on integration of the functional disciplines. The students come from all agencies within DOD having systems acquisition responsibilities as well as from other government agencies and the defense industry. The students have an opportunity to interact with current program managers and senior officials of DOD and the defense industry. The course addresses all phases of the program life cycle through development, production, post-production support, and product improvement.

The PMC objectives [4] require that each student be able to:

• Identify, analyze, evaluate, act upon, and resolve acquisition tasks, problems, and issues.
• Understand the roles, activities, and relationships of government and industry organizations that participate in and affect the acquisition process.
• Understand the concepts, principles, language, and integration of the functions which must be performed in acquisition management.
• Develop the resource management skills needed to effectively and efficiently utilize people, money, facilities, information, and time in the accomplishment of acquisition objectives.
• Recognize environmental factors which influence and constrain the acquisition process and foster an understanding of how such factors can be dealt with considering the uncertainty and change associated with the management of systems and equipment acquisition and modification programs.
• Deal with others through an increased understanding of human behavior with emphasis on leadership, team building, integrity, and ethics.
• Apply the skills of oral and written communications essential to planning, organizing, directing, controlling, reporting, and documenting acquisition tasks, actions, and issues.

The emphasis in the classroom is on sharing of the acquisition management experience of the students, whether it be in a lecture/discussion, case study, or simulation. The simulation and integrated subject areas are intended to cross multiple functional areas and require integration across a number of individual areas. The 13 functional areas are:

1. Acquisition Policy and Environment
2. Contractor Finance
3. Contract Management
4. Cost/Schedule Management
5. Funds Management
6. International Program Management
7. Logistics Support
8. Managerial Development
9. Manufacturing Management
10. Principles of Program Management
11. Systems Engineering
12. Software Management
13. Test and Evaluation

An important integrating function across these functional disciplines is the Industry Program where each section of 30 students is assigned a specific acquisition program. The students become familiar with the specific weapon system through study of program documentation and participation in meetings with both the government and industry program managers. Once the preparation is complete, the students visit the contractor's plant where interaction between contractor employees and students enables an understanding of the current management of that program. The industry program provides another bridge between the academic environment and an actual acquisition program.

The PMC is based on a set of 23 capstone competencies from PMC 92-1 [5]:

1. Able to develop an acquisition strategy that is the framework for integrating functional activities essential to fielding a defense acquisition program and is the basis for developing specific program management documents.
2. Comprehends how to translate operational requirements into design requirements which are explicit functional quantitative ranges or point values of performance and is able to do a trade-off analysis.
3. Comprehends how to analyze an escalating threat relative to an existing system to determine viable courses of action.
4. Able to analyze the technical adequacy of the existing design to meet known technical requirements.
5. Able to review and evaluate program documentation to ensure that the needs of an individual program are consistent with sound business practices, common sense, and the degree of risk involved.
6. Comprehends the processes of cost and budget reviews and how they interrelate with the program review/decision process and associated reporting. Able to prepare and analyze a program cost estimate and POM/budget input.
7. Able to develop, analyze, and update performance, schedule, cost, and program baselines.
8. Able to analyze contract cost/schedule performance data to determine contract and programmatic impacts and recommended actions.
9. Able to assess a program's readiness to exit from a given phase of the acquisition process, pass the required review, and enter into the next phase.
10. Able to prepare a solicitation that effectively communicates the government's requirements, acquisition strategy, and evaluation criteria.
11. Comprehends both the competitive and noncompetitive award processes. In the noncompetitive process, able to apply the techniques of pricing, fact-finding, evaluating cost and technical proposals, and negotiating from both a government and contractor perspective.
12. Comprehends the financial and technical systems, processes, and practices used by defense contractors to manage weapon system acquisitions, including the motivations and constraints in their implementation.
13. Comprehends how to develop and implement solutions to management issues associated with international considerations in defense programs.
14. Comprehends the philosophy, techniques, and tools of total quality management and how total quality management should be applied in the program office and the contractors' organizations.
15. Able to apply quantitative and qualitative methods to analyze an issue and present a decision briefing to higher authority.
16. Comprehends how congressional activities impact acquisition management.
17. Able to formulate tailored, human-skills approaches based on individual differences, the interpersonal communication process, and small group dynamics to maximize interpersonal effectiveness in the program management environment.
18. Comprehends how to identify and plan for the personnel and functional management support required to execute the acquisition strategy of a defense acquisition program.
19. Comprehends how to develop, document, and staff a test and evaluation program in DOD.
20. Able to analyze program office and contractor status/plans for mission critical computer resource (MCCR) development, integration, management, and support.
21. Comprehends the DOD logistics policy and related management tools, such as logistic support analysis (LSA), the logistic planning and contracting processes, and how these should be applied to program office and contractor logistic efforts during the acquisition life cycle.
22. Comprehends the principles of manufacturing management and how these principles should be applied to influence the design process, transition to production, and execution of the manufacturing plan. Able to analyze program office and contractor status/plans for transition to production.
23. Able to develop plans and review program status for deployment.

Figure 3. Capstone Competency Importance

These capstone competencies flow down to specific desired learning outcomes in each of the functional areas and integrated activities. Desired learning outcomes into the capstone competencies provide the primary areas of learning and are augmented by experiential learning outcomes developed by senior faculty teams.

SURVEY OF PMC GRADUATES AND SUPERVISORS

During late 1991 and early 1992, DSMC and the services conducted a survey of graduates of PMC between 1988 and 1991 and their supervisors. The survey was designed to gather data on the utilization of PMC graduates in acquisition-related assignments, including the type of assignment and the on-the-job performance of the graduate. The study was also designed to gather information on the relative importance, as perceived by the graduates and supervisors, of the 23 capstone competencies on which the PMC course Curriculum is based. The survey addresses 1954 graduates from January 1988 through mid-1991. Sixty percent responded. The supervisory response rate was 46 percent.

The survey showed significant improvement in utilization of graduates in acquisition-related assignments compared to previous studies. About 98 percent of the 1988-1991 PMC graduates were utilized by the services in acquisition-related assignments. The data shows 28 percent of the graduates working as program managers or deputy program managers, 35 percent of the graduates in Program Management Office (PMO) positions or matrixed to the PMO, and the remainder largely utilized in staff positions at all levels in the acquisition organizational structure. The data further showed that performance of the graduates exceeded expectations.

The data on capstone competencies showed the relative importance of each of the 23 competencies. In addition, the respondents were ask to provide the level of learning believed to be appropriate for utilizing the terms “Know,” “Understand,” and “Able to Do” based on Bloom's Taxonomy. The following concentrates on the relative importance aspect of the survey.

The PMC survey of graduates and supervisors was supplemented by interviews with senior acquisition leaders again assessing the importance of the competencies. Identical questions were asked of 41 senior program management leaders in DOD, again obtaining the relative importance of the capstone competencies. For the senior acquisition leaders, the survey was conducted using personal interviews and a more open format than the survey questionnaire used for the graduates and their supervisors. A comparison of the relative importance of the 12 highest ranked competencies as ranked by each of these three groups is shown in Figure 3. There were significant breaks in the importance percentile between the 11th and 12th ranked competency for graduates where the delta percentile was 4.5 percent. For the senior acquisition leaders and the supervisors of the graduates, the significant breaks were between the 10th and 11th ranked competencies where the delta percentiles were 3 percent and 5 percent, respectively. Thus Figure 3 isolates on those competencies judged to be most important out of the 23 previously listed.

Looking at the top rated competencies, there appears to be good correlation between the responses of the graduates and the supervisors of the graduates where the first nine as ranked by the supervisors are included in the first 11 as ranked by the graduates. An overview of the top ranked competencies also shows significant differences between the rankings of the senior acquisition leaders and that of the graduates and supervisors. It should be remembered in making these comparisons that the senior acquisition leader survey was conducted in an interview format as opposed to the questionnaire provided to the graduates and the supervisors of the graduates.

Some of the interesting differences between the three groups on individual competencies. The competency on developing an acquisition strategy (1) is ranked number 3 by both the graduates and supervisors and number 18 by the senior acquisition leaders. A possible reason is the senior acquisition leaders may regard the development of an acquisition strategy as an area that will have significant review in the acquisition management organizational structure and significant help available to the individuals developing the strategy. Another competency where there was a marked difference in the ranking was number 8, “analyzing of the contract cost/schedule performance data.” This competency was rated 1st in importance by the senior acquisition leaders, only 6th by the supervisor, and 11th by the graduates. A reason for this discrepancy may be that the survey of senior acquisition leaders took place in 1991, when the cancellation impact of the A-12 Stealth Navy Attack Aircraft Program was a major concern to the administration and the Department of Defense. This program was canceled primarily because of inaccurate cost/schedule program status reporting and failure to accurately report it to the highest levels of DOD and Congress. Because of the importance of this event in 1991, it is likely that this competency was significantly increased in importance by the senior acquisition leadership.

While there are some significant differences between the ratings, three competencies were all ranked in the top four of each of the three rating groups. These were number 6, processes of program financial management; number 17, applying management and leadership skills; and number 15, presenting a decision briefing to higher authority This strong consensus among the graduates, supervisors, and senior acquisition leaders make these three exceedingly important to a project management curriculum. Note that only the program financial management competency is largely knowledge-based while the other two have primarily to do with developing skills, whether leadership skills as in competency 17 or decision briefing skills as in competency 15. The challenge of improving leadership skills in the educational environment is a significant one. In the PMC, the approach is to make each individual student aware of his own skills through individual assessment instruments and feedback in small group activity For decision briefings, the approach is to provide the structure and elements of a good briefing and to encourage practice with feedback by peers and faculty.

The correlation between the ranking of the competencies for each pairing of the groups (graduates versus supervisors; supervisors versus senior acquisition leaders; and senior acquisition leaders versus graduates) provides another indication of the similarity of rankings. The Spearman Rank-Order correlation coefficient, rs, was used to determine the correlation between each of these three pairs. As indicated by looking at the data in Figure 3, the correlation between the graduates and the supervisors of the graduates is the greatest with an rs, = +0.81. Somewhat surprisingly, the next highest correlation is between the graduates and the senior acquisition leaders where rs = +0.64, while between the senior acquisition leaders and the supervisors the correlation coefficient is +0.46. Because of the difference in methodology of survey for the senior acquisition leaders and the 46 percent response rate for the supervisors, there is some question about the validity of these statistical correlation figures.

The information from the PMC graduate study and response to DOD Acquisition Career directives will assist DSMC in improving the PMC curriculum. This also has implications for other project management education. A similar process is used for other DSMC courses. The entire study is planned to be published in a future issue of the PM NETwork. It may be obtained by requesting reference 6 from DSMC.

ACKNOWLEDGMENT

The PMC graduate study and this paper benefited from the following personnel at DSMC: Mr. Robert Crittenden; Dr. Alan Beck; Lt. Col. Terry Raney; Lt. Col. Carl Bryant and Ms. Tina Richards.