Case study in developing federal organizational project management capability

An Overview of a Performance Driven Organization

Since the VA Acquisition Academy (VAAA) opened in 2008, it has repeatedly been recognized as an industry leader in training and development for the Federal Acquisition Workforce.

VAAA was created to address the growing acquisition workforce challenge facing the Department of Veterans Affairs and the Federal Government overall. With the downsizing from the last decade and the loss of institutional knowledge resulting from baby-boomer retirements, the acquisition workforce has been strained to keep pace with the increased amount of and complexities associated with outsourced work in support of the VA mission.

In addition, oversight of the acquisition function across the entire Federal Government has grown dramatically in recent years. All of this, combined with recent mandates by the Office of Federal Procurement Policy (OFPP), as implemented by the Federal Acquisition Institute to meet required certification standards for the acquisition workforce, prompted the VA to take action to build the VAAA.

Through our professional schools — Program Management, Acquisition Internship, Contracting Professional — the VAAA is committed toward not only meeting OFPP certification mandates, but also toward ensuring that our workforce development efforts take a holistic and synergistic approach to:

• Create a workforce that is regarded as “best in class” where acquisition professionals are referred to as “trusted business advisors” who are sought out by their peers and customers for their expertise in crafting the most effective and efficient acquisition strategies to meet mission needs.
• Incorporate electives in each school that addresses the entire suite of competencies needed for success: innovation, leadership, team building, communication, interpersonal skills, and others.
• Align course content to achieve greater consistency and standardization in performing the acquisition function that is grounded by the Federal Acquisition Regulation's (FAR) Guiding Principles for exercising sound business judgment.
• Leverage institutional knowledge and wisdom of the VA's existing acquisition workforce by creating knowledge sharing activities and opportunities for all employees coming through our programs.
• Bridge the gap that often exists between program and contracting offices to create “acquisition teams” that are committed to collectively achieving results.

The Impetus

As greater numbers of troops return from deployment, the VA must proactively improve and expand its service delivery capabilities to America's military personnel. Secretary Eric Shinseki launched several major initiatives in response, including but not limited to: Eliminating Veteran Homelessness, Enabling 21st Century Benefits Delivery and Services, Automation of the GI Bill, VLER, and several critical initiatives.

Rather than risk the assumption of their successful implementation, Deputy Secretary Scott Gould initiated a long-term investment in 2010 to improve the capability of the VA acquisition workforce to support their successful delivery and improve the organization's capabilities in effectively managing those taxpayer funds that have been appropriated to fulfill the above-mentioned investments.

The Mandate

April 25, 2007, former Administrator, Office of Federal Procurement Policy (OFPP), Paul Denett, outlined that ‘well-trained and experienced program and project managers are critical to the acquisition process and the successful accomplishment of mission goals. A strong partnership between program and project managers and contracting professionals requires a common understanding of how to meet the government's needs through acquisitions that deliver quality goods and services in an effective and efficient manner. This memorandum establishes a structured development program for program and project managers that will improve this partnership and our collective stewardship of taxpayer dollars.’

As a result, the Federal Acquisition Certification for Program and Project Managers (FAC-P/PM) was established to provide guidance on general training and experience requirements for program and project managers in civilian agencies. The FAC-P/PM model outlined essential competencies needed for program and project managers.

The Audience

An estimated 5,000 VA acquisition professionals will participate in the Project Management Improvement Program (PMIP), working collaboratively to broaden the organization's capabilities. In making this investment, VA anticipates that as sound processes and practices are identified, shared, honed and repeated over time, the resulting momentum will transform the agency's performance capability.

These acquisition professionals oversee some of the most important initiatives within the Department, aside from the Secretary's major initiatives — some of which are joint ventures with other major agencies, including DoD and HHS — VA acquisition professionals are working on streamlining and amplifying the IT infrastructure, overseeing major hospital activations, and veteran-facing initiatives to improve patient service and care.

The VAAA Solution

VAAA Program Management School embarked on establishing a PMIP to address the challenges that both Secretary Shinseki and Deputy Secretary Gould were attempting to mitigate.

The focus of PMIP was to not only enhance the existing skill sets of the acquisition workforce, but to also establish business acumen, create alignment of technical and process-oriented acquisition management procedures, and reinforce the value of enterprise-wide collaboration during project execution.

The VAAA vision to improve the capabilities of the VA acquisition workforce wasn't narrowly focused simply on a small segment of the population. If PMIP was to accomplish its goal of ‘creating alignment of technical and process-oriented acquisition management procedures and reinforce the value of enterprise-wide collaboration during project execution,’ the scope of the training initiative had to include both new recruits, as well as seasoned acquisition professionals.

The FAC-P/PM memorandum outlined a three-tiered model — Entry, Journeyman and Senior/Expert. The PMIP program instituted by the VAAA based its FAC-P/PM training model directly from the OFPP guidance as outlined — a three-tier competency structure, with each focused on developing the prescribed competencies: program management, acquisition, cost estimation and earned value management, and all the required leadership and interpersonal skills to advance a project successfully.

In addition to the FAC-P/PM policy, the VAAA in its curriculum design looked internally at several active policies relative to project/acquisition management — Project Management Accountability System (PMAS), ProPath, and the VAAA Program Management School's Enterprise Program/Project Management Training Model (EP/PMM). The EP/PMM was developed by the VAAA Program Management School to provide an organizational agnostic best practices program management framework. This enables the VAAA Program Management School to deliver training relevant to multiple organizations within VA and other federal agencies.

Furthermore, the VAAA design team evaluated all previous training investments at VA that were associated with developing project management disciplines — which techniques and topics implemented were successful, and which techniques and topics didn't add value. Beyond this, extensive market research was conducted on a variety of industry related acquisition/project management strategies, disciplines, and processes.

The VAAA FAC-P/PM training design that delivers on the PMIP vision was designed and developed in 2010–2011. This government owned curriculum has four primary project management learning phases: Concept Definition, Concept Planning, Development & Implementation and Operations Maintenance & Close-Out.

The learning objectives within each of the four phases complement the targeted level of learning: Entry, for a project team member; Mid, for leading complex projects; or, Senior, for managing programs. Learning objectives, associated activities and classroom discussion are all Learner-centered, and applicable to all civilian agency professionals.

This specific phased approach simplifies the transfer of knowledge and lessons learned, as the acquired processes and learned activities are definitively aligned within explicit phased swim lanes. You are learning how you perform on-the-job. The Learning Objectives are:

Concept Definition:

1.   Manage programs and projects in accordance with federal and agency standards

2.   Organize program/project activities around a federal program/project management life cycle

3.   Lead a team to achieve program/project goals

4.   Complete the accomplishments of the Concept Definition phase as they apply to mid-level programs/projects

5.   Interpret U.S. legal, government, and jurisprudence practices as they relate to Concept Definition and programs/projects

6.   Compile the business requirements to support the development of a Strategic Statement of Need for a program/project

7.   Given a prioritized list, modify capability gaps and human performance deficiencies

8.    Develop a prioritized list of recommended solutions for programs/projects

9.   Select a concept that best supports the business need

10. Develop an initial cost estimate to support a Strategic Statement of Need

11. Compose the initial business case to prepare the Strategic Statement of Need for control gate review

12. Justify the final Strategic Statement of Need to obtain approval for your program/project

13. Prepare for the Concept Planning phase

14. Prepare an action plan

Concept Planning

1. Lead a team to achieve program/project goals

2. Integrate project roles and responsibilities for project management plan and Concept Planning success

3. Apply U.S. legal government and jurisprudence practices as they relate to Concept Planning legal, government, and your program/project

4. Develop a strategy to manage stakeholders as they impact program/project success

5. Propose a program/project charter to obtain commitment from senior leadership

6. Develop a requirements package that is traceable to the business need and identifies contracting specification

7. Conduct a Preliminary Design Review (PDR) for a program/project

8. Develop a project management plan for a program/project

9. Develop an acquisition plan for a program/project

10. Investigate logistical concerns and their impact on a program/project

11. Develop a product-oriented work breakdown structure (WBS) that defines all the work required to accomplish the scope of the program/project

12. Establish a team to accomplish program/project objectives

13. Prepare an estimate using different types of estimating techniques

14. Develop an integrated master schedule that identifies the tasks and resources required to accomplish the scope of the program

15. Develop a sustainable cost estimate to support a program life cycle within the context of the capital budget process

16. Prepare a plan for managing quality in programs/projects

17. Create a risk management plan and the prioritized risk with response strategies for a program/project

18. Establish a performance baseline for earned value management (EVM) as it pertains to the program/project

19. Construct a change management plan for a program that incorporates the change management plans for the projects

20. Write acquisition package

21. Prepare for the Development phase

22. Prepare an action plan

Development & Implementation

1. Program and project management
2. Program/project activities
3. Lead a team
4. Control and Monitoring

   a. Tracking cost and schedule

   b. Validating performance and quality achievement

5. Incremental development strategies
6. Operations and Maintenance preparation
7. Prepare an Action Plan

Operations Maintenance & Close-Out

1. Lead a team to achieve program/project goals
2. Integrate roles and responsibilities for Operations & Maintenance and Closeout success
3. Transfer management responsibilities for programs/projects to the operations manager
4. Describe the process for closing out a program/project
5. Connect the results of a final EVM report for a program/project to the benefit of another program/project
6. Lead a lessons learned workshop
7. Complete the process for program/project closeout
8. Prepare an individual action plan

The goal of PMIP is to capture and perpetuate lessons learned to improve business processes over time and create sustainable change agency-wide. Therefore, all learning activities within the VAAA FAC-P/PM program are focused on producing working artifacts and improved perceptions on the value of the acquisition profession; not simply ‘check the box’ training for test requirements.

The Action Plan

As part of the training program, each acquisition professional, with oversight and support from his or her supervisor, is being held accountable for creating and implementing a tailored action plan that addresses a known deficiency in the workplace — an existing workflow, a communication strategy, project processes, requirements gathering techniques, etc.

Each participant works with his or her supervisor to identify a performance gap; then, as part of their training, identifies in real time, a lessons learned that can immediately improve the deficiency. A business case for change is created, documented and implemented over time.

Over 12,000 Action Plans have been developed, completed or are currently being implemented as part of the PMIP program.

Sample Action Plans can be focused, but not exclusively, on the following:

• Program Goal Statement
• Work Breakdown Structure (WBS)
• Milestone Schedule
• WBS Dictionary
• Risk Register
• Traceability
• Program Baseline Review
• Critical Path Analysis
• Tracking Tasks
• Acquisition Artifacts & Lifecycle Overview
• Project Governance & Control Metrics
• Program Dashboards
• SOW (Statement of Work) /PWS (Performance Work Statement)
• QASP (Quality Assurance Surveillance Plan)
• Proposal Instructions (Section L)
• Evaluation Factors (Section M)
• Acquisition Compliance Matrix
• IGCE (Independent Government Cost Estimate)
• Acquisition Plan
• Program Control Dashboards
• Corrective Action Plans and Methodologies
• Stakeholder Management
• Communication Management

Senior leaders at the VA have begun to benefit from the reduced variables and complexities normally seen during a review of a portfolio of project investments. This more consistent application of project management artifacts improves decision making, productivity and customer satisfaction levels as it improves clarity of reporting to stakeholders.

Program leaders who have benefitted from the individual Action Plan improvements are now engaged with the VAAA to implement rapid program improvements.

Closing the Gaps Rapidly

VAAA PM School has also developed a PM Fellows (PMF) Program. The PMF program is a holistic program designed to go beyond the traditional Federal Acquisition Certification in Program/Project Management (FAC-P/PM) by building and reinforcing critical project management, leadership, and business skills within specialty tracks. The PM Fellows Program is targeted for GS levels 9 to 13 who have not yet taken the FAC-P/PM certification. Benefits of the PM Fellows Program include:

• Enhancement of project management, leadership, and general business core competencies, knowledge, skills, and abilities to positively impact the success of VA
• Maximization of “actively managed” on-the-job training, providing a network of individuals to support the participant, while addressing real-time work related challenges within the curriculum.
• Significant contribution to the development of program managers, accelerating knowledge and insight within his or her specialty position or field.
• Further development of the participant before, during, and after the program concludes through a robust mentoring component
• Comprehensive two-year training program yields graduates with an equivalent of 5 to 6 years project management experience
• Risk-reduced completion of projects from home organization while in training with guidance and assistance of project management subject matter experts

While half-way through its pilot, this comprehensive program is designed based upon the successful contracting internship program. We have demonstrated that we can get an employee to the competency and confidence level he or she normally achieves through 5 years of experience in just 2 years. In the PMF program, after their first year, the PM Fellows are managing complex projects to a level that demonstrates about 2 years of training and experience. The target of achieving 5 years of experience in just 2 years is on-track based on program performance expectations.

Assurance of Capability through Certification

The Federal Acquisition Certifications are based on completing required training and having a minimum years of experience. VAAA is not about “check-the-box” training and we take significant steps to evaluate the effectiveness of our training to ensure the value expected is achieved. However, a long-term measure of competency for which a certification should validate is best achieved through demonstration of skills to specific standards. VAAA did not find it practical to evaluate the execution of skills on-the-job, so VAAA PM School developed a practical exam to validate attainment of competencies was required. This 5-day practical exam, the employee is evaluated by a detailed rubric for individual and team based performance provide the VA assurance that those certified by VA are capable of successfully managing their programs and projects.

Development of Project Management Teams

This assurance provided by the FAC-P/PM certification is with one caveat: successful when placed in an organization with mature program and project management culture/capability. VA has a wide array of project management maturity levels with the vast majority at the infancy of implementation of project management practices.

To address the challenge of supporting project management teams while the VA develops its individual, team, and organizational project management competency, VAAA developed a Project Acceleration Workshop series. The concept is based on the Bible parable: “If you give a man a fish he eats for a day, if you teach him to fish he eats for a lifetime.” However, it can take a long time to teach an organization to fish and they will starve to death along the way without direct support.

Project Acceleration Workshops isolate and address management impediments within critical programs/projects that are negatively impacting the cost, schedule, performance, or customer satisfaction.

The Project Acceleration Workshops are organized around specific programs and projects that are both highly visible and critical in nature. These Project Acceleration Workshops are attended by the supporting program/project team, to analyze gaps within the management practices, and to formalize documentation and reporting mechanisms. Areas of focus are, but not limited to, Program Goal Statement, Work Breakdown Structure (WBS), Program Requirements, Integrated Master Schedule (IMS), Acquisition Package, Risk Plan, Stakeholder Management Plan and other critical management disciplines. All Project Acceleration Workshop guidance is consistent with the best practices provided as part of the FAC-P/PM Certification, as well as those outlined in guidance from OMB and the Government Accountability Office (GAO).

At the conclusion of every Project Acceleration Workshop, each project team departs the workshop with several actual project-specific artifacts that are formally capturing actual requirements, project-specific risks, project-specific schedule details, contractor requirements, proposal requirements and many other critical data points necessary to keep the program on budget, on schedule and to their stakeholders’ satisfaction.

Quantitative data are still being gathered, however, stakeholders of these initiatives are beginning to see the common results:

• Consistent adoption of mature project artifacts for the major programmatic effort
• Strengthen and improve project governance
• Improve line-of-sight between project teams and senior executives
• Increase the precision of ongoing measurement of program health
• Boost the quality and timing of interventions to keep projects on track

Consistent with the PMIP mission, the Project Acceleration Workshop methodology is also helping develop one common language — project delivery — within this major Department.

Workshop participants all agree on how this ‘just-in-time’ approach to training has elevated their appreciation and understanding for the discipline of project management after they've been able to directly and immediately apply the acquired principles to an effort that their directly accountable for steering home successfully.

These facilitated workshops have been able to support numerous program offices and replacement Hospitals Activation teams such as the Washington, DC VA Medical Center Strategic Management Office, New Orleans Medical Center Project Legacy Activation, and Palo Alto Medical Center and Polytrauma Rehabilitation Center Activation. Recently, VAAA has partnered with the VHA National Activation Office (NAO) to incorporate the lessons learned from Project Acceleration Workshop with current activations and to develop a custom series of Activation Project Acceleration Workshops to help ensure that every Major Hospital, Outpatient Clinic (OPC), and Community-Based Outpatient Clinic (CBOC) construction activation team follows a consistent activation process as outlined in the NAO Activation Process Guide.

The Results

The cumulative results indicate, on a 4-point scale, that Learners reaction and experience in the PMIP program continue to remain high. ‘Instructor’ scores from nearly 13,000 enrollments is 3.73 (93% satisfaction) and their ‘Overall’ satisfaction with the training remains at 3.53 (88% satisfaction).

Given the success of the program, the VAAA Program Management School has been working closely with their colleagues on the PM Functional Advisory Board at Federal Acquisition Institute to further unify the common best practices of program and acquisition management that will improve performance across all civilian agencies. As a result, several agencies are now collaborating with the VA by sending their acquisition professionals to train alongside VA acquisition professionals, which encourages interagency collaboration, information sharing, and networking.

Currently VAAA is delivering our FAC-P/PM certification courses to the Federal Bureau of Investigations, Department of Transportation, Department of Interior, Department of Health and Human Services, U.S. Department of Agriculture, Department of Education, and the Small Business Administration.

The Increase in Confidence and Performance

As VAAA measures the impact at the individual level, surveys and interviews to both training participants and their supervisors help validate, qualitatively, the impact the PMIP program is having on our acquisition workforce.

• Nearly 90% of the interviewed participants confirmed that their project management knowledge/ skills increased as result of the training.
• The perceived increase on a scale of 1–4 was on average 3.3

Most comments received from supervisors, as indicated below, are ironically not focused on known skills improvements, rather business improvements and improved project outcomes. A sampling of some of the routine comments VAAA receives as a result of the VAAA FAC-P/PM training experience:

• “[Our success on this project] is a direct result of the [training] conducted by your team. I have shared this success story with leaders at the highest levels in VA. Your team's efforts were acknowledged during yesterday's VBA Transformation Planning Update with Dep. Sec. Gould, the Under Secretary of Benefits and the Director, ePMO; all were elated to hear of the value added by VAAA.”
• “The employees that have attended the training, they have become more confident that they have the tools and skills necessary to make impacts on their functional areas.”
• “My employees now have a better understanding about the complexity of project management and that each integral part of the project management cycle is critical to make the project be successful.”
• “The training has definitely helped to facilitate more collaborative relationships with the Department of Defense and the benefits are being reaped by our Veterans in VISN catchment area.”
• “My trained staff has started to see the 2nd and 3rd order effects of decisions and the necessity to include other stakeholders in their planning and communication.”
• “My PMs are more focused, more engaged, and understand the importance of including all key staff in project management.”
• “Employees who have completed the training take more efforts to manage their programs and are more open to stakeholder involvement and better communications with contracting officer and other agencies involved with the projects.”
• “There has been an increased awareness toward project management, therefore better project outcomes.”
• “[Their] management approach is more methodical, organized, and focused.”
• “The team now applies a lot of best practices and concepts learned in class. For example, [the staff member] has enhanced our project plan, acquisition plan, risk management plan, and other critical management artifacts.”
• “[There has been a] positive impact on customer service by enhancing project management skill.”
• “[As a result of training] projects are being completed on time and tracked for performance issues. Lessons learned from each project are carried forward to help improve project implementation for other projects.”
• “The staff member who has completed the training was able to apply her knowledge and skills to the overall CPAC implementation project specific to cost, schedule and performance.”
• More result stories are available at http://www.acquisitionacademy.va.gov/testimonials.asp

The Program/Project Performance Results

Impact Survey Results:

• 79% of students state they were sufficiently prepared to implement their organizational improvement action plans as a result of the training
• 71% of supervisors contacted indicated positive and noticeable changes in their staff members’ project management behavior
• 74% of the interviewed professionals (including those who identified initial performance challenges) are now seeing positive impacts on the programs’ or projects’ cost, schedule or performance targets with 24% seeing significant impact and 47% seeing moderate impacts.

Industry Average Training/Certification Impacts:

^* PMI’s study showed that the average project was not achieving original goals 36% of the time which was putting $0.12 of every $1 at risk.

^** VA's approximate annual non-commodity acquisitions

Specific VA Program Savings Attributable to VAAA’s PMIP:

While direct project cost savings are at times difficult to attribute to training, VAAA is compiling reported cost savings from our stakeholders that they have documented as a result of the FAC-P/PM training initiative. Return on Investment, for this PMIP initiative, has been documented in several formats: cost savings, cost avoidance and improved efficiencies and productivity.

One notable example comes from a recent participant working on a complex capital asset management project that has been impacted by several major challenges over the years. The project involved the transfer of real property and involved federal and state agencies, as well as veteran stakeholder groups.

As an outcome of the FAC-P/PM training action plan, the participant developed an integrated management plan and a cost/budget plan using earned value management. “We've made a significant turnaround with these improvements,” he said. “As a result, the IMP was developed and implemented and delayed tasks have been put back on track with active engagement by all key stakeholders.”

“We are pleased to report that this $200 million Base Realignment and Closure property acquisition and $200 million major construction project has been put back on track, and we have received external stakeholder buy-in. The take-away from the FAC-P/PM training certainly contributed to our success.”

The Veterans Health Administration recently established a Health Care Program Executive Office charged with establishing programs to standardize acquisition of healthcare requirements. VHA implemented the VAAA’s Enterprise Program/Project Management Training Model within their PEO. The VHA PEO achieved US$390 million in program savings as documented in VA Office of Inspector General report, “Audit of Savings Reported under the Office of Management and Budget's Acquisition Savings Initiative” dated 9/30/2012 [11-03217-293].

To date, VAAA is supporting over 8 major healthcare initiatives across the country, as well as 4 to 5 other major VA program initiatives. The pace in which these project teams are developing their management artifacts is now far exceeding their past performance. One program team — IOM — had a direct cost savings of over US$2 million as a result of the competencies acquired, and workshop deliverables, from the Project Acceleration Workshop; prior to the workshop, these skills and artifacts were contracted out to a firm that provided this expertise.

VAAA Recognition Earned

The below awards were based on the training design and the results of the VAAA’s PMIP

• 2011 and 2012 CLO Learning Elite Award for learning strategy, learning impact, leadership commitment, learning execution, and business performance results
• 2011 and 2012 Learning! 100 Award for outstanding learning culture, innovation, and collaboration that drives performance
• 2012 CLO Learning in Practice Awards: Blended Training and Vendor Partnership
• 2012 – Ranked among Global Elite in 2012 Training magazine's Top 125