Introduction
DVC's Mission and Goals
DynPort Vaccine Company LLC (DVC), a joint venture company between DynCorp and Porton International Inc., was awarded the Joint Vaccine Acquisition Program (JVAP) contract in November 1997 to develop and license biological defense vaccines to protect United States (U.S.) armed forces. These included vaccines for Smallpox, Tularemia, and Botulinum. In addition, DVC has contracted with the Department of Defense (DoD) and undertaken efforts to transition other vaccine candidates to full-scale development. These include Venezuelan Equine Encephalitis, Plague, and Next Generation Anthrax.
The working model for the contract is for DVC to serve as the Prime Systems Contractor, whose responsibility is to define vaccine development requirements and identify qualified companies with the capabilities needed to develop and deliver the vaccines. DVC has no vaccine production facilities of its own, so it subcontracts with qualified companies that possess the requisite experience, facilities, and capabilities including:
• Defining processes for producing the vaccine product
• Scaling-up these processes to manufacture required quantities
• Developing the tests required to establish the product purity and quality
• Determining the effectiveness of the vaccine
• Conducting clinical trials in humans to demonstrate safety
• Submitting applications for licensure of vaccines to the Food and Drug Administration (FDA).
DVC oversees all of this effort through its Program Management Office of highly qualified staff of professionals with the requisite skills, experience, and responsibility for overseeing seven distinct vaccine development programs.
DVC must interact and engage in both government and private industry environments. Because these environments are uniquely different, the company has developed a fully integrated set of tools and processes that respond to these environments and provide rigor in the management of each vaccine program. In addition, DVC integrates a well-defined and robust risk management framework that enables each program to identify and focus on the areas that require attention and close management.
DVC's challenges include:
• Managing concurrent development of multiple vaccines
• Emphasis on “due diligence” for DVC's direct control over subcontractor's processes
• Evolving regulatory requirements dictate continual monitoring and application of recent changes in regulations across all vaccine development programs
• The DoD mandates significant planning and monitoring of progress. This requires the application of a DoD acquisition framework to all DVC programs as well as monthly reports on program status
• Utilization of government funds and tight budgetary controls demand proactive management of each program.
Responding to the Challenge Using an Integrated Master Plan Approach
Overview
DVC responded to these challenges by developing a comprehensive program management planning document, referred to as an Integrated Master Plan (IMP). The IMP outlines each vaccine strategy along multiple dimensions and consists of five unique components, all interlinked and updated as needed. Each section, while unique in nature, serves a specific purpose. The sections that comprise the IMP include:
• Technical Approach Summary (TAS)
• Contract Work Breakdown Structure (CWBS)
• Schedule
• Cost and budget
• Risk management.
The intended purpose of the IMP is to provide technical, schedule, cost, and risk information, identified by a unique CWBS identifier, to both internal and external stakeholders. It illustrates the long-range strategy required to develop the vaccine by clearly delineating the scope of work within the project and controlling scope creep. The IMP serves as a strategic cost-modeling tool to ensure resources are properly aligned. It also holds a record of all identified risks in the program and is periodically updated to reflect progress and changes in program strategy including realized risks and funding adjustments. In short, it characterizes the contract between DVC and the U.S. Army, through the Joint Vaccine Acquisition Program (JVAP).
Technical Approach Summary (TAS)
The TAS defines the technical and regulatory strategies for advanced product development that will ultimately lead to licensure by the FDA. It is a high-level description of the major steps required for licensure, including process development, product manufacture, non-clinical testing, process validation, clinical trials, and major regulatory submissions and milestones (INDs, BLAs) required during the product development cycle. These steps can be matched specifically to the major elements of a program schedule and to the CWBS. The document is, therefore, not a detailed statement of work, but rather a statement of program, technical, and regulatory objectives against which subcontractors will provide statements of work for program execution. In this respect, the TAS provides the technical and regulatory basis of the cost estimate for the product licensure program. The document also captures product-specific performance targets as defined in the DoD Operational Requirements Document (ORD). These form part of an additional section within the document that details program-specific assumptions that underpin the technical and regulatory sections. The TAS is maintained as a controlled document as part of the IMP through version control, and is updated as a result of changes to these assumptions, to FDA regulatory requirements, or changes as a result of technical findings during product development. Therefore, the TAS also serves the essential purpose of maintaining product history.
Contract Work Breakdown Structure (CWBS)
The CWBS is a product-oriented family tree composed of the processes, services, and other work tasks necessary to accomplish the project objectives as identified in the TAS. It organizes, displays, and defines the vaccine activities to be developed. The CWBS acts as a framework for all activities in the program. Once items are cataloged, the CWBS aids in the management of activities by linking technical approach, schedule, cost, and risk.
In addition, it provides a quick and easy reference point to drill down to lower levels, both within vaccine programs and to other products in the pipeline. Therefore, the CWBS forms the basis of task identification by assigning a unique number to a task that links all communications and deliverables. This reduces the likelihood of anything being overlooked because the CWBS numbers occur in all sections of the IMP. It also acts to identify all tasks that are crucial to the program and ensures that these activities are incorporated in the schedule, cost, and risk sections.
Schedule
The schedule consists of a dynamically linked set of tasks that not only shows the activities needed for development of the product, but also clearly identifies the linear relationship these activities have on other tasks downstream. To achieve this, the schedule is cross-referenced to the CWBS and TAS and is fully integrated with the cost section of the IMP.
Development of a schedule begins by examining and incorporating program development templates that already exist as part of lessons learned and best practices from other product pipelines. Once the backbone is in place, a more tailored approach is assembled using input from vendors and subcontractors. Common throughout any DVC schedule are milestones that have been agreed on between DVC and JVAP. These milestones serve as key performance indicators monitored by DoD officials on a regular basis.
DVC recognizes that an important aspect of the schedule design is its integration with the cost and risk sections of the IMP. Every month, as the schedule is compared with performance progress, the time phasing of cost and risk are also updated to coincide with the schedule updates. The company has developed platforms that provide schedule updates to functional departments within DVC. The schedule also allows DVC to more accurately predict schedule constraints and downstream impacts.
In addition, the establishment of dynamically linked tasks on the schedule also ensures that resources are acquired on a “just-intime” basis. It is very costly to have under- or over-allocated resources not being maximized for performance. By linking different vendor schedules to the master schedule, DVC has solved this problem. When performance upstream causes the critical path to shift downstream, DVC is able to respond quickly by implementing risk mitigation plans. Since the schedule system is integrated with the cost system, impacts to the critical path and budget are assessed in real time—a most powerful tool.
Cost
The purpose of the cost section is to identify time-phased financial resources. By ensuring that the cost section is linked to the schedule, it is easier to mitigate situations where time and money may be an issue.
The main components of the cost section include the most likely budget estimates of timing and delivery of key products or results based on the CWBS, schedule, and basis of estimate. A beneficial aspect of the cost section is that each activity has links to the CWBS and schedule sections with a one-to-one match. Every activity within the cost section is time phased against the schedule. By having the exact dates integrated from the schedule to the cost section, DVC ensures the best timing and delivery of key resources.
The basis of the cost estimate is derived from subcontractor proposals, estimates, and industry practices. Therefore, the basis of estimate serves as a forecasting tool for DVC. Because the cost estimates are time-phased against the schedule, it is important to schedule first and estimate second. By following these guidelines, DVC is able to apply time-phased valuation of money and overhead rates in our cost projections to the customer.
Over time, DVC performs strategic budgeting exercises using the cost data. This allows us to respond to “what-if” drills from the customer. The data enables the team to generate monthly reports that give financial predictions based on trend analysis. These reports also enable the team to identify over-allocated resources that are misaligned in the wrong time periods, making the cost section of the IMP one of the most effective tools. This has proven to be very beneficial in making strategic financial decisions.
Risk
The purpose of DVC's risk section of the IMP is to eliminate or reduce management by crisis, through identification and mitigation of a risk event, and minimize the impact to the program's performance, schedule, and cost. This is achieved by ensuring that risk management is built into the overall approach and is designed to identify risks early so that appropriate action can be taken before the development pipeline is disrupted.
DVC recognizes that risk applies to every program and service within the company, and as such, all employees and associates play an important role in risk management. A Risk Management Team is responsible for ensuring that risks are identified, assessed, mitigated, and communicated. To assist the team in their efforts, DVC has developed a comprehensive risk manual built on industry best practices. The manual is integrated with JVAP's risk management framework, providing the customer with a seamless platform that reaches across all product pipelines and facilitates communication between DoD agencies and offices. This manual was designed using the following matrix and decision points:
• Risk Preparation
• Risk Identification
• Probability of Risk Occurrence
• Risk Impact
• Severity Index
• Mitigation Strategy
• Mitigation Selection
• Contingency Planning
• Risk Communications
• Execution of Risk Plan
• Reevaluation of Risk Plan and Analysis
• Risk Repository.
The identification of risk events begins with extrapolating similar risks from other pipeline products using a comprehensive lessonslearned approach. Once similar risks have been identified and incorporated, the Risk Management Team proceeds line by line through the CWBS to identify inherent risks associated with each activity. The Risk Management Team assesses each risk event by applying rigid standards using the framework identified above. Following this exercise, a risk probability is calculated and a cost is determined.
With each identified risk event, a mitigation strategy is developed to ensure that proactive oversight is maintained at all levels of the program's development. Finally, once the exercise is completed, the risk budgets are incorporated into the program baseline depending on the risk severity index obtained. The risk budget, when added to the total program budget, gives a more realistic assessment of the program costs. This approach provides the vaccine development team and senior management with a means of assessing the impact of changes in program strategies.
Using the IMP Day-to-Day
Good management practice dictates that work will be defined, planned, budgeted, and monitored. The IMP incorporates these good management practices by integrating technical effort, schedule, and costs into a project plan. This plan is the basis for measuring performance against planned and actual work accomplished. Using this tool, product managers can better manage the work, communicate performance, and forecast estimates to complete.
Program schedules are based on the scope of the project plan. These schedules are updated on a monthly basis. Status information is provided to the product manager from internal and external sources (subcontractors) for this monthly update. This information is evaluated and validated prior to incorporating the reported performance into the schedule. The time-phased budget from the cost section is compared to the subcontractor's invoice to measure the value of the actual work accomplished. Variances from the plan provide the product manager with information useful in the management of the effort. Based on this information, corrective action plans can be developed, implemented, and monitored for more effective program management.
Adjustments to the IMP capture changes in each vaccine program. Shifts in project requirements and priorities, introduction of new technology, changes in FDA regulations, adjustments in funding authorization and replacement of subcontractors are examples of the types of changes that a typical vaccine program deals with on a regular basis. Because change is a normal “condition” in vaccine development programs, adjustments to the IMP are always done in a controlled manner.
DVC uses a formal change control process to focus discussions about the impact of changes to the program and to adjust priorities. This approach helps to ensure that the IMP always contains the up-to-date “vision” of what the program is attempting to achieve. In addition, the document trail of controlled changes provides rich product history of why decisions were taken. This history is vital for long duration vaccine development programs.
The IMP is the program's strategic plan against which all other activities and decisions can be assessed, and provides the team with a statement of the program's vision of how things are expected to proceed. Therefore, the IMP also acts as a mechanism to help unify teams working on the program and keeps them appropriately focused.
Product development is inherently associated with risks. With a defined plan, it is easier to communicate with team members when a risk has been realized and a different path is required to get to the next step. Program plan awareness by senior management and the customer creates the necessary foundation for understanding the issues when realized risks are escalated. In this instance, the IMP provides an invaluable tool in responding to “what-if” scenarios. Because these “what-if” drills occur frequently, the data contained in the IMP can be used to quickly provide the necessary information needed to assess the impacts on the program.
The integrated development, review and approval of the IMP results in agreement on the program's scope, budget, and schedule. This provides DVC management and the customer with the tools needed to report program status to DoD agencies and offices based on first-hand knowledge of the program requirements.
Benefits and Challenges in Using the IMP
The IMP serves as the basic contract with the customer. It provides the tools needed for the timely review of program performance and indicator of future trends. These data points are integrated with DVC's Defense Contracting Management Agency's certified Earned Value Management System (EVMS) and are fundamental in the development of monthly cost performance reports. As stated above, the schedule and cost sections are integrated, providing for the proper time phasing of resources on a “just-in-time” delivery basis. Because these sections are tied to one another, the management teams at DVC are given performance indicators well in advance to give ample time to identify and implement corrective action plans and risk mitigation.
The hallmark benefit of the IMP is that all aspects of the program are contained in one location and tool. This tool is accessible to all members of the team electronically. It is updated monthly, and because it is placed under formal change control, all members are ensured that they are viewing the latest version. Therefore, the IMP serves as an allinclusive tool containing scope, schedule, cost, and risk.
The challenges to using this tool include the monthly requirement of updating and providing the current status of all of the information. This requires due diligence from all members of the team to ensure that scope and performance impacts are incorporated and communicated to the customer. The initial resources required to implement the tool are expansive, but once the tools are set up, the maintenance becomes routine. Depending on the size of the program, the data points requiring data entry could be time consuming. Therefore, each team member has been assigned primary areas of responsibility for ensuring that updates are completed and verified on time.
However, in the end, the benefits far outweigh the challenges. Because DVC is committed to ensuring the success of the Program Management Office, the company is fully dedicated to providing the resources required to update and maintain the systems. In the past, DVC responded to “what-if” drills by assembling the entire team around a war room setting, and drawing out the strategy changes and their impacts to cost and technical scope. Now, by integrating all sections together, the drills are run with only a few key team members. Further, because the schedules are modular in nature, the “what-if” sessions begin with changes to the schedule modules. The cost, risk, and technical changes naturally flow out from these schedule changes. This has reduced the amount of time required to respond to external challenges. When new innovations and strategies are discovered, the program's scope can be modified in as little as a few hours.
Conclusions
In this paper, DVC has shown how program changes and external influences dictate that our systems be integrated and responsive in providing accurate up-to-date reports. DVC must demonstrate to its external stakeholders that we are well positioned to respond to changing environments. DVC has responded in several ways. First, we provide rigor in our management approach by developing and utilizing fully integrated tools and processes that respond to changes. These integrated tools and processes are collectively known as the Integrated Master Plan (IMP) and consists of a technical approach summary, contract work breakdown structure, schedule, cost, and risk sections. The technical approach summary defines the technical and regulatory strategies for advanced product development. The contract work breakdown structure is a product-oriented family tree composed of the processes, services, and all other tasks required to accomplish the objectives identified in the technical approach summary. The schedule clearly identifies the linear relationship between all activities and is integrated with the cost and risk sections to ensure that resources are acquired “just-in-time.” Further, the cost system is a vital tool in DVC's management arsenal whereby all activities, costs, and budgets are time-phased across the program's life span. As the schedule is updated every month, so too is the product's cost profile. Finally, the risk management plan integrates all sections together to determine what areas of product development require more rigorous management and oversight to mitigate impacts to the program's life-cycle schedule and cost.
DVC has also demonstrated that the day-to-day management of the IMP is vital in forecasting resource requirements, responding to customer “what-if” drills, justifying product requirements, assessing internal and external impacts to the schedule and cost, and performing strategic planning. We have shown how the integration of these tools allows us to maintain the project's strategy. By placing these documents under formal change control, DVC is able to quickly trace back the product history from planning through advanced development and FDA licensure; thereby providing all team members with a history file and complex lessons learned that are applied to other DVC pipeline products.
However, with the integration of numerous tools that require constant updating, DVC has had to be sure that the processes are flexible enough to respond to real-time changes in strategy and external influences while still maintaining the integrity of the program. A system is only as good as its ability to accept input and generate accurate output in a dynamic environment. To that end, DVC's infrastructure and business approach is centered on dynamic processes that enable product teams to be innovative in responding to external challenges, thereby securing DVC's future in the biopharmaceutical arena.
Disclaimer
The views, opinions, assertions, and findings contained herein are those of the authors and should not be construed as official U.S. DoD or U.S. Department of Army positions, policies, or decisions, unless so designated by other documentation.
Acknowledgments
The Joint Vaccine Acquisition Program funded this program through the Department of Defense Contract # DAMD17-98-C-8024.
The working group wishes to thank the following for their contributions to this paper: Randall Gaston, Terry Irgens, Michael Langford, Gretchen McLaughlin, Robert Shumate, Robin Sloan, Scott Stewart, Becky Whitmore, and Team JVAP.
