IM project success through client managed delivery
“Information is power.” We hear that every day. To make effective use of information, we need to manage it. And to help us manage it, we rely on information systems, information technology, and knowledge workers to make sure the right information gets to the right people (and place) at the right time. Today, in both business and military environments, a robust information management (IM) capability is recognized as being essential to success.
While IM has become a key aspect of doing business, the delivery of this crucial capability regularly encounters significant challenges, to the extent that major IM projects are often perceived by management teams as being high-risk activities, likely to fail. This perception calls into question the competence of both IM project managers and the organizations responsible for delivery of IM projects. More importantly, this perception may lead to conclusions that: IM projects are not well understood; initial project estimates are not to be trusted; management should expect to pay for sizeable levels of unexpected work on IM projects; and IM investments involve opportunity costs that are not well understood.
This situation leads to two key questions: Does this perception reflect reality? And—if it does—are there any best practices that can improve the success of IM projects? This paper briefly examines the reality of IM project success and failure, based in part on DND experience, and then discusses a best practice that has been developed within DND to address the most problematic of IM projects.
What Is the Success Rate of IM Projects?
A sampling of the literature validates that the successful delivery of IM projects is seriously challenged, and that betting on dogs or horses may be a better use of IM investment funds, if management is looking to choose winners. The literature also highlights the frustration of management teams: because IM is crucial to success, management teams are forced to invest in IM projects, knowing that they do not have all of the facts and may well be making investment decisions without a clear understanding of the project cost. Careful examination of the facts, however, leads to an appreciation that there are specific types of IM projects that are seriously challenged and that the identified concerns do not apply to all projects.
Three years ago DND undertook a full review of industry, Government of Canada, and DND IM project delivery performance to better understand why IM projects have achieved varying degrees of success and to develop strategies for dealing with these projects. This review confirmed that IM projects were problematic:
• The Standish Group Chaos Report noted (over the time from 1994 to 1998) that Application Development projects in industry had an average success rate that improved from 16% to 27%.
• A 1995 review of IM projects by the Treasury Board (the Canadian government agency responsible for program-level oversight of government operations) indicated that IM projects were failing at an unacceptable rate and that there were serious deficiencies in the way that Departments were managing these critical investments.
• The review of DND projects highlighted many of the same issues and problems with similar results.
In response, DND has undertaken significant action to improve IM project selection, management and governance processes. Greater emphasis is placed on business cases, projects are designed with off-ramps and key decision points, in-house development is restricted in favor of Commercial-off-the-Shelf (COTS) software solutions, and added emphasis is placed on the development and certification of project staffs. While these actions have had positive results, some types of IM projects remain challenged.
The DND review also noted that there is a spectrum of IM projects. While still facing challenges, the projects that deliver IM infrastructure (e.g., radios, satellite systems, networks and computer centers), did not experience the same failure rates as IM business transformation projects—those projects that either substantially change the way that users conduct their business, or those that involved substantial process reengineering. These projects are particularly subject to evolving requirements and changes in the business and technological environment into which the project will be delivered. For example, the implementation of corporate financial, human resource, logistics or operational command and control systems experienced serious challenges to the traditional performance measures of “on time, within budget and within scope.”
Using Pareto analysis (the 80/20 rule) DND determined that we needed to develop best practices for IM business transformation projects as a key strategy to improving our overall IM project performance record. This analysis led to the development of a DND best practice—Client (or in the case of government—Crown) Managed Delivery (CMD). While not a guaranteed recipe for success, CMD has resulted in improved project delivery performance on a number of DND‘s most challenging IM projects.
What Makes the IM Business Transformation Project Challenging?
In examining IM Business Transformation (IM BT) projects, DND identified a number of features that make these projects different from Engineering-Procurement-Construction-Commissioning (E-P-C-C) projects. Probably the most significant difference is that, even when a core COTS product is used, each implementation is unique, in essence representing a “never-done-before” IM system that has no track record in a business-operating environment. User requirements and changes to business processes must still be defined to support configuration of the COTS solution. The requirements definition process itself is different, often being more of a negotiation of tradeoffs, rather than a strict documentation of user needs. This situation makes it difficult to estimate costs and perform detailed project planning for the total project during project initiation, and leads to a fair degree of risk for client organizations.
The DND analysis identified the following challenges associated with delivery of IM BT projects. These are the challenges any best practice must be able to address:
• Lack of clear, well-defined requirements at project start: The approach must support managing the risks that this initial situation implies, as well as risks associated with the evolution of requirements over time.
• Lack of focus on client (or Crown) business objectives and business environments: For IM BT projects, the approach must focus on addressing the business objectives of the client/user, not simply on delivering tools.
• High user expectations: The users in our organizations expect IM solutions to create improved productivity and, perhaps based on good marketing by industry leaders, they also expect that delivery and implementation of IM solutions should be easy, painless and fast. The approach must support faster (and/or phased) delivery of IM capability to address these expectations.
• Lack of integration of business processes with technological solutions: The approach must support the conduct of reengineering to align new business processes with the COTS (or other) IM solution.
• Resistance to change: Resistance to change may result in user-imposed delays on the project, or in the need to make unanticipated changes to the solution. (Experience indicates that users resist COTS business processes and instead want to custom-configure the application to follow their “target” processes.) The approach must offer flexibility in implementing the solution to effectively deal with the “people side” of change.
• Changing technologies: Most large DND IM projects extend from three to five years from project approval to implementation. The approach must provide for the flexibility to take advantage of new technologies or to deal with technological obsolescence.
• Contractors need to make money: Firm Fixed Price (FFP) contracts limit the flexibility that a contractor has in dealing with the risks involved in the IM BT environment (e.g., changing requirements) and the likely conflict that results. The approach must facilitate adapting the solution to meet changing requirements, while avoiding unproductive contractual posturing.
• Traditional contracting approaches (e.g., FFP) are inherently confrontational: Confrontation detracts from focusing on project success and instead leads to contract management rather than project management. The approach must enable client and contractor personnel to work together, focusing on project delivery rather than on scope changes requiring contract amendment.
Recent Trends in Project Delivery
Government of Canada Procurement Strategies
Within the Government of Canada a number of approaches have been, and are being, tried to improve the delivery performance of IM BT projects. One recent strategy that has had some success is Benefits Driven Procurement (BDP). BDP aims at creating a contract environment that will avoid the confrontation that arises from the use of an FPP contract when requirements are not clear or subject to change. It focuses on defining benefits that can be achieved by the project, and aims to create a common focus on project success by permitting both the Crown and the contractor to share in these benefits through a predetermined formula. This approach, therefore, supports the contractor in playing a substantial role in requirements definition, project planning, overall project management and project staffing. BDP remains a valid approach in use on one major DND IM project and, with some modifications, is being considered for a number of new projects.
There remains a major challenge that can arise in a BDP relationship: that of agreeing on what represents a benefit, and how it should be shared. For example, a cost avoidance may be considered a benefit; however, if it does not affect the current baseline budget, it may be difficult to quantify the benefit in a way that it can finance the contractor's share of the derived benefit. Another issue of BDP relates to how to deal with externally imposed delays or changes to priorities or requirements, perhaps driven by changing military commitments or government direction. These can impact on the contractor's part of the benefit equation, if the harvesting of the benefits is delayed or negated. These possibilities need to be properly addressed in advance; otherwise, potential changes in monetary return to the contractor can lead to confrontation and a loss of focus on project success, or potentially, litigation. As noted, BDP remains a viable approach in selected circumstances (e.g., where the Crown has less understanding of how the (new) business area will operate). Where it does not apply, however, another approach is needed.
US DoD Initiation of Integrated Project Teams
The US DoD has successfully applied the concept of Integrated Project Teams (IPT) as a means of limiting negative confrontation between project stakeholders, including contractors. Key features of the IPT concept are formation of teams composed of all relevant stakeholders to deal with issues, and formation of common oversight teams to ensure common direction and guidance. This approach facilitates decision-making based on timely input from the entire team. It focuses on open communications and permits early issue identification and resolution. The approach retains segregated teams for the performance of work. While this contributes to clear accountability, it constrains the benefits that may be realized through the integration of work activities and the optimization of work assignment based on expertise.
DND has piloted the concept of Joint Project Teams (JPT), that extends the IPT concept to an actual integration of contractor and Crown project staffs within a single project office. This approach not only includes a joint stakeholder oversight process, and rapid issue identification and resolution processes; it also includes joint teams performing the project work. This approach permits optimal integration of project delivery skills and experience, which in turn contributes to improved communications and requirements development, and decreased skill redundancy overall.
One of the key benefits of the JPT is that it supports development of a single, fully integrated schedule, visible to all participants. The JPT concept has not been extended to include work that is discrete and can be subcontracted, but rather is used selectively for work that requires joint visibility and performance. The success of JPT has served as the foundation for development of the broader CMD concept, which creates the contractual environment needed to make JPT work.
CMD builds on these successes to further improve the performance of IM BT projects.
Introducing Client Managed Delivery
Client (or Crown) Managed Delivery (CMD) is a project delivery approach that has been developed to enhance the success of IM BT projects. (Note: “Client” in this instance refers to the overall organization; i.e., DND, that benefits from the project, as compared to the “user” of a system, who may also be the internal client for the project team.) The key features of CMD are:
• Integration of client and contractor resources into focused joint project teams (JPT)
• Use of a contracting strategy that reduces project risk by creating common objectives for the JPT, thereby reducing contractual confrontations
• Assumption of the project integration role, and hence overall project risk management responsibility, by the client
• Client ownership of project contingency reserve funds.
In most instances, the contractor provides system integration and other specialized methodologies, as well as personnel and expert services. Contractor commitment to meeting the client's business objectives is assured through the contracting strategy.
Joint Project Team (JPT)
In E-P-C-C projects, it is common for both the client and the contractor to establish their own project management offices. The first manages the definition of requirements and the contract (from the client point of view), and conducts acceptance, while the second performs systems integration and implementation, and manages the contract (from the contractor point of view). With a JPT, there is one project team comprising both contractor and client personnel, mixed as necessary to deliver the project.
Within this project team, it is essential that certain functions and roles be performed by the client, including project manager, project control officer, client contract management, client financial management, and client requirements management and acceptance. The client must ensure it has the capability to manage the project and render decisions in a manner that supports timely delivery of the project. Either contractor or client personnel (based on optimal skills for the job) can fill the rest of the project team functions. For the client, the only other consideration is to assign personnel in key business and technical expertise areas so as to ensure good knowledge transfer from the project to the in-service environment.
This JPT approach permits the best skilled individuals to be assigned to the team. As with the IPT concept, a project charter is essential to spelling out the roles and relationships of the various participants in the JPT, how issues will be escalated and resolved, and how any differences in organizational culture and practice (e.g., eligibility for bonuses, rewards, overtime) will be addressed. Key to success is ensuring that the contract payment plan and other conditions (e.g., incentives) reinforce the relationship and do not become a cause for confrontation.
The contracting strategy must ensure that both parties can focus their respective efforts on delivering the project by working together, rather than continually preparing for possible contract confrontations. FFP contracts do not support this situation for IM BT projects.
The CMD contracting strategy encourages the contractor to assign qualified personnel to the project, supports the contractor's receiving adequate compensation from the relationship, and avoids establishing unrealistic expectations. Cost-plus and incentives-based contracts may be a preferred approach to enhance the development of common objectives. These types of contract have traditionally been avoided because they have the potential to reward the contractor for letting the project carry on rather than wrapping it up quickly. To avoid this result in CMD, it is essential that the client manages the schedule and have a solid performance management framework for the project. The risk to the project is substantially less in a cost-plus or incentives-based contract than with use of an FFP contract; for IM BT projects, such a contracting strategy substantially reduces overall project costs. Additional work needs to be done to develop a contracting framework to support optimal operation of the JPT within CMD.
Exhibit 1. How CDM Meets the Challenges of IM BT Projects
Project work that can be well-defined, scoped, and costed is a candidate for traditional FPP contracts, since both the client and the contractor can understand the requirements and establish appropriate acceptance criteria. The procurement of COTS licenses, for example, could be through an FFP contract, even though the subsequent configuration and installation effort may be conducted using the CMD approach. The probability that the work could be affected by the changing IM BT project environment will dictate whether an FFP contract is appropriate.
Key to success of CMD is that the client maintains responsibility for project integration. In traditional contract arrangements, the client uses an FFP contract to mitigate risks by establishing a fixed price for delivery of a working solution. The contractor holds the responsibility (and risk) for project integration. For IM BT projects, it is highly probable that an FFP contract will not in fact protect the client from integration risks. Rather, the FFP contract becomes an obstacle to resolving the problems that arise because of changing requirements and circumstances. In IM BT projects, the client is better positioned than the contractor to make the necessary tradeoff decisions that will be needed when (not if) requirements and/or technology change. By retaining the project integration responsibility, the client can make these decisions—and the necessary changes to project resources, scope and schedule—with a minimum of conflict with other stakeholders.
In traditional project delivery, where integration and implementation are contracted to a contractor as a risk mitigation approach, the contractor normally includes contingency in the project budget. The amount of contingency is based on expected risks, including the probability of “evolving” requirements. For IM BT projects, the probability of change is high, and the project budget will contain a significant provision for risk. By maintaining responsibility for project integration, the client maintains control over this contingency cost factor. Typically, by maintaining control over the contingency reserve budgets, the client benefits by being able to better prioritize the use of the funds. The client may also realize reduced total project costs.
Positioned for Success
The CMD approach integrates the features outlined above to create a win-win situation between the contractor and the client. By maintaining overall ownership of the project, the client is positioned to deal directly with changing events—requirements, operations, technology, etc.—rather than having to initiate contract amendments in the traditional FFP contract. The contractor is assured of covering costs, while being rewarded (through an incentives component of the contract) for commitment to meeting project objectives. As a validation of CMD, Exhibit 1 aligns the features of CMD with the required features of an IM BT approach, and shows how the challenges associated with IM BT projects have been addressed.
CMD provides “built-in” flexibility to enable continuous validation and adaptation of the IM solution, to ensure its continuing relevance in a rapidly changing technological and operational environment, while ensuring that the IM solution continues to meet business objectives. In the extreme instance, where the project evolves to being no longer relevant, the CMD approach supports early and easy termination of the project.
CMD is distinguished from other approaches in that the client assumes responsibility for the project success, rather than trying to shift the risks to a contractor for a specified fee. In this way, the potential environment of conflicting contractor-client objectives is avoided. There is, therefore, motivation for the contractor and the client to work together to deliver the project successfully.
Lessons in CMD
When to Use CMD
CMD is not the right approach for all IM projects. Rather, CMD is being presented as a best practice for IM BT projects where some or all of the following conditions exist:
• There is a business transformation component to the IM project, involving a significant change to business processes, a broad user community, a major change management component, and a longer implementation timeframe (i.e., a three- to five-year period).
• The client has the capability to manage large projects; i.e., IM project management expertise. (Note that for any large IM-dependent organization, this capability should be considered a core business skill.)
• The client does not have all of the specialized expertise (e.g., systems integration, configuration management, system architecture) needed to deliver the project, and will therefore need to contract for appropriately skilled resources.
• The business requirements and processes are not well understood, are subject to change over the period of the project, and/or may be influenced by events external to the organization.
• There is a considerable effort involved in configuring the COTS software application; e.g., SAP R/3, PeopleSoft, MIMS.
Conditions for CMD Success
In choosing to pursue CMD, there are a number of considerations in structuring for success:
• Trust: A JPT can only function if the parties to the project team understand the charter and work together in a relationship based on trust. Both parties to the arrangement need to provide a degree of visibility into each other's operations; if there are reasons why a trust relationship cannot be established (e.g., past history, security), a JPT should not be attempted. For such cases the IPT approach remains an option and, while not achieving the full benefits of a JPT, may permit a modified CMD approach to proceed.
• Procurement/Contracting Process: There may be reluctance within the procurement/contracting community to pursue CMD, as it is contrary to the traditional approach of passing implementation risk to the contractor. This reluctance may be addressed by highlighting the improved project performance that has been achieved using CMD, as compared with the history of failure associated with FFP contracting for IM BT projects. Additionally, the concern might be expressed that a JPT makes assignment of accountabilities more challenging. The overriding advantage is that the client has increased flexibility to deal with issues around contractor performance. It is recognized that additional work needs to be done to develop a contracting framework to support optimal operation of the JPT within CMD.
• Performance Management Process: Not unique to CMD, but critical to its success, is the establishment of a robust performance management process that permits monitoring of the project progress. To enhance project control, it is essential that a strong requirements management process be established and linked to the overall project performance.
The CMD Track Record
The CMD approach has evolved over the past three years in response to both senior management direction to significantly improve the performance of IM projects, and the frustration of both contractor and Crown project staffs that traditional approaches to project delivery were not working.
In 1998 CMD concepts were applied to a Major Crown Project (i.e., project value over $100M) that had been established using the traditional approaches and that was nearing complete collapse. The project was to deliver a military logistics management application that had encountered changing and complex requirements in a changing technological environment. In restructuring, the Crown assumed responsibility for project integration, took control of risk management, combined the two project teams into a single JPT, and carefully managed the remaining project funds. The result was a complete reversal of the project outcome: the Crown was able to make necessary scope tradeoff decisions (without extended and difficult contract discussions), project completion estimates more closely matched the available budget, and delivery began in accordance with baseline schedules and budgets.
CMD has been tailored for use on other smaller projects (i.e., the degree of JPT integration was more limited) with an equal degree of success. A major health information system that will serve Canadian Forces members around the world is currently being structured from the start using CMD. Again, DND is seeing a remarkable degree of success in being able to deal with changing requirements and external issues, without getting into extensive confrontations with the contractor. In this instance, the most difficult challenge has been convincing the contractor that the Crown really wants to do business using this new approach!
Into the Future With CMD
CMD has been successfully applied by DND on a number of projects and has demonstrated significant improvement in the success of its IM BT projects. The CMD approach has its challenges and, in some organizations, it is possible that CMD may not be viable because the client lacks IM project management capability. To make CMD easy to use, contracting and procurement organizations must become engaged in creating enabling contracting frameworks.
DND will continue to evolve the CMD concept as more IM BT projects are structured using this approach. The CMD concept and features as discussed in this paper will serve as a jumping-off point for further improvements in project delivery and project success. In the fast-changing realm of IM BT, CMD offers one way of meeting the increased demands placed on IM project teams. The key benefit of CMD is that it allows, and in fact encourages, the project stakeholders to focus on project success, rather than on confrontation. The approach can and will be improved with added experience. Use it! Improve it!
Proceedings of the Project Management Institute Annual Seminars & Symposium
November 1–10, 2001 • Nashville, Tenn., USA