Approving systems projects
eight questions an executive should ask
Rodney Bailey, PMP
Our business journals and corporate boardrooms are replete with stories of runaway projects in the information systems development arena. A U.S. Air Force logistics system fails to produce after a half billion dollars have been spent on it … The Federal Aviation Administration and IBM struggle to complete the Advanced Automation System for air traffic control, which is projected to exceed its original schedule by several years and has already exceeded its original budget by over $1.5 billion … AMR Corp.'s attempt to build a reservation system for hotels and rental car companies results in a $165 million write-off and several lawsuits … A large publishing company attempts to develop a new system for organizing subscriber/customer information, utilizing internal resources for the management and development of the project; after spending more than two years and millions of dollars in the effort, the project is halted and professional project management consultants brought in to reorganize it … A medium-sized public utility embarks on the development of a new system for all its core business functions; the project will cost from $50 million to $100 million more than expected and exceed its scheduled implementation by more than four years. Finally, a study of 62 open systems projects found that “roughly 70 percent will come in materially late, materially over budget, or will fail to meet significant user expectations” (see Tom Ingram's article, “Managing Client/Server and Open Systems Projects: A 10-Year Study of 62 Mission Critical Projects” in the June 1994 Project Management Journal, and his update of that study in the December 1995 PM Network).
With this kind of performance track record, many have suggested that businesses should refrain from doing large-scale IS projects at all.
Yet no facet of corporate existence is more affected by the forces of change acting upon business today than information technology. Driven by the need to seize the initiative in new markets, maintain a competitive advantage, or take advantage of business partnering opportunities, both domestic and foreign companies must continually adopt new information systems technologies or upgrade their existing systems. The information infrastructure for many companies exists on platforms that are obsolete, have poor integrative capability, and worse, may become unsupportable in the future. These systems will have to be upgraded. Competitive forces that require companies to exchange data electronically will also generate large IS projects, as will the need to reduce costs through improved technology.
Therefore, we must assume that the development of new information systems on both a large and small scale will continue, and will represent a significant area of investment for many organizations in the future. Unfortunately, our performance in managing the projects in which these investments are made is not very solid. If we cannot avoid the need for new systems development, we can and should identify the common causative factors that make these systems projects succeed or fail.
Since these IS projects represent investment of corporate resources, they normally require some level of executive approval. Executives who sponsor or approve such projects must be knowledgeable, trained and informed to ensure that the proper investment decision is made relative to cost/benefit and strategic direction. They may also need further training and guidance in learning what questions to ask to satisfy themselves that the project will be wellmanaged and have the best chance to deliver a successful product. What are the major factors associated with large-scale IS project success? How can the executive identify those factors? Presupposing the technical and business merit of the project, this framework of questions can ensure the probability of a successfully delivered information system. The questions focus on those areas essential to successful large-scale IS project management.
Do I Have the Right Person Managing This Project? A common mistake that companies make is to assign project management responsibility to someone who has technical competence and an aptitude for teamwork, problem-solving and working with people. Often this person may have been the initial driving force for the project itself. As desirable as these traits are, they may prove to be an inadequate substitute for the knowledge and experience of applied project management concepts and methodologies. By the time the technical person gets up to speed as a project manager, it may be too late to avoid serious problems. On the other hand, the project manager with a heavy business focus but lacking technical understanding may give priority to cost and schedule concerns but overlook technical issues. Successful projects utilize project managers who have a balance of technical and applied project management knowledge, and ideally who have managed other IS projects of progressively increasing size, as Erwin Martinez noted in his article in the June 1994 Project Management Journal (“Avoiding Large Scale Information Systems Project Failure: The Importance of Fundamentals,” pp. 17–25).
What are the Project's Objectives and Measures of Success? The executive should expect very detailed, specific answers to this question. He or she must be satisfied that the objectives and measures are understood and shared by the development team and the eventual system users. The objectives serve as the project “compass,” keeping the project focused and moving in the right direction. The measures of success, which should contain both quantitative and qualitative indicators, let the project team, customers and sponsor know when the project has achieved what it was initially chartered to accomplish.
As suggested by Martinez, the objectives and measures should be addressed as part of the scoping activity, which should also identify:
- Business functions and processes affected by the new system
- Tangible and intangible benefits sought
- Organizational areas affected
- Major problems to be addressed
- Major issues to be resolved during the project
- Systems to be replaced
- Proposed solutions to technical goals.
What Process will be Used and What Elements will be Addressed in the Project Plan? In study after study, this area stands out as a primary factor in the success of IS projects. According to Ingram's study of open systems projects, a primary mistake made by unsuccessful project teams was to charge ahead with a new technology or process without reconciling the key issues identified through the planning process. This definitive planning step should identify organizational responsibility down to the lowest level of task detail, as well as the types and quantities of resources required to perform each task. The systematic utilization of a work breakdown structure methodology and software will help define the complete scope of the project and support a detailed understanding of the cost estimate. A Critical Path Method scheduling tool should be used to communicate the planned sequence of work activities, and also help identify where scheduling or resource usage conflicts occur. One of the more valuable outcomes of the planning phase is the early identification and resolution of potential conflicts, as well as a thorough understanding of the scope of the project in detail by the full project team and customers.
What are the Known and Unknown Risks to the Project, and What Steps will be Taken to Mitigate Them? An experienced IS project manager recognizes that both business and technical risks exist and that strategies must be developed to address such risks. One major advantage of a thorough risk assessment is that the executive will begin to gain an understanding of the quality of the estimate he or she is being asked to approve for the project. Many, if not most, formal risk assessment methodologies provide range estimates or some type of statistical confidence interval associated with the estimate.
The contracting approach is one method for addressing business risk, since at one level the contract represents an agreement as to how risk is shared between the owner and the contractor. The type of contract used should be consistent with the degree of known versus unknown risk factors. The inexperienced project manager tends to view lump sum fixed price contracts as the panacea for addressing all contract-related business risk. In actuality this may be the worst type of contract to use when a large number of unknowns exist. It may result in a significant number of expensive change orders or a vendor who cannot perform the work contracted for within the contract terms.
Another strategy for dealing with business and technical risk is the system development approach. Whether utilizing the standard Systems Development Life Cycle (SDLC), prototyping, piloting or packaged commercial procurement, different risks are increased or decreased. For example, whereas many IS professionals believe that uncertainty can be wrung out of a project by doing very careful requirements analysis, others believe that on many projects this is fundamentally erroneous and advocate what is called Managed Evolutionary Development, an approach that recognizes the existence of unknowns and seeks to resolve them in a carefully managed way as the project proceeds (See Gary Anthes' “Welcome to the Unknown Zone,” in the Feb. 8, 1993, issue of Computerworld).
It is worth noting here that in Ingram's study of open systems projects, a major factor that was viewed as leading to a successful outcome was an effort to minimize the use of “bleeding edge” technology unless it was absolutely vital to the project, at which point it would be checked in a pilot environment. This resulted in minimal loss of time and money spent investigating such new technology. Therefore, an understanding and recognition of the degree of use of new technology should be identified as a potential risk factor to the executive.
What are the Major Decision Points and Which Require Formal Executive Approval? The purpose of this question is obvious, but its application is often too informal. Executives find themselves incrementally authorizing the continuation of the project through the approval of lower-level documents (i.e., contracts, purchase orders) that are presented over time, without the benefit of a total project review that would provide for the formal assessment of project performance against the initial objectives. The project may have grown or changed in ways that have not been adequately communicated to the executive.
One methodology for project planning, discussed in Mitch Betts' “How to get Runaway Projects On Track” (Computer-world, March 15, 1995), addresses this issue by including the identification of five funding points at different stages in the project's life cycle at which the project manager and executive sponsor can formally discuss the status of the project scope, schedule or budget. If they find that major changes have occurred, the sponsor has to make decisions about cutting scope, increasing resources, or possibly canceling the project. The important thing to note is that these are milestones at which deliberate, fully informed decisions are made to continue or cancel the project.
How will Scope and Change Control be Managed? An effective scope/change management program is also a common denominator to most successful projects. Change is almost a certainty on an IS project; it cannot and should not be eliminated, since it may represent desirable or essential improvements in the product.
However, changes must be managed through a formal, deliberate decisionmaking process and not allowed to occur by management default. A change control log that identifies basic information about the proposed change should be maintained. Since changes are key factors in extending the schedules, inflating the budgets, and potentially lowering the quality of the product, each proposed change should be accompanied by an assessment of its impact on the project's schedule and cost, along with an analysis of the risks and benefits. A formal process should be in place for reviewing and approving or denying the proposed change. Quentin Fleming and Joel Koppelman discuss this in “Taking Step Four with Earned Value: Establishing the Project Baseline,” in the May 1995 PM Network..
What Early Warning System will Indicate Trouble? Often the executive or sponsor only hears good news about a project through normal status reviews, and problems are not identified early enough to properly address them. The executive should ensure that the project manager develops an unfiltered process and reporting routine that shows where the project may be experiencing difficulties, and that the information is made accessible to the executive. This early warning system should establish trigger points/events that by definition require a formal review of the project (see Stephen Sawle's “Telecommunications Project Justification” in the August 1993 PM Network). Typical trigger points/events might be:
- A certain percent cost overrun occurs or is projected
- A defined amount of schedule slippage occurs or pre-identified milestones are missed
- Excessive overtime utilization occurs within a given time period
- Constant plan changes are made with little schedule change
- Quality processes are skipped or overlooked (i.e., software reviews, etc.).
All these can be indicators of potential problems on the project. One effective way of providing this information is to incorporate these as threshold levels within routine performance indicator graphs. For example, a threshold level for overtime usage is established, and a graph is routinely produced that shows current usage relative to the established threshold. If the threshold is exceeded, the executive can then communicate with the project manager to determine the significance of the potential problem that has been identified.
Would I Approve This Project if I Knew That it Would Take Twice as Long as Planned, Cost Twice as Much as the Estimate, or Deliver Only 75 Percent of its Stated Benefit? This question may appear to be facetious. However, if the answers to the other seven questions do not provide proper assurance of effective project management processes, this question should be considered very seriously. In cases where the project is marginally cost/beneficial, where the investment required is significant, or where time to market is critical to the system's value, the executive who is not satisfied with the answers the project manager provides may wish to reconsider the investment or reassess the project approach.
The proper management of IS projects is critical to the success of many organizations and to the executives who sponsor such projects. I have endeavored to show that there are common processes, tools and disciplines that will improve the chances of project management success, and that there are crucial mistakes that can be made that undermine the successful delivery of an information system. Moreover, it highlights the executive sponsor's responsibility for ensuring that the right people, tools, and processes are being applied to the project, and provides a guidance document that will help him or her ask the correct questions and have a level of understanding necessary to evaluate the answers before granting approval to an information system project investment.
Rodney “Butch” Bailey, PMP, is a project manager for Entergy, Inc. He has extensive experience in managing facilities and systems projects, establishing project management programs, and providing training in project management.
PM Network • May 1996