Into the sunset
agile risk management for replacing aging legacy systems
With new technologies being regularly introduced, many organizations feel compelled to seek new information systems to replace their aging legacy systems. The promise of a modern user interface, lower maintenance cost, and shorter training cycles seem to be all too tempting to resist for organizations across the business spectrum. The pursuit of a new replacement to an existing legacy system can be so alluring at times, that business and IT leadership tend to overlook the many risks that are associated with the implementation of a new system. Indeed, these risks are real as some statistics show three in five IT implementations do not do what they were supposed to do for the expected costs and within the expected timeline.
There is also an expectation gap of what range of features the new system can deliver on day one in production compared with the retired legacy system. This gap can often lead to disillusion about the value of the new system and produce many change management challenges that can derail the implementation process and thwart the full adoption of the system by the different user groups.
This paper explores the many drivers for replacing an existing legacy system, both genuine and superficial. It provides a comprehensive overview of the main risk areas that often jeopardize the implementation of new replacement systems including the unbalanced business case, ever-expanding scope, resistance from within the organization, and vendor risks to name a few. Finally, the paper includes many lessons and agile tips gained from the author's years of real-life project experience as well as a wealth of experience obtained through the review of existing literature. Ultimately, this paper aims to equip project managers to spot risks early in the project's life cycle, and engage a variety of agile mitigation strategies to neutralize these risks and improve the project's overall chance of success.
What Is a “Legacy System”?
For the average consumer of technology and personal electronics, an iPhone 2 or a BlackBerry Storm is a legacy system. Other newer and shinier models deem these “older” technologies near obsolete. For business users, however, any system that doesn't have a user-friendly graphical interface (character-based, green screen, commands-driven, etc.), and was developed many years or even decades ago is considered a legacy system.
However, in their book An Executive's Guide to Information Technology: Principles, Business Models, and Terminology, Robert Plant and Stephen Murrell (2007) expand the view of legacy systems to include any system that is unable to adequately support the organization's current and future business processes. Based on this inclusive perspective, a system that was introduced say in 2002 with a perfectly acceptable graphical user interface, and runs on an upgradable platform can still be considered a legacy if it fails to meet the above-mentioned criterion.
Organizational Drivers for Replacing Legacy Systems
It is interesting to observe how different organizations move ahead with replacing their existing legacy systems that have served them for years and have become almost a part of the corporate identity as well as an important intellectual asset. The type of drivers that lead to the initiation of the replacement project has a direct impact on the formulation of the business case, and as we will discuss further in this paper, can introduce certain risks to the replacement project. In his book, Why New Systems Fail: An Insider's Guide to Successful IT Projects, Phil Simon (2011) discusses several of the most common drivers, some of which are covered in the following discussion.
Organizational Growth and Acquisition
When a company starts as a new business entity, its needs are limited; with a couple of hundred employees and a small client base, the system they choose (or develop) would reflect these limited expectations. As the company grows over time, it requires a much more powerful back office system to handle increased transactions. Not only can the front end become insufficient, but it requires a more robust database or back end to meet its business needs. In this case, the company can easily justify the costs of the new system.
Mergers and acquisitions are also causes (as well as consequence) for growth, and can lead an organization to give up a functioning system and adopt another to increase efficiencies and leverage other systems with the other organization they merged with or acquired.
Since a new IT system is considered an organization asset, there is an opportunity for certain IT costs to be capitalized, which positively impact the overall project cost from the organization's point of view. This is usually an auxiliary driver in a business case but cannot be used as a main driver since there are other easier ways for an organization to improve their tax profile than to engage in major (and awfully risky) system replacement project.
Although not very common drivers for outright abandoning an existing legacy system, they can become persistent and pressing enough to bring the subject to the attention of the top brass in an organization. These may include:
- Suspension/discontinuation of support;
- Dissatisfaction with the service, and turnaround time;
- Excessive support costs; and
- Forced, frequent, and costly upgrades.
Severe Limitation to Competitiveness
This is one of the drivers that can resonate with the business community at large and therefore can be used as a catalyst for change. These limitations can include:
- Long training cycle, which can be particularly important with businesses with high employee turnover;
- Unreliability/unacceptable down time, which can be observed by the business at the operational level, and therefore would negatively impact their perception of their legacy system; and
- Functional deficiency, which manifests itself when the business just cannot do certain operations or needs to apply many workarounds to complete simple tasks.
Why Does an Organization Wait So Long?
With all the strong impetuses discussed previously, one would be inclined to think that organization are trampling over themselves trying to replace their legacy systems, right? Well, the reality is there are many organizations still choose to continue to maintain their legacy systems over replacing it with a newer, more modern system. The reasons for this “stick with the legacy” attitude include these:
- If it ain't broke, don't fix it—why rock the boat when we can just get by with the system we have?
- Living in oblivion—business executives honestly believe (rightly or wrongly) that their legacy fulfills all their business needs.
- The cost of action—replacement projects are not cheap and many organizations feel reluctant to commit to the cash outlay.
- Fear of the unknowns—there are many of these when embarking on a replacement initiative.
- Fear of the known—horror stories of replacement projects gone terribly wrong and corporate's risk aversion
- Corporate culture and attitude towards adopting new technologies—this is best described by the technology adoption life cycle graph shown in Exhibit 1.
Exhibit 1: Technology Adoption Life Cycle (TALC) graph (Moore, 1999)
Common Risks for IT projects
Project Management Institute (2009) emphasizes that all projects are uncertain. Uncertainty is inevitable since projects are unique and temporary undertakings based on assumptions and constraints that may or may not be accurate. When it comes to information technology projects, risks are plenty and can jeopardize projects of all types. The results of inadequate management of these project risks can be observed in the staggering failure rate of IT projects. In his book Risk Management for Software Development Projects, John McManus (2004) cites the results of an expansive historical research covering 8380 software applications which produced the following eye-opening results:
- Challenged projects: 53 percent were completed but incurred cost and schedule overruns, resulting in fewer features than originally specified.
- Cancelled projects: 31 percent of projects were cancelled at some time during the development cycle.
- The average cost of overrun for challenged and cancelled projects was 190 percent of the original cost estimate.
- The average schedule overrun for challenged and cancelled projects was 222 percent of the original estimate.
- On average, challenged projects were delivered with only 31 percent of the originally specified features and functions.
The above research also attributed the causes of failure to mainly two groups of reasons: managerial and technical. Managerial issues included causes like inappropriate project structure and channels of communication, inappropriate resource allocation, poor planning methods, and inadequate risk management. Technical issues included causes like inadequate requirements analysis, poor technical design, inappropriate development, and testing tools and methods, and poor documentation.
Special Challenges for Legacy Replacement
Legacy replacement projects naturally face all the common risks for any IT projects and then add a few special ones of their own. It is important to highlight the special nature and characteristics of replacement projects before we start a detailed discussion on the added risks. The following discussion aims to illustrate the uniqueness of replacement projects as opposed to regular IT projects using some analogies and observations gleamed from the author's own project experience.
Unseating the Incumbent
Any casual observer of politics will agree that it is much harder to beat and unseat an incumbent office holder than to compete for an open seat. Take the office of the U.S. presidency for example; there are only four U.S. presidents in the 20th century who accomplished this incredible feat of unseating an incumbent president. This is usually attributed to the power of the incumbent, which allows them to command loyalty, dictate the agenda, and exploit peoples' unease with change. These same forces can be observed at play in favor of the incumbent legacy systems, albeit more subtly.
Being in production, the legacy system is a live system and therefore dictates many of the parameters for the replacement system. It commands the loyalty of an army of IT professionals within the organization who built their careers for years caring and feeding the beloved legacy. They may complain about how cumbersome it can be at times, or even make fun of its clumsy, unattractive appearance, but make no mistake it is their cumbersome and clumsy legacy! Many of the IT and business staff alike can understandably feel nostalgic about their legacy and choose to fight the efforts to retire it, compounding the change management challenges inherent to IT projects.
High Expectations/Low Tolerance
A legacy system usually evolves over many years, sometimes several decades, with numerous enhancements, patches, and releases to support an evolving business. Despite its complexity, staff members who had been present throughout the lifetime of a system would recall the humble beginnings of their system, how it started small and just grew over the years.
One of many challenges for a new system therefore is to match that final evolved state, without the luxury of having a decade's worth of timeline and budget to make it happen. On day one of the implementation, many organizations simply expect the first release of the new replacement system to match the full functionality of the last release of the retiring legacy.
Sometimes that challenge of replacing a highly complex, overly evolved system just proves too much as in the case of the IRS legacy system replacement. The IRS legacy system was developed in the 1960s and the agency has been trying for years to replace it to no avail. After 20 years of trying and over $5 billion, there is no end in sight for replacing this gigantic legacy system. To this day, the system still suffers several deficiencies including lack of information in real-time, internal systems that cannot talk to each other, and the need for manual entry of paper returns. The delay in replacing the system has also been quite costly in other ways. In 2006, legacy computer problems caused the IRS to erroneously hand out an estimated $318 million in fraudulent refunds that could have been prevented, had a new replacement system been implemented earlier.
Conquering the Castle
The more one is involved in replacement initiatives, the more the whole endeavor feels like an attempt to conquer a besieged castle! The invading army (project team) surrounds a well-fortified castle (the legacy system that is nestled safely in production). The invaders tighten the siege, plot their moves, and prepare to storm the castle. The inhabitants of the castle continue to defend their territory and valiantly fend off the invaders' attacks.
Now the conventional wisdom gives the invaders an edge, after all, they are on the outside and time is on their side, right? Well, not quite! You see, in this example, the castle inhabitants can count on steady waves of supplies, while the invaders have limited stash and a limited time window to complete the task. If you are curious how this could be, consider that the legacy system (the castle) is the one in production supporting business operations and therefore can count on full support from the operational budget. The replacement project (the invaders) has, on the other hand, a limited project budget and a fixed window to get the job done. No wonder many invading armies had to retreat in defeat before seeing the inside of that elusive castle!
Agile Approach to Anticipating and Managing Risks
Having been branded as a software development methodology with its own set of vocabulary and terminologies, we tend to forget that “Agile” was always popular business acumen as well as a part of the common lexicon. Webster dictionary defines agile as “having a quick resourceful and adaptable character <an agile mind>”. Oxford dictionary defines it as “able to think and understand quickly.”
Throughout this paper, the term Agile is used to refer to the act of being proactive in both anticipating and mitigating the special risks that are associated with the legacy replacement projects. It denotes an awareness of the types of risks that may be lurking ahead and the ability to act before these risks materialize and their negative impacts metastasize. Regardless of the source of risks, and as we will discuss further below, the project manager's sometimes limited opportunity to influence the initial choices in project's lifecycle, it is still the project manager's primary responsibility to anticipate and act proactively to ensure that these particular risks do not derail the replacement project.
The Business Case: Foundation for Success or Predictor of Failure?
Often times project managers are engaged in a project after the business case has been completed and signed off, which deprives them of the opportunity to influence or vet the fundamentals behind the project's inception. Flawed reasoning or skewed value proposition in the business case can go undetected by the project manager until related risks start to materialize in the project. Not entirely familiar with the business case, project managers often tend to make the mistake of dealing with the symptoms of the risk rather than addressing the root cause. The sections below survey the less tangible risks that may arise from improperly formulated business cases, driven solely by either IT or business drivers, which if not mitigated, can seriously affect the success replacing a legacy system.
The IT-Driven Business Case
Legacy Is Just “Too Hard and Expensive to Support”
Many of the legacy replacement projects are initiated to relieve internal pressures within the IT department of an organization. These may include: difficulty in finding and retaining resources with the required technical skills to support the aging legacy, lack of enthusiasm for the archaic technology with which the system was implemented, the perception (whether justified or not) that the system is too expensive to keep maintaining. All these reasons, and more, sound very real and reasonable to IT professional but they hardly resonate with the general business community which expects their IT department to just “deal with it” to keep them up and running. Their focus is getting their business done, and if the system is functioning, then the IT should do what they are paid to do to prevent system-caused disruptions to the business.
The CIO of our main commentator just upgraded!
System envy is real. Keeping up with the Joneses is not confined to cars and houses; organizations tend to also covet the shiny new technology used by their competitors. CIOs compare and measure their status in the business community by what kind of technology they have running on their servers, and what kind of major initiatives they have going on within their respective departments. No CIO wants to get together with his peers on the golf course to talk about his 25-year-old legacy system and how reliable it is running on the super-efficient AS/400 platform! While this bragging right means a lot to the CIO, it rarely ever matters to the business leaders who are usually driven by business results and bottom-line margin, or to the rank and file in the business, who just want to do their job unimpeded by technical disruptions.
Vendors' Pressure and Lofty Promises
Account managers across the spectrum of software and hardware vendors often tend to paint a bleak picture of the prospects of the current system in hope of securing an upgrade deal for an application or a platform. Reasons can be driven by a negative impetus like limited or suspended support of a certain platform, or a positive incentive like touting the benefits of an upgrade and downplaying the disruption of the transition.
The Business-driven Business Case
One of the least exciting reasons to upgrade or change a legacy system is to fulfill some mandated regulation, which may not have an apparent business benefit that will help sell the upgrade to the business community. This is further complicated if the regulation is vague or the consequence of non-compliance is uncertain. Records management is but one example of regulations-driven initiatives that is usually fraught with risk, as the cost of non-compliance is usually a remote occurrence from the average business user's mind, which is much keener to retain maximum flexibility in managing their documents. In this case, the user community will find every reason to drag their feet through the implementation, and try to shift the project ownership from one department to another (IT, legal, privacy commissioner, etc.).
Top-Down Business Edict
Some replacement initiatives are justified solely based on a perceived return on investment point of view, which is usually a main concern at the top brass level. However, if the business case does not highlight a set of tangible benefits to the rank and file business community, it will be that much harder to secure their buy-in and cooperation. Top down edicts can certainly get an initiative started, and the support of the top brass is vital to the existence of the project, but the ultimate success of a project is determined largely by the backing of the business community at large, and their desire to see the new system implemented.
Merger and Acquisition Mandated Replacement
This is another type of a top-down decision where, as a result of a business decision made at the top, the legacy system of one (or both) company may need to be retired to make room for another company's system (or a totally new system altogether). The risk here is manifested through competing loyalties between the teams from the two companies. As we discussed earlier, the legacy system embodies the business practices of a company; during a merger/acquisition situation, feelings about company loyalty and loss of corporate identity run high within the business community. To replace a company's legacy system in the midst of this may seem like rubbing salt on an open wound.
Agile Risk Management Tips
As discussed earlier, project managers may not always have a great deal of influence on formulating the business case, since it is often completed prior to his/her assignment to the project. Having said that, there is no excuse for not studying the business case and understanding the risk areas that could be introduced to the project because of how the business case was structured and sold to the business.
For example, when finding that a business case of a project is primarily IT-driven, an agile risk manager would realize that this situation have the potential of encountering many forms of resistance, both overt and covert, from the business community. As will be discussed further in this paper, the success of a replacement project depends largely on the business' willingness to make sacrifices and live through the pain of a new system implementation. It is therefore a vital success factor that the project manager be cognizant of the enthusiasm gap that can result from a skewed business case's drivers and actively deploy strategies to clearly delineate and communicate a set of tangible benefits that will touch as many business stakeholders as possible. This includes emphasizing functional gains, efficiencies, and operational wins at various business levels. Organizing road shows and embedding business staff within the project team will help improve the adoption of the new system by the business and create change champions along the way.
Alternatively, when a business case neglects the impact of the project on the existing IT organization, and ignores the anxiety of the staff about their role in the new world, it risks alienating the very people the project needs to produce its deliverables. Giving the existing IT organization a stake in the success of the replacement can loosen their bond with the existing legacy and help them to start embracing, even outright championing the new system.
Being proactive in anticipating and mitigating the imbalance that may exist in a business case can make the difference between a successful implementation or a struggling initiative.
Implementation Scope and the Challenge of a Moving “Goal Post”
Ever Expanding Scope
Defining what will be delivered in terms of what will be available on Day 1 in the new system is arguably one of most challenging aspects of a replacement project. It is not that the scope is not known at the onset of the project, rather it is because the target keeps moving throughout the project's life cycle. Remember that the legacy is a production system, with its own operational budget. This challenge varies depending on how dynamic the business environment; certain types of businesses are harder to freeze and wait for the new system than others. These include: regulations-based government agencies, retail, and businesses plagued with high turnover at the top management level. As discussed earlier, the IRS system is case study for risks and challenges in replacing highly evolved and evolving systems (remember the government continually tinkers with the tax code and tax exemptions every year) but by no means is a unique example. The author of this paper had similar experiences delivering replacement systems to two departments of licencing in the US, which were negatively impacted by the continuous changes to local legislations that occur whenever changes happen at the top executive level. Replacing such a system sometimes feel like attempting to redesign and rebuild a densely populated city like New York, without evacuating it first or being allowed to disrupt the “daily pattern” of its residents' daily lives.
One of the first steps in a project's life cycle is to initiate a requirements gathering phase. This usually involves assigning a few business analysts to meet with the business users to document what business functions to include in the new system and define how these functions should behave. This is particularly true for custom development projects where the project starts with a blank page. Now, that is like asking a carpenter to build a table and then being surprised when it is made out of wood! For business users, who many of them spent the majority of their professional life using a particular legacy system, there is usually one way of looking at things. Replicating the legacy, with all its pitfalls, is a real and frequently occurring risk for a business undertaking an expensive and painful replacement initiative. True, the new system will have better looking interface and a few bells and whistles here and there, but make no mistake, that amounts to no more than putting lipstick on a pig!
Agile Risk Management Tips
At the heart of managing the scope risks is the project manager's ability to articulate to the business the type of sacrifices they will need to make to maintain a stable implementation scope. The choices for the business are straight forward:
- Freeze the legacy for the duration of the project except for emergency bug fixes; or
- Accept a reduced set of functionalities for this first release of the new system than what was available in the retired legacy.
When a project manager allows the scope of the new system to continue to evolve based on the legacy system, the new system may never become a production system. Regardless of how good of a change management practice put in place, there will come a time for the business to bite the bullet and make one of choices described previously.
As for the risk of introducing meaningful, not only superficial, upgrades in the new system, the project manager would need to seriously consider a business process evaluation and possibly re-engineering as a predecessor to the analysis and system design phases. Remembering that good system design cannot fix broken business processes may be a useful tip to keep in mind before jumping head-on into the requirements gathering phase.
Choosing the Right Type of Replacement System for the Organization
In-house Developed Solution
The IT department of an organization assumes all the main responsibilities of developing the replacement system. It may choose to involve some independent contractors to supplement its workforce, but essentially this is a ground-up operation and the in-house IT department is in the driver seat. From risk perspective, most would agree that this is by far, the riskiest of all types when it comes to potential for delays, budget overruns, and lack of access to proper technical staff when needed just to name a few.
Areas of Strengths
- It can potentially produce an easier hill to climb when it comes to change management since the internal IT staff would be inclined to feel as much a part of the new system as the legacy.
- System is custom-built for the organization's business requirements, so no need to compromise on features.
- Full control of upgrades and enhancements to accommodate business needs.
- Organization retains the system knowledge since developers are internal.
- No licensing or maintenance fees to pay to vendors.
Special Areas of Risks
- The organization cannot leverage any existing work neither in best practice business processes nor in development and quality assurance.
- Being a bottom-up development, it involves a great deal of design work; a missed requirement early on is both common and expensive to introduce into the application.
- The organization may struggle to locate the right technical resources when needed from a limited pool of talents
- Quality of deliverables may be compromised due to the overreliance on internal resources that may not be best suited for the tasks.
Independent Software Vendor
This is a variation on the previous type; it is still a custom development from the bottom-up but instead of developing the system in-house the organization opts to outsource the bulk of development activities to an external company specializing in system development and integration.
Areas of Strengths
- The ability to leverage an extensive experience gained by the vendor through pervious implementations;
- Full control of the scope of future upgrades and enhancements;
- Benefits from the vendor's ability to tap the market for technical talents when needed; and
- Typically can be cheaper and quicker than internal development.
Special Areas of Risks
- Like the previous type, the organization cannot leverage any existing work neither in best practice business processes nor in development and quality assurance.
- Again, being a bottom-up development, it involves a great deal of design work; a missed requirement early on is both common and expensive to introduce into the application.
- Culture clash between the vendor and the client organizations can negatively affect the success of the implementation.
- Over reliance on one vendor can be too risky for some organizations.
Commercial Off-the-Shelf (COTS)
Instead of building their own, an organization often opts to buy a mature application (or a suite of applications) to avoid the many risks of the bottom up system development. This approach covers both the implementation of a monolithic enterprise resource planning (ERP) application, or cobbling together a suite of best of breed applications.
Areas of Strengths
- Being a presumably mature application, it is therefore expected to be relatively bug-free.
- Benefits can be realized from the input of other organizations using the application, which leads to business process improvements and more stability to the platform.
- Being a mainstream application, more resources can be hired from the labor force with experience using the system.
Special Areas of Risks
- A generic system will never match end-user business requirements out of the box, which often leads to customization efforts. These customizations, if not controlled, result in a massive project scope increase with huge budget overruns.
- On-going support of the new system can typically run at about 20% of the initial license fee, depending on the vendor.
- Customizations come at the organization's expense; support will not normally cover modifications under the vendor's default agreement.
- If best of breed option is chosen, additional risks of compatibility and building customized interfaces have to be factored in the equation, which substantially increase the implementation scope and add more uncertainty to the schedule.
Software as a Service (SaaS)
This is a pay-as-you-go model with relatively small upfront setup fees. It presents an attractive alternative to organizations weary of all the risks associated with owning a particular solution, and therefore willing to be flexible when it comes to supported features, and business processes within a new solution.
Areas of Strengths
- Potentially quicker activation time, since no major customization are expected (or even permitted) within this model.
- Less reliant on the in-house IT organization as vendor handles upgrades.
- Potentially lower cost of ownership (based on number of transactions), with relatively small up front set up cost.
- Organizations pay only for software used; no money is wasted on shelf ware.
Special Areas of Risks
- Hosted software and data give rise to security concerns in an organization. If privacy and data security are big concerns to the organization, this option may introduce unacceptable risks.
- There are typically hidden costs that may not be investigated up front, including ridiculously high cost charged by the vendor for creating custom reports that are over and above the standard ones.
- Very little room for customization, which may require the business to make unacceptable compromises to their processes.
Agile Risk Management Tips
As we learned from the previous discussion, each option comes with a unique set of strong attributes as well as particular areas of risks. Choosing the right replacement type cannot take place in vacuum, and will have to be considered in the context of several other factors. These may include, the size of the organization, the maturity and depth of its IT department, available budget, overall risk tolerance, number of transactions performed by the legacy (key factor when evaluating SaaS solutions), as well as security concerns and timeline constraints.
As project managers, many times we are assigned to projects where the replacement system types have been already selected (usually as part of the business case) and therefore do not normally have an opportunity to influence the type of solution that end up being selected for the organization. However, being an agile risk manager, one needs to be aware of both the strengths and risks intrinsic to each replacement type and actively engage in devising strategies to lower the impact of these inherent risks on the project.
Let's take, for example, the COTS implementation scenario; one of the main strength areas of this option is leveraging an existing solution that stood the test of time in production. One common temptation, as we have learnt from the above discussion, is that during the customization phase of COTS, the business often tries to recreate this mature application into its own image (or more accurately, the image of its legacy) rather than change the business processes that are supported by the legacy system. For an agile risk manager, mitigating this risk means being involved in these discussions and urging the business at every turn to make the tough decisions of changing their processes whenever feasible to meet the functional scope of the package. It also means working with the vendor to identify the impact of these identified enhancements on the implementation project and highlight which enhancements can be incorporated as part of the base product, hence can be supported by the vendor during the regular upgrade cycles. By being proactive in realizing these special risks and managing their impact on the implementation, an agile risk manager can add tremendous value to a particularly risky replacement project.
This paper equips project managers involved in a legacy replacement initiative with an awareness of the types of risks that often lurk ahead undetected, and the ability to act before these risks materialize and there negative impacts metastasize. To this end, the paper focused on the special nature of projects that deal with replacing IT legacy systems, drawing on the author's own experience as well as a wealth of experience obtained through a review of existing literature. In addition, the paper provided a comprehensive overview of risks that arise within three main areas: formulation of the business case, managing the implementation scope, and choosing the right type of replacement system.
In conclusion, it is imperative to remember when approaching risk management that it is not always feasible or even desirable for a project manager to eliminate all risks and threats from a project. Consequently, we have discussed the importance of being an agile risk manager and that regardless of the source of risks and the project manager's sometimes limited opportunity to influence the initial choices in project's life cycle, it is still the project manager's primary responsibility to anticipate and act proactively to plan for and mitigate these special risks before they materialize and impede the overall success of a legacy replacement initiative.
Broache, A. (2007, April 12). IRS trudges on with aging computers. Retrieved from http://news.cnet.com/2100-1028_3-6175657.html
Evans, N. D. (2003). Business innovation and disruptive technology harnessing the power of breakthrough technology … for competitive advantage. Upper Saddle River, NJ: Financial Times Prentice Hall.
Heldman, K. (2005). Project manager's spotlight on risk management. San Francisco, CA: SYBEX.
Lientz, B. P., & Larssen, L. (2006). Risk management for IT projects: How to deal with over 150 issues and risks. Amsterdam, Holland: Elsevier/Butterworth-Heinemann.
McManus, J. (2004). Risk management in software development projects. Burlington, MA: Elsevier Butterworth-Heinemann.
Moore, G. (1999). Crossing the chasm: Marketing and selling high-tech products to mainstream customers. New York: HarperCollins.
Plant, R. T., & Murrell, S. (2007). An executive's guide to information technology: principles, business models, and terminology. Cambridge, UK: Cambridge University Press.
Project Management Institute. (2009). Practice standard for project risk management. Newtown Square, PA: Project Management Institute.
Simon, P. (2011). Why new systems fail an insider's guide to successful IT projects (Rev. ed.). Boston, MA: Course Technology.
© 2012, Osama Aziz
Originally published as a part of 2012 PMI Global Congress Proceedings – Vancouver, British Columbia