Abstract
A key component of well managed project portfolios is the skilled management of life-cycle stage gate reviews. Prior to deciding on whether to authorize the next stage of the project, astute IT Project Sponsors dialogue with their teams on important business and technical aspects of a project. This interactive session, lead by two skilled facilitators and using the forum format, will draw on the insights of experienced practitioners. After an overview of the session agenda, participants will first identify questions that they think are important at each respective gate review. Next, participants will zero-in on the questions they feel are the most critical for sponsors to ask at each gate review prior to making Go/No Go decisions. Then, the facilitators will lead a debriefing segment. Finally, arrangements will be made to ensure that participants can receive the session output following the conference.
Introduction
Over the last decade, information technology (IT) organizations have invested tremendous effort and resources in improving their project performance. Many have initiated project/program offices to facilitate the advancement of project management practices and methodologies by their project people.
This focus on performance improvement of project results and processes has been interrupted in many organizations during the last five years. The dot.com boom (1998-2001) drew many talented IT people away from traditional employers, the economic slump (2001-) put thousands of IT people out of work, and the increasing outsourcing of IT functions to both U.S. and foreign firms has put new wrinkles into the management of projects. In addition, the impact of the economic recession has caused reduction in staffing or the closing of project/program offices tasked with facilitating stronger results.
Even with all these powerful distractions, there has been growing pressure on IT to deliver more value. This proceedings paper will examine the track record of IT project performance, how the performance of sponsors is connected to projects, how stage gate reviews offer one opportunity in this turbulent environment for IT organizations to increase the effective engagement of project sponsors and better manage/control projects in their portfolio, and how structured facilitation can contribute to the process.
Project Success/Failure
Though individual IT organizations can track the rate of their own progress via internal metrics and/or benchmarking comparisons, few large studies exist (accessible in the public domain) which tell us whether or not industry progress is being made. The most well known series is the Standish Group's CHAOS surveys, conducted since 1994. The latest CHAOS study found that application development project success rates continue to improve. The 2004 study reported project success rates of 34 percent (Software Magazine 2004), as compared to 28 percent in 2000, and 16 percent in 1994 (Johnson et al.,2001). The 2004 report estimated total U.S. project waste to be $55 billion dollars out of the $255 billion spent.
To place this data in a larger context, a review of 49 studies of organizational change from 1989-2000 (Smith, 2002) provided median success rates by change type. In addition to software and technology change, Smith reviewed studies about, and provided median success rates (p. 27) for strategy deployment (58%), restructuring and downsizing (46%), mixed change efforts (39%), TQM-driven change (37%), mergers and acquisitions (33%), reengineering and process design (30%), business expansion (20%), and cultural change (19%). Though Smith's study does not distinguish between projects and programs, his review provides useful perspective. Technology projects and large scale change share an important feature: the extent of human change involved requires active leadership. Smith's review included five of the Standish Group studies. Apart from those, the success rates of the six other software and technology change studies ranged from 33-40 percent.
The Project Sponsor's Role
Research studies continue to identify project sponsorship as a critical element that influences project success and failure (Connell, 2001; Johnson et al., 2001; Oz et al., 2000). The sponsor's role is to ensure that the stakeholders will realize the project's ROI, or terminate the project, if warranted, to minimize losses. Unfortunately, sponsors are often ineffective in their role.
Just as general business literature acknowledges the value of leadership, project management literature emphasizes the importance of the project sponsor, as well as how sponsors can be more effective. Sponsor selection (Currey, 1995; Rocque, 2003), key focus points of a sponsor (Slater, 1998), questions sponsors need to ask at various project stages (Currey, 1995; Knutson 2001), and how performance tools may assist sponsors (Rocque, 2003) are part of the body of literature available for sponsors to access. Literature that specifically discusses sponsor performance gaps and/or how various organizations have attempted to close the gaps is scarce.
Some thought provoking research by the team of Thomas, Delisle, and Jugdev has shed light on the way executives view project management (Thomas et. al., 2002). Partially funded by the Project Management Institute (PMI®), their extensive research uncovered some interesting things, among them that many executives see the discipline as primarily tactical, and as an add-on to an employee's job. Executives don't necessarily see that processes and projects require different management controls.
Another study (Rocque, 2003) found that only three (3) out of thirty (30) diverse organizations surveyed, or 10 percent, reported doing something specific to prepare project sponsors for that role, though 70 percent indicated that project management or change management methodologies were in place in their organizations. Respondents commented that sponsor performance varied widely in their organizations and impacted project outcomes, but that little was in place to address the variance, or support intervention. A common experience across respondents was that sponsors might be actively engaged at project kick-off, but became more distant (from the project team) as the project evolved. Among the responding organizations, three had pinpointed the need to assist sponsors and had taken action. Of these, two offered a half-day workshop for sponsors and one had established gate review questions that sponsors were required to ask at formal stage gates meetings.
In the 2002 book, Execution: the Discipline of Getting Things Done, Larry Bossidy and Ram Charan provide further perspective on this subject. The authors use examples representing numerous companies to make key points about how CEOs and other C-level executives make the mistake of tending more to strategy than deployment, despite the fact that the two are interdependent. Curiously, the authors rarely use the words “project” or “project management” in the book, perhaps a reflection of what Thomas et al. found. One area of particular emphasis by the authors is capacity, i.e., executives often set objectives and strategy without first delving into whether or not the organization can deliver --- or if it cannot, determining whether they can position it to do so. Bossidy's record of success as a senior executive brings much credence to his insightful observations about why and how leaders are ineffective.
Executives who do not understand what to expect in deployment are at an important disadvantage in leading their organizations. The sources discussed above, and practitioner experience would suggest that many executive sponsors oversee projects in their area of responsibility with the same controls as they manage other types of business. Generally, they delegate oversight of projects to the normal chain of command. Sometimes, though not always, a lower level sustaining sponsor will step to the plate, assuming the sponsorship responsibilities.
This picture is complicated by the trend toward flatter organizational structures in which line executives have increasing numbers of functions to oversee. It is not unusual for a busy executive sponsor to limit his/her direct involvement with all or some projects to short, periodic updates. Many project managers have experienced “absent”, and under informed project sponsors. In these situations, even if formal gate reviews are conducted, the sponsor's preparedness to ask good questions may be impaired, the resulting view into the project may be unreliable, and the decision making that follows may be faulty.
A combination of project success/failure data and practitioner experience indicates that there is a clear opportunity to significantly improve the performance of project sponsors. As organizations advance their project management maturity, one opportunity to establish or improve controls is via the use of well designed stage gate reviews, in which the sponsor participates effectively. Project office or other internal consultants can enable better performance by sponsors by defining much of what the sponsor needs to ask during a stage gate review.
Stage Gates
Stage gates, also called phase gates, end stage reviews, or kill gates, refer to a formal review conducted at prescribed points in the project's life-cycle, as seen in Exhibit 1. The purpose is to determine whether a project has approval to proceed (“Go”), will be terminated (“No Go”), or will be asked to complete specified actions before the gate decision can be made. Organizations that use stage gates typically define which key stakeholders will be present, what the agenda will cover, and what criteria the project must meet prior to gaining a green light.
Product managers at Corning, Inc. formulated their “stage-gate innovation” five-stage process in 1986, making it one of the earliest companies to use formal stage gates in their product development process. In 1991, a senior Corning IT executive borrowed the process from the line and used it for the first time on an IT project, with very successful results (LaPlante, 1994).
Robert G. Cooper, a professor and consultant, is credited with introducing the stage gate process in the late 1980's to the commercial product development field. Over the years, many organizations have adopted his evolving process or derivatives of it. Where and when the stage gate reviews occur are largely a function of the product development life-cycle in use, and the nature of the particular project. If designed effectively, stage gate reviews can reduce risk and help the stakeholders in managing the business opportunity to deliver the maximum ROI.
Of course, organizations have implemented the stage gate concept using various life-cycle designs. Just as in IT, product development gurus and practitioners have debated for years the respective merits of waterfall vs. iterative and/or rapid development life-cycle models, including concurrent engineering. Though some critics of Cooper's stage gate process point to inflexibility of the process, the potential of excessive management control, and negative impact on the speed to market, the approach appears to be effective for many. A benchmarking study conducted by the Product Development Management Association (PDMA), identified the stage gate process as a hallmark practice of many of “the best” performing companies (Griffin, 1997).
In the late 1990's, leading thinkers in the product development field added a new focus: portfolio management. (Though new to the product development discipline, applying portfolio management has been practiced by businesses for fifty years.) As a result, product development life-cycle models now typically include an initial gate (Gate 0 in Exhibit 1) for assessing one opportunity against others (Cooper et al., 1997; Nelson et al., 1997).
Exhibit 1 – Project Life-Cycle Stages and Stage Gates
For the information systems and technology field, evidence of serious interest in portfolio management and IT/ business alignment began to appear around 2001. At this point the Year 2000 push had concluded, the economy began to recede, and IT value and expenses were under intense scrutiny. CIO Magazine has featured numerous articles about portfolio management in its 2001-2004 issues, but had far less coverage in the 1990's. At the Society for Information Management (SIM) conference in 2003, a number of “early adopters” described their 2-3 years of experience with new portfolio management processes (Hoffman, 2003).
Today, portfolio management is fast becoming a prime focal point for IT organizations. Yet, using formal stage gates to manage Go/No Go decisions appears to be far less common, at least as reflected in the business literature. This may be one sign that IT project management maturity still has far to go, and that strategy, rather than deployment continues to draw more executive focus. However, IT organizations using SEI-CMM or other maturity models, or even Six Sigma, to drive improvement would (theoretically) use end stage gate reviews as part of their systematic practices.
Commercial product development intended for the marketplace and an organization's IT projects share a number of key aspects beyond the similarity in life-cycle management challenges. Those similarities include volatile “market” conditions, intense pressure to deliver quickly, and the heavy dependence on creativity in development. More than twice as many companies that are “the best” at commercial product development use a formal stage gate process than not (Griffin, 1997, p 440). IT organizations not currently using stage gate reviews may want to take note.
This PMI forum session will use a facilitative approach to directly explore two related subjects: stage gates and the IT project sponsor's role. Experienced practitioners from diverse organizations will draw on their project experience and lessons learned to formulate questions that may strengthen the performance of the project sponsor during stage gate reviews. A five-stage project life-cycle, as shown in Exhibit 1, with six gates, and including a portfolio gate will be used.
Forum Session Design
Facilitation as a Technique
Across all types of organizations and industries, skilled facilitators contribute meaningfully every day to helping groups reach objectives. Use of facilitation to improve group dynamics and meeting outcomes has been common since 1980 (Viali, 2000). IT organizations may be most familiar with Joint Application Development (JAD) techniques which strive to maximize collaboration at critical points in the project life-cycle, while building the partnership of stakeholders. Within structured meetings, JAD employs techniques that elicit information, enable dialogue, and facilitate decision-making (Kerr, 1989; Andrews, 1991; Hollander & Mirlocca, 1995; Viali, 2000).
Organizations, especially those already familiar with JAD techniques, can easily extend the use of facilitation beyond project teams. Facilitators can assist an organization to periodically “mine” specific lessons learned from groups whose cumulative experience on an aspect of project management is worth documenting and incorporating into the project management methodology. Though many organizations collect lessons learned project by project, the “mining” approach generates a different “slice” of lessons learned data, not so unlike using the flexible reporting or analysis features of a computer system to enable further discovery of hidden relationships between data items.
This can be an additional strategy for project office consultants who seek to improve the methodology, as well as retain the buy-in of their customers. Just as the project manager acts as an integrator for a particular project, project office consultants act as integrators for the overall process guiding the portfolio of projects. This integration responsibility is reflected in the multi-faceted nature of the role, i.e., it depends upon being a broker, translator, implementer, and facilitator (Sullivan, 2000). So, while integrating new dimensions of insight from the client base, project consultants may also use this process improvement technique to create needed focus by the client base.
A variation of this internal information mining design, that is, bringing together experienced individuals from multiple organizations, can further enrich the mix of perspectives. This PMI forum session will employ the latter, tapping the expertise and experience of individuals representing many organizations. The session design will be described in some detail so that organizations seeking to replicate it will have a guide.
Session Design Overview
The main purpose of the session is the generation of specific questions that project sponsors can ask to enable meaningful dialogue and decision-making during a stage end gate review. In a session that is highly interactive, the session facilitators will first overview the session design and agenda with participants. Then, participants will identify important business and technical questions, highlight the most critical ones, and debrief the key findings and the process they have experienced. The session will conclude with arrangements to receive the session output.
Session Preparation
In the same way that front-end loading is important to project outcomes, advance preparation for any facilitated meeting is a critical factor in its success. For this PMI forum session, the arrangements include obtaining a room that is large enough for the facilitation event and that offers 3-4 walls for posting of group output. The facilitators prepare visual aids that will support the creation of meaningful session output. When two facilitators conduct the session, advance coordination of session roles and responsibilities must occur.
The session's agenda consists of the following segments:
- Welcome, overview, and ground rules (10 min.).
- Participants generate/post questions (30 min.).
- Individual participants vote (15 min.).
- Debriefing and general discussion about output/process (15 min.).
- Summary, arrangements for deliverables, and closing (5 min.).
Welcome, Overview, and Ground Rules (10 min.)
The session facilitators officially welcome the participants to the session and provide brief introductory remarks, including the session objectives, the planned agenda, and standard meeting ground rules. In addition to providing visuals that show the context of each gate (Exhibit 1) and examples of typical subject matter (Exhibit 2), the facilitators quickly review the activities in each stage of the life-cycle to be used, and sample questions that will be visible in each stage gate's designated posting area. To minimize frustration for participants, as well as foster ease of transfer later, the facilitators mention that index cards which permit participants to jot down ideas that occur to them at any point are available at each gate's posting area.
Exhibit 2 – Stage Gate Review Agenda Topics
Participants Generate/Post Questions (30 min.)
The facilitators provide instructions for this “roving” brainstorming segment. The objective of this segment is to generate questions that sponsors need to ask at each respective gate. The facilitators point out the designated areas to record the questions (one per 8-1/2 x 11 page). Participants may work in small groups, or independently, or both, as they choose. Participants may spend their time at one or more gates during the 30 minutes. The facilitators suggest that each participant should strive to generate 1-3 unique questions, posting each question as soon as it is identified, to avoid congestion in the last five minutes of this segment. Finally, facilitators remind participants about the visual aids that were discussed in the previous overview, and which are posted for their reference. During the segment, the facilitators will move around the meeting room to ensure that any assistance required is provided.
Individual Participants Vote (15 min.)
The session facilitators explain that the purpose of this agenda segment is to highlight the questions that participants feel are the most critical of those generated. The total participant group is divided into six segments and asked to begin voting at one of the respective posting areas, and then rotate in the gate sequence. Participants visit the output pages for each gate and pen in a small circle on the page for the questions that they select, limiting their votes to a number provided by the facilitators that day, based on session attendance and output quantity.
Debriefing and General Discussion about Output/Process (15 min.)
The session facilitators lead the debriefing segment by first reviewing the questions that received the greatest amount of votes, though acknowledging that what are critical questions for a particular project or organization can vary. The facilitators encourage discussion about both the session content and process, recording the discussion points for the session documentation.
Summary, Arrangements for Deliverables, and Closing (5 min.)
The session facilitators thank the participants for their contributions, discuss arrangements for participant receipt of the session output, and conclude the session.
Summary
This PMI Forum Session has been designed to explore the questions that IT project organizations can build into the design of stage gate reviews. Having a list of excellent questions to ask will enable busy project sponsors to more effectively dialogue with project stakeholders and position them to make better Go/No Go decisions. The authors feel that the session design can be re-used to “mine” many other types of lessons learned as well, employing groupware when geography is a challenge (Ward 1997). The organization can feed the lessons back into their methodology, and ultimately, improve project management practices and results.
