still the magic bullet, or do you need a blended solution?
Executive Director of Product Strategy, ESI International
While Agile project management has enjoyed a reputation as the next “magic bullet,” many companies have unsuccessfully adopted this approach due in large part to their lack of readiness assessment, unfocused skill development, and existing corporate culture structures. This paper will provide a guide to the pitfalls of Agile project management adoption as well as best practices for organizations to develop a blended, customized approach that matches their unique situation.
The Agile culture will be clearly defined along with the differences between the traditional Waterfall and Agile approaches themselves. Key challenges to Agile adoption, such as access to project portfolio management (PPM), existing structures and corporate culture, as well as concerns on the executive, team, and stakeholder levels will be addressed. Organizational, project portfolio, team, and project manager readiness assessments will also be discussed. Finally, the attributes of an Agile project manager will be examined along with examples of successful adoptions of the hybrid approach.
Originating in software development, the Agile approach to project management is not the “magic bullet” many wish it to be. While it has distinct advantages over the more traditional Waterfall approach, Agile is not the end all, be all solution for every environment. This paper will seek to address what Agile is, what it isn't, and what qualities project managers will need to acquire to utilize the best approach that is unique to their situation.
An Agile environment contains seven distinct elements that every organization and its leaders exhibit:
- a collaborative leadership style
- an ability to embrace change
- an iterative approach to product development and testing
- a people-driven focus (versus process-driven)
- a concentration on features instead of systems
- project managers that are facilitators, not controllers
- project simplicity
Exhibit 1. The Agile environment
As seen in Exhibit 1, it becomes crystal clear why Agile does not work for everyone. If you were constructing a building such as the Burj Khalifa, the world's tallest building located in Dubai, a strictly Agile approach would cause more problems than it would solve. For one, such building projects are not simple, nor do they take a people-driven focus. They are complex and process-driven. At the same time, there may be aspects of the project that could embrace an Agile approach, such as the selection of bathroom hardware or the interior design of individual rooms. The actual steel and glass construction itself, however, would not benefit from a more malleable method. As this illustration suggests, the challenges of Agile adoption are many.
The Differences Between Traditional and Agile Approaches
Differing Metrics in Agile and Waterfall
Because of the nature of Agile projects, many metrics from traditional project management (Gantt charts, work breakdown structures [WBS] or issues logs) are not available. Since Agile project managers (PMs) are not developing long-term, multimonth project plans with predictions that reach far into the future, the familiar Gantt chart and WBS are rare in Agile programs. Agile PMs utilize other metrics, such as burn-down charts in a highly iterative environment as illustrated in Exhibit 2, to understand where their projects stand. The PM is responsible for the overall delivery of the project's objectives and performs this work by keeping the team focused, encouraging teamwork and individual productivity, ensuring that the team has the resources required to be successful and managing the customer relationship as the product is built. Agile PMs are not focused on updating Gantt charts or keeping the project dashboard current; their focus instead is on keeping the performance of the team at the highest level of productivity and collaboration and on isolating the team from distractions, barriers and digressions that could harm their concentration. Since, in Agile environments, the team members are expected to manage their own workloads, PMs serve to ensure that their teammates have the information, resources and technical coaching they need to be successful.
Exhibit 2. Burndown chart
The Necessary Parameters for Adopting the Agile Project Management Framework
To adopt Agile project management, companies must take an iterative (or an Agile) approach to introduce the framework within their organizations. They must become familiar with Agile frameworks as mentioned previously, assess their current capabilities to adopt Agile project management, develop and implement short-term and long-term initiatives, and adopt the framework over a period of time.
Both traditional and Agile project delivery embody similar principles and practices that aim to deliver measurable results. Traditional project delivery can be described as a “waterfall” approach, which presumes that the requirements, expectations, duration, activities and outcomes of projects can be predicted accurately and planned in a sequence before any actual development activity takes place.
In contrast with traditional project methods, Agile methods emphasize the incremental delivery of working products or prototypes for client evaluation and optimization. While so-called predictive project management methods assume that the entire set of requirements and activities can be forecast at the beginning of the project, Agile methods combine all the elements of product development, such as requirements, analysis, design, development and testing—in brief, regular iterations. Each iteration delivers a working product or prototype, and the response to that product or prototype serves as crucial input into the succeeding iterations.
Delivering “customer value” is a key aspect of Agile project delivery. Agile project management is conducted through the collaboration of a small, co-located team that usually consists of the customer/end user, a project manager, a business analyst (or the role of business analysis) and specialist(s). Agile theory assumes that changes, improvements, and additional features will be incorporated throughout the product development life cycle, and that change, rather than perceived as a failing of the process, is seen as an opportunity to improve the product and make it more fit for its use and business purpose.
Key Challenges for Implementation
The migration from traditional product development and project management methods to Agile methods requires substantive changes in the manner in which certain functions—such as gathering user requirements, deriving a project schedule, engineering the product, managing the team and measuring progress—are performed. The variations between traditional and Agile methodologies indicate that “organizations must rethink their goals and reconfigure their human, managerial, and technology components in order to successfully adopt Agile methodologies” (Nerur, Mahapatra, & Mangalaraj, 2005, p. 75). Companies wishing to adopt Agile development and project management frameworks must consider key challenges such as access to project portfolio management (PPM), existing structures and corporate culture, as well as concerns on the executive, team and stakeholder levels.
Project Portfolio Management
As mentioned previously, Agile approaches are best suited for innovative, exploratory/experimental projects such as new software systems with requirements emerging as development proceeds or new product development efforts for a quick-moving marketplace like consumer electronics. PPM is a criteria-based decision making model for allocating scarce organizational resources to the most critical programs and projects. Where traditional projects may be funded using a well-worn forecasting process, a company calendar (e.g., fiscal quarter or year), or project milestones, Agile projects may have to be funded iteratively based upon their deliverables and changing requirements. Many organizations have difficulty managing a bifurcated project portfolio.
Organizational Structures and Cultures
Moving to an Agile framework is also an exercise in cultural migration. Depending on the geographic location, the business (i.e., products and services delivered), and the organizational structures and culture, some firms will make the journey from traditional methods to Agile methods in an enthusiastic and seamless fashion, others will display considerable resistance to the Agile ideas, and others are simply a poor fit for these approaches. For example, highly regulated industries that require extensive bureaucracies, intricate processes or detailed documentation will probably lack tolerance for the lean, nimble, artifact-light approach that Agile advocates.
The challenges in migrating from a traditional environment to an Agile environment involve resistance and objections that may occur at three levels:
Executive and senior management commitment and support is critical for adopting Agile. Key management concerns that must be addressed include these:
- Predictability—Traditional managers like to work within predictable environments that allow them to outline detailed requirements, plan a complete project, forecast the budget, and manage resources. Agile keeps their focus on the delivery of value to the customer rather than strict compliance to a rigorous set of procedures.
- Extensive Time Commitment—Managers must be prepared to accept and sponsor the intensive level of collaboration and involvement that Agile methods require. They may have to forgo written status reports in exchange for the daily stand-up meeting.
- Resources Management—Instead of being task managers, they must be ready to trust their project teams to be self-directed and to tolerate a bit more resource risk as they discover which team members are prepared to take the leap to Agile approaches.
- Risk Management—Managers must prepare to accept the reality of project uncertainty, risk and cost and abstain from arbitrary schedules and budgets.
- Metrics and Measurements—Managers have to accept that the traditional ideas of success and failure will be transformed in an Agile environment. Success will not be measured by compliance to plan or strict change control, but by the outputs delivered by the project teams.
The Team Level
Teams that harbor misconceptions that Agile teams don't plan, can't estimate, don't document, and can't scale can be significant impediments to any Agile migration. A central tenet of the Agile movement is the requirement for highly skilled developers. Since Agile teams are expected to be small, self-governing, and self-regulated, there is a high expectation in regard to the personal attributes of team members—they should enjoy the special challenges of working in an Agile environment; be prepared to forgo personal recognition in favor of team accomplishment; and enjoy working in a highly transparent environment in which their work products, creativity, and diligence are visible to their teammates and customers.
Highly skilled and motivated team members will inevitably have their share of conflict as they strive to complete project goals. Agile project management requires team-building with a collaborative focus in which all team members cross-communicate and are well-integrated. An Agile project manager knows which questions to ask and which information to pass along. Without a strong leader heading the pack, Agile project management remains a well-intended, poorly executed idea.
The Stakeholder/Customer Level
The trepidations that customers and stakeholders express include the fear that scope will lurch out of control. They will lose the traditional signposts of progress on which they have come to rely, and estimates of time and cost will not be available to help them allocate budget and staff. They also convey unique concerns, such as the Agile requirement for intense collaboration and constant availability, and its effect on their own workload. Sales and management teams may express concerns about “account management” as customer representatives are integrated into Agile teams.
Further, when project management teams are in implementation mode, it can be challenging to stop the process in mid-iteration if project priorities have changed. A project manager may be so driven to complete the iteration that it becomes difficult to cease and desist, then shift focus on something else. Sometimes clients will claim in mid-feature creation that the product is not what they were looking for after all. For anyone with a “completion complex” mentality, Agile project management can be frustrating at best, even if it is better for the customer and the company at large.
Learn about Agile Project Management
Before launching the “transformation to Agile” project, it is important for key executives, senior managers and project managers/leaders to learn more about Agile frameworks. Referencing the twelve principles as outlined in The Agile Manifesto is a good starting point. Learning can be achieved by taking courses, by reading books and publications, and referring to web resources, such as the Agile Alliance (www.agilealliance.org) and the Project Management Institute (PMI®) www.pmi.org). Just as Agile is an iterative approach, learning should also be performed iteratively to reinforce the adoption of some degree of Agile project management.
Attributes of Readiness
Before an organization can take on the Agile approach, it needs to address three levels of competency: organizational level, project portfolio-level ,and project manager-level readiness. In addition, an organization's pre-existing structures need to be assessed.
Organizational Readiness Assessment
“Are we ready to execute an Agile project?” is a question that can be answered by conducting “Strengths and Weaknesses” and “Organizational Readiness” assessments. The organization needs to identify and evaluate the various organizational forces in place that may help or hinder its transition to Agile project management.
Executives and senior managers need to determine:
- the degree to which the organization values innovation and creativity over organizational stability
- the level to which the organization can make independent, product-related decisions without consulting other organizations
- the organization's willingness to work with uncertainty
- the organization's ability to allocate resources full time to one project rather than assign to multiple projects
- the organization's ability to understand and embrace multiple approaches to documenting and measuring project progress
- the capacity to partner with the organization's customers
Project Portfolio Assessment
As previously mentioned, Agile approaches are best suited for innovative, “never-been-done” projects and not the best fit for repetitive, well-documented, low-variability, low-uncertainty, and production-style projects. Executives, senior managers, and project managers/leaders review the project portfolio and divide it into two discrete portfolios by determining which projects are best suited for Agile project management and which are suited for traditional project management frameworks.
Project Manager (PM) Readiness Assessment
“Are the PMs ready to execute Agile projects?” is a question that can be answered by conducting a “Project Manager Readiness” assessment. Executives, business managers and all project managers need to determine:
- the degree to which the project manager focuses on the customer rather than on following standard project management procedures
- the level to which the project manager values innovation and practical processes over sticking with the plan
- the project manager's comfort level with an uncertain and changing environment
- the project manager's skill and commitment to sharing information as needed with all stakeholders
- the project manager's level of commitment to the team and the willingness to promote team collaboration
- the project manager's ability to motivate the team, delegate, and then get out of the way
Project Team Readiness Assessment
“Are the project teams ready to function within Agile project management?” is a question that can be answered by conducting a “Team Readiness” assessment for each team member. Executives, business managers and the PMs need to determine:
- the team members’ ability to make independent decisions
- the team members’ commitment and capability to collaborate and work as a group
- the team members’ ability to communicate in person
- the degree to which the stakeholder is willing and able to become a team member
- the team members’ ability to problem solve and come up with new ideas
- the team members’ knowledge and experience with the application area and the tools for creating the project result
Existing Product Development and Project Management Methodologies
The organization's culture, structure and methodologies will determine the effort required to transition to Agile project management. Therefore, it is important for executives, business managers and PMs to ask the following questions:
- Which existing processes, tools, and templates for executing projects can be applied to the Agile project management framework?
- How will jettisoning certain processes and structure impact the business?
- How much effort and investment in time and resources will be required to develop new tools, templates and processes?
- Will the metrics and measurement techniques to determine project success (or failure) need to change?
- Will reporting methods be different for Agile versus traditional projects?
- How will stakeholders and customers react to the change?
- How will the existing culture and organizational structure be impacted by Agile project management?
Once organizational, project portfolio-level and project manager-level readiness have been determined and the organization's structures and methodologies have been assessed, the project managers’ attributes need to be examined next.
Attributes of an Agile Project Manager
More than ever, project managers have evolved from managers to leaders that affect more than just the details of a project. Now they are more responsible for ensuring their team members have the parameters they need to complete the actual job. Instead of being task masters, they have transitioned to be more like champions of the work their teams need to complete.
Project managers’ main focus has become how they can remove challenges and barriers to their teams instead of merely following the schedule. Traditionally, a project manager has been considered a type of “babysitter” who has to be on top of the work and hold their team members accountable for any missteps. In an Agile environment, team members are given a great deal more ownership and accountability as they are brought into the fold from the beginning. What an Agile team requires is a manager who removes obstacles and barriers to do the work. One of the most important skill sets an Agile project manager requires is interpersonal skills.
An Agile project manager no longer needs to be monitoring the schedule as much as they have had to in the past. According to the traditional Waterfall approach, a project manager spends 80 percent of the time minding the schedule. In an Agile setting, 20 percent is spent on managing schedule or the work and 80 percent on removing barriers so the others can set their minds on the tasks themselves.
The Pivotal Role of the Project Manager
Once again, the project manager's role takes center stage for a successful melding of both approaches to occur. During each stage of project development, the project manager assumes different roles. According to Bruce Tuckman's model, project managers’ leadership style moves from forming to storming to norming to high-performing (Tuckman, 1965, p. 384).
During the forming stage, the project manager acts like an authoritarian figure, orienting the teams members to assume the task. The next phase results in storming during which time the team members react emotionally to the task, raising resistance and exerting influence. In the traditional project management approach, project managers spend most of their time in this position.
The norming stage refers to in-group cohesiveness in which new roles are defined and acted out. Finally, the project manager can move into the high-performing stage of constructive action because a structure has been put into place to actually support task performance. While it is every project manager's desire to remain in this space, the traditional method makes it difficult for team members to reach the last two stages. Due to its rigidity, the traditional environment causes teams to remain in storming mode most of the time.
When team members have a mixture of Agile and traditional methods, they are able to switch hats more readily. For those unwilling to forgo the step-by-step Waterfall method completely, there is a new variation in the form of a hybrid approach that assimilates the best of each.
Agile Hybrid/Blended Approach
According to Alistair Cockburn, “We must truly get comfortable with the fact that every project needs its own, ever-so-slightly different and personalized way of working” (Cockburn, 2000, p. ?). There is no magic bullet cure-all approach; however, a hybrid approach can solve these challenges by offering up a unique personalized blend of both traditional and Agile methods.
It is well known that the waterfall method originated in manufacturing and construction as a way to design and implement highly structured physical environments to avoid cost-prohibitive errors or changes in project orientation. The phases included Conception, Initiation, Analysis, Design, Construction, Testing, Production/Implementation and Maintenance, each step falling down the chain in order like water off a cliff. This tangible method was later adopted for software development, but not without its problems.
While the Waterfall method serves inflexible, step-by-step projects, such as the erection of a building or a new automobile, the Agile method works better in more flexible environments in which project managers can learn from mistakes, pull back and reorient themselves as needed. Overlapping phases in software development, for instance, allow for more flexibility than the rigid phase-by-phase waterfall approach. If one aspect of the software does not work after testing, an IT manager can backtrack to fix the problem more readily than attempting to rebuild an entire program after discovering it doesn't work.
For some projects, a hybrid approach is most appropriate to get the best of both worlds: predetermined parameters and the freedom to move within them.
How the Blended Approach Works
As mentioned previously, strict, compliance-driven environments are a challenge for an effective implementation of the Agile approach because most ground-level workers are accustomed to the traditional approach. Spending a majority of their time in storming mode, these workers are often disoriented when faced with policy shifts that require new ways of doing things. However, certain aspects of Agile can lead to greater efficiency. Adjusting workflows to adopt new government regulations, for instance, is a great example.
For instance, a recent federal government mandate that requires the conversion from paper to electronic medical records (EMR) by 2014 can be an administrative nightmare for large health care organizations. Hospitals and health centers are then tasked with adjusting their work flow to accommodate such government policies without incurring penalties. They are required to quickly develop new processes before implementing a new software system. That's where a hybrid Agile/Waterfall approach can help.
Another example taken from the retail industry highlights how efficient a blended approach can be. Several years ago, a major grocery store chain introduced a hand-held self-scanning device to alleviate the customer's checkout experience. The hardware itself was created using the more linear, Waterfall method of project management. As with any widget that is replicated a thousand or more times, no changes were required to the hand-held scanner gun itself. However, the software inside the device required a more Agile approach. The first generation device allowed customers to scan and check out. The next iteration added a new feature by tracking the individual customer's purchasing habits and offering personalized coupons. An additional benefit to the self-scanning gadget is the elimination of product waste, manual inventory lists and overhead costs. Automating inventory tracking allows for just-in-time product delivery, which raises efficiency and keeps prices lower. In the words of Jim Highsmith, “If you want to innovate, you have to iterate” (Highsmith, 2009, p. 39)! Without the Agile aspect, the grocery store would have lost out not only on enhancing the customer experience over time, but also its overall operations.
Although the Agile movement was the “brain child” of the software development and IT world, it has evolved over the past several years, finding its way into more traditional environments for greater efficiency and malleability. There is no question that today Agile project management can be, and has been, applied successfully to a broad range of projects while keeping some aspects of the Waterfall process in place. Users and stakeholders have benefited from the Agile project management approach — one in which the end user and the project team are partnered in a collaborative effort focused on the project vision and the end result.
Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler M. et al. (2001). Manifesto for Agile Software Development Retrieved from www.agilemanifesto.org
Cockburn, A. (2000, January 4). Balancing lightness with sufficiency. Retrieved from http://a.cockburn.us/1616
Highsmith, J. A. (2009). Agile project management: Creating innovative products. Boston, MA: Addison-Wesley Professional.
Nerur, S., Mahapatra, R., & Mangalaraj, G. (2005). Challenges of migrating to Agile methodologies. Communications of the ACM, 48(5), 72–78.
Tuckman, B. W. (1965). Developmental sequence in small groups. Psychological Bulletin, 63, 384–399. The article was reprinted in Group Facilitation: A Research and Applications Journal - Number 3, Spring 2001 and is available as a Word document from http://dennislearningcenter.osu.edu/references/GROUP%20DEV%20ARTICLE.doc
© 2011, Nancy Y. Nee
Originally published as a part of 2011 PMI Global Congress Proceedings – Dallas/Fort Worth, Texas, USA