Abstract
In 2003 we looked at the basics of project management practices used in Agile methodologies. We discussed project management challenges and opportunities of Agile software development projects and how Agile changes the role of the project manager. Now six years later, using Agile within project management has become very trendy, but is not necessary fully understood. As more and more organizations claim to become Agile, little has been said or published about what Agile means for project management tools.
This paper describes how the main principles of Agile are applied to the project planning processes for scope, time, and cost management. It looks at the triple constraint from an Agile perspective. We will also provide recommendations on what the critical success factors are for applying and gaining acceptance of these “Agilized” processes and tools.
Introduction
What is Agile? According to Merriam-Webster, agile is “marked by ready ability to move with quick easy grace” and “having a quick, resourceful and adaptable character.” (Merriam-Webster, 2009) Agility means the capability of rapidly and cost efficiently adapting to changes. It is public knowledge that IT and software development projects have a history of running over budget, delivering something other than what the customer expects, or being delivered late or not at all. The search for root causes and corrective actions has been going on since acknowledging the issue in the mid-1980s. The Standish Group (1995) found that the top two causes are “incomplete requirements” and “lack of user involvement.” Agile development methodologies came to existence in the ‘90s with the sole objective of tackling the root causes of IT project failure and increasing overall project success.
Then in 2001, the Agile Manifesto (Agilemanifesto, 2001) was created, describing the values and principles of the Agile software development processes. The three main focal points within Agile are (Highsmith, 2002):
- Organizations are “chaordic,” having characteristics of both chaos and order.
- Trust is the basis for all internal and external relationships, with a focus on collaboration and not contract negotiations.
- Focus lies on people and interactions first and on processes second.
The fundamentals of the Agile methodologies characteristics are (Hazzan, 2008; Schwaber, 2002):
- Iterative development with small product releases—delivering products frequently allows customers to provide regular feedback and gives the team the ability to react to changes in the market place.
- Test-driven development—developers write the test cases for the new software before they implement the code changes. Preparing tests before coding improves quality and facilitates rapid feedback.
- Collocation—team members (including customer representatives) are collocated to facilitate communication and minimize wasteful waiting times.
- Self-organizing teams—team members assume multiple roles, improvise, and spontaneously collaborate, while simultaneously learning from and teaching their peers without a command-and-control manager. A team normally consists of a project manager (also called coach or scrum master), product owner, customer representatives, developers, testers, and architect.
- Product (requirement) backlog—an evolving, prioritized queue of business and technical requirements. At every iteration, the backlog is reviewed by the full team and functionality is selected to be built in the next iteration. The product owner is responsible for managing and controlling the product backlog.
Since the creation of the Manifesto, globalization and technology have increased the speed at which the environment we live and work in changes. In today's fast-paced world, customer's objectives are constantly being revised. The drive for Agile methodologies originally came from the technical ranks within the projects. In the last several years, Agile development methodologies have proven themselves to be a valid alternative to the traditional methodologies, especially for smaller projects. Ambler's Software Development Project Success Survey of 2008 (Ambler, 2008) states that 70% of Agile projects are completed successfully, versus 66% of traditional projects. Agile is about to move into the next level of maturity, which would include expanding the Agile processes to address the planning side of larger projects and dispersed teams.
Agile Principles and the Project Management Constraints
Project planning is focused on answering the questions, what needs to be built, when it needs to be completed, how much will it cost, and who needs to be involved. As project managers, we also want to know any dependencies between activities so we can minimize idle time and optimize the schedule.
The conventional wisdom is that estimates improve as the project progresses. Barry Boehm was the first to graph uncertainty levels at different stages in the life cycle of a high-tech project in 1981. The result later was nicknamed “the cone of uncertainty” by McConnell (2006). The idea behind the cone is that at the start of the project there is significant uncertainty about “what” needs to be built. This makes it very hard to answer the “when will it be completed?” and “how much will it cost?” questions. While the project progresses, the project team gains a better understanding of what it needs to build and how they are going to build it. As a consequence it becomes easier to estimate the completion date and project cost, therefore lowering the uncertainty of the estimates. Of course, at the end of the project there is no uncertainty anymore, just reality.
Agile and traditional project planning deal differently with this uncertainty. Ultimately the goal is to create a project plan that moves the project from its initial state to where the project meets the project objectives. The traditional project planning approach creates a project plan by following several planning steps. These steps include:
- Determine the project objectives
- Collect the project requirements
- Define the project scope on a work level
- Identify dependencies between activities
- Estimate work effort and dependencies
- Prepare the overall schedule and project budget
- Receive approval
- Baseline the plan
Traditional project management assumes that projects are managed within certain constraints. In project management, these constrains are often referred to as “the triple constraint.” The PMBOK® Guide (PMI, 2004) refers to the triple constraint as “a framework for evaluating competing demands.” The competing demands are often illustrated as a triangle where each side or corner refers to one of the constraints that are applied to manage a project.
Exhibit 1 shows illustrations of the triple constraint triangle for a project. Quality is often assumed to be fixed, and therefore many organizations forget to make quality requirements a part of their scope, cost, and time planning analysis. We assume that quality affects each of these constraints and has to be taken into consideration by all constraints.
Exhibit 1: Illustrations of the triple constraint triangle (Gray-Larsen, 2006; Lewis, 2002)
Exhibit 2 shows the PMBOK® Guide definitions for the triple constraints and quality.
Exhibit 2: Project key constraints definitions (PMI, 2004)
Understanding how to manage a project within these constraints can make the difference between project success or failure. The constraints are related to and dependent on each other. A change in one constraint has an impact on the others (Koppensteiner, 2008). Often, changes are related to scope changes or alteration of deadlines. Adding functionality to the scope would require increasing the budget to fund additional resources in order to keep the deadline and satisfy the quality criteria. If both budget and deadline cannot be changed, scope has to stay constant too.
In traditional project management approaches, project execution would start after receiving approval for the project scope, timeline, and budget. Throughout the life of the project, progress will be measured against the baseline plan. The approaches deal with uncertainty by implementing a rigorous control process. Any significant changes to the project scope, timeline, and budget will need to go through a change control process and approval before being incorporated into the plan.
This stringent change control process coupled with long development cycles has created a customer mindset that all requirements need to be part of the project scope (e.g., in form of specifications) from the start of the project, since it is so hard to change anything after the project has started. The customer views development as a black box: they provide their requirements at the start of the project and then have to wait until the end of the project to see the results. When change happens or is needed, it takes a long time to be incorporated in the project. As a result, a customer is likely to receive a product/service that fulfills the specifications and not necessarily his needs at the end of the project.
Agile project planning acknowledges the fact that the scope will change throughout the project. It believes change is inevitable. Only the scope of an iteration is fixed. This means the customer can change the priority of their requirements and add or remove requirements after every iteration. This provides the customer with a great deal of flexibility and also makes them part of the team throughout the project. Of course there are limitations to the amount of change the project can absorb, based, amongst others, on chosen architecture: if you were planning to build a two-story building and halfway through, the customer then wants it to be a 15-story building, you will have to start from scratch.
The project team makes detailed estimates for the functionality implemented in the next iteration only. Agile project management recognizes that scope, time, and costs will change as part of iterative planning activities. From a triple constraint perspective, agile projects can be considered time-“boxed,” as each iteration of development has the same duration; the resources can be assumed to be somewhat flexible, and scope is most flexible.
Agile Planning Processes and Methods
Preplanning
Before the start of an Agile project the organization identifies a project vision, product roadmap, and a business case. The Agile project life cycle starts with a pre-planning step. This includes collecting and prioritizing business and technical requirements (the product backlog), team formation, and high-level time and cost estimation. The product backlog contains all requirements needed to deliver the final product, system, or service. It is collected from various sources and then prioritized. The team and experts meet and provide high-level estimates for each feature, which include the time it takes to perform all requisite architecture, development, and release activities. Based on these estimates, a first rough estimate for the entire project duration (time), as well as for the overall cost for the realization of the backlog (scope), can be determined. Estimating is an iterative process (Schwaber, 2003). In this step the team also chooses the duration of the iterations, if it has not been decided by the organization upfront.
Planning
The next step is the planning of releases and iterations (Exhibit 3). The organization can decide to release each iteration to the customer when it becomes available or release a collection of iterations at once in one defined release. Release planning can be done either right after the creation of the product backlog (Sliger, 2008) or after the first iterations have been completed (Schwaber, 2003).
Exhibit 3: Processes for planning of an Agile project (based on Sliger, 2008)
Release Planning
During release planning the project manager, product owner, project team, and other stakeholders break the functionality in the product backlog into a number of iterations, ensuring that each iteration can be completed within the duration of an iteration. They then assign iterations to releases (release backlog). Every release delivers a working product increment. Based on this release plan, the dates for the release milestones, as well as for the final product release, can be identified (time). The overall project cost can be determined from the labor cost of the Agile team and the number of identified iterations (Sliger, 2008). Overhead and material costs have to be added to this number. Exhibit 4 shows the inputs, tools and techniques, and outputs for the “Plan Releases” process.
Exhibit 4: Plan Releases: Inputs, Tools and Techniques, and Outputs for an Agile Project (Sliger, 2008; Schwaber, 2002; PMI, 2004; Highsmith, 2004)
Iteration Planning
Agile project planning focuses on the planning of the next iteration. After each iteration the project team meets to identify the content (scope) of the next iteration (iteration backlog). Exhibit 5 shows the activities during the iteration planning meeting. At the beginning of the meeting the team verifies that there is agreement on the product backlog and the priority of the listed features. Reprioritization will happen if needed, for example, when the team is unclear with the current order of features listed in the product backlog (step 1 in Exhibit 5). After getting more detail on the features from the customer (if needed), the development team chooses features from the product backlog they think can be accomplished within the next iteration (step 2). In step 3 the team identifies the tasks (user stories) required to implement the chosen features, identifies initial task sequencing, assigns task owners, and estimates tasks (time). During step 3 the team might discover that the effort to complete the chosen features is more or less than the time allotted for the iteration. As a result, features might be removed or added to the iteration backlog (step 4). When functionality is moved back to the product backlog, it can lead to a reprioritization of the product backlog.
Exhibit 5: Iteration's planning process
The planning process is iterative. The team might go through the full planning circle several times. At the same time, planning meetings are limited to four hours. The guideline is that the team should not spend more than two meetings to identify the scope and tasks of the next iteration (Schwaber, 2003). The idea behind this is that the team learns from its actions and becomes better at estimating over time. After a number of completed iterations the team members can measure their team velocity, which is the average time they spent on implementing one feature. When they understand and know their team velocity, they can come up faster with their estimates during their planning meetings. Since iterations are short (one to six weeks), uncertainty is limited and estimates are in general quite good.
The cost for an iteration can be determined by the labor cost of the Agile team for the duration of an iteration. As the product backlog and the team velocity changes, the cost for the project changes too. Exhibit 6 shows the inputs, tools and techniques, and outputs for the “Plan Next Iteration” process.
Exhibit 6: Plan Next Iteration: Inputs, Tools and Techniques, and Outputs for an Agile Project (Sliger, 2008; Schwaber, 2002; PMI, 2004; Highsmith, 2004)
Manage Product Backlog
After each iteration, the time to complete the full project can be recalculated based on the results of the previous iteration and any changes the customer(s) has made to the product backlog, including priority changes or adding or deleting features on the product backlog. The scope is only locked in for the duration of the ongoing iteration.
As the product backlog can be modified throughout the duration of the project, the number of iterations and releases are subject to change as well. The outcome of iteration and release planning can influence each other (refer to Exhibit 3). In case less (or more) features would be selected during planning of the next iteration, the content and number of future iterations could change. As a result, the collection of iterations assigned to one release could change as well. The activities of changing the scope, priority, and estimates of the product backlog are summarized in the “Manage Product Backlog” process. See Exhibit 7 for the inputs, tools and techniques, and outputs for this process.
Exhibit 7: Manage Product Backlog: Inputs, Tools and Techniques, and Outputs for an Agile Project (PMI, 2004; Sliger, 2008; Schwaber, 2003)
Parting Thoughts
We described how the main principles of Agile are applied to the project planning processes of scope, time, and cost management. Scope planning activities in Agile projects take place throughout the entire duration of a project. Within the capacity of the chosen architecture, the customer has the ability to add and remove features from the overall scope (product backlog) at the start of every iteration. Only at the end of the project is the complete scope known. Traditional planning approaches, on the other hand, try to fix the scope as much as possible early in the project.
Time management in Agile projects start in the pre-planning step, where the then-known project scope (product backlog) is estimated by feature and on a high level, planned out by iteration and release. Since iterations have a fixed duration (timebox), the project timeline is based on the number of iterations necessary to deliver the product or service needed. In traditional planning approaches the project timeline is calculated by identifying every task, sequencing the tasks, and estimating the individual tasks. Agile planning processes only look at task-level estimates to make sure a feature can be completed in an iteration.
A key success factor in Agile project planning is that the project team is known at the start of the project and team members are 100% dedicated to the project. Agile cost planning is based on this. Cost are calculated by taking the team members and their rate, and multiplying this by the number of iterations and then adding cost like overhead, materials, and third-party expenses. This is a top-down estimating approach. Traditional cost planning strives to do bottom-up estimating to increase accuracy. This assumes a fixed scope that can be broken down in work packages, which then can be estimated based on who will execute the work.
Scope, time, and cost planning go together both for traditional planning approaches as well as Agile planning approaches. From the point of view of the triple constraint, traditional approaches focus on keeping the scope axis fixed, while for Agile approaches the focus is on keeping the duration of an iteration fixed.
The outlined Agile planning processes in this paper are just one part of the processes and methodologies of Agile project management. Implementing Agile planning processes can challenge the status quo of project management approaches and culture in an organization. Critical success factors are dedicated teams including customer representatives, fixed development iterations, and acceptance of top-down estimates for the project timeline and cost. Highly bureaucratic organizations will face challenges to make the mind shift from applying rules and regulations in a command-and-control fashion to trusting an empirical approach like Agile. One of the success factors is that management and project team members acknowledge and accept that the Agile team will improve its estimates throughout the duration of the project. An organizational culture that encourages team spirit and learning is key to successful implementation of Agile project management.
