Why agile may not be the silver bullet you're looking for
Robert Tarne, CSM, PMI-ACP, PMP
Agile has become a common practice across organizations, and examples of successful agile projects are not hard to find. However, a basic agile approach may not be effective for a more complex project. This paper will start with a review of basic agile techniques. It will then discuss some of the challenges that may come with a more complex project that this technique may not be able to solve. The final section will look at some of the techniques for scaling agile and how they can help on more complex projects, which may make agile the silver bullet you're looking for after all. This paper is based on the experiences (and challenges) the author faced when first encountering more complex projects, and the techniques he learned as a result.
The agile process begins by generating the product backlog. This is a list of features that may be included in the project. One key when developing the backlog is to keep the items at a high level. This may be done with user stories or simple descriptions of features.
A user story is a one-sentence description of a feature from an end user's standpoint. For example, if the project is to create a website for selling T-shirts, one user story may be, “As a customer, I want to be able to search for T-shirts by size.” Though the user story may seem like a requirement, it is really just the representation of the requirements, which is a key distinction when it comes to change management.
The product backlog may not be complete at the start of “build” phase of the project, but that is expected. Once the team believes they have enough in the backlog, the first iteration can be planned. The first step in planning is to prioritize the product backlog: The most important features or user stories should be at the top of the list. This activity is often called grooming the backlog.
The work starts with planning the iteration. Using the prioritized backlog, the team selects the work they plan on completing in the iteration. Then, they go through more detailed planning around the tasks to complete the backlog items, and then the work begins. An iteration will typically last between two and four weeks, at which time the team demonstrates the completed work. This process is repeated for as long as the project is ongoing.
With this approach, very little time is spent prior to the start of the building of the product. As we will see later, this is one potential source of problems on an agile project. The team gathers a set of user stories and jumps in to start delivering those stories, with little thought about things such as design.
Challenges on Agile Projects
When following a simple agile approach as described above, there are a number of challenges that may be encountered on more complex projects, including issues with the product design, technical issues with the solution, and issues with scaling the project team as the complexity of the project increases.
The Manifesto for Agile Software Development states, “The best architectures, requirements, and designs emerge from self-organizing teams.”
This idea of emergent design implies that up-front design is discouraged. However, on a more complex project, the team must take into account the overall design of the product and not just jump into development with the hope that the design will emerge successfully.
Similar to the design, the architecture cannot be ignored on more complex projects. Although it might be okay to just start building when developing a simple website, a more complex business application needs some up-front architecture. Some questions that must be addressed early on include how to address the application in context of other systems, how to support scalability, and how to ensure reliability.
Many sources, including the Scaled Agile Framework (SAFe) documentation, state that the ideal team size should be seven, plus or minus two people. More complex projects may need more people than this. Simply adding people to the team isn't the answer, however. A structured approach must be used when moving from a simple team to a larger organization creating a more complex product.
Addressing the Challenges
Now that we've identified some of the challenges when applying agile to more complex products, let's look at some of the techniques that can address these challenges. Specifically, we'll look at how Disciplined Agile and the Scaled Agile Framework can address the challenges we've identified.
Disciplined Agile Delivery
The Disciplined Agile Delivery framework has been developed by Scott Ambler and Mark Lines based on their book Disciplined Agile Delivery: A Practitioner's Guide to Agile Software Delivery in the Enterprise (IBM Press, 2012). Disciplined Agile extends the techniques found in Scrum with additional practices such as agile modeling.
There are several types of models that can be applied to an agile project.
Business Process Diagrams (see Exhibit 1) can be used to show the activities and participants in the business process. Using this modeling approach will help ensure that key steps of a solution are not overlooked.
Exhibit 1. Business process diagram.
Context Diagrams (see Exhibit 2) can be used to model the product in terms of the other systems with which it has to interact. This will serve as a starting point for the overall architecture of the product.
Exhibit 2. Context diagram.
Object models can be used to identify the key business data elements that will be used in the product. This will help ensure that there is understanding about where data are coming from and how they may need to be stored.
These are three examples of how models can address the design challenge issue with more complex agile projects by following some of the guidelines within Disciplined Agile.
One role identified in Disciplined Agile is that of the architect owner. The architecture owner is the person who owns the architecture decisions for the team and who facilitates the creation and evolution of the overall solution design. Recognition of the need for this role in Disciplined Agile demonstrates the need for more focused effort on the technical aspects of the project and therefore can address the issue of architectural challenges.
Scaled Agile Framework
As shown above, Disciplined Agile Delivery has enhanced basic agile practices in order to address design and architectural challenges. And though Disciplined Agile also recognizes the need for additional roles in more complex problems, it is not as robust as some other advanced agile approaches when it comes to scaling up to larger projects and programs. This is where the Scaled Agile Framework comes in.
The framework is based on work by Dean Leffingwell, including his book Scaling Software Agility: Best Practices for Large Enterprises and his blog, scalingsoftwareagilityblog.com. The framework continues to evolve, and is currently on release 3.0. It is based on a number of principles such as applied systems thinking and decentralized decision-making.
The framework is developed across three levels: team, program, and portfolio. Within each of those levels are practices to follow to guide activities and coordinate across the levels.
At the core of the team level are Scrum and Extreme Programming techniques. Team objectives are established to guide the team's work. User stories are the basis for managing the work in two-week iterations. A single Scrum Master can serve multiple teams.
At the program level is a concept called the agile release train. This is the mechanism to coordinate the work of up to five to ten teams on a common release schedule. There is a release management role to support the release train, which operates on an 8– to 12–week cadence. Similar to Disciplined Agile, in this framework, the need for an architect role is recognized at this level.
At the portfolio level, Kanban is used to select high-level work packages called epics. Budgeting is performed at this level and occurs every 6 to 12 months.
This framework delivers an effective way to move from a singe agile team of seven people (plus or minus two) to projects and programs that may have 100 or more people
In its basic form, agile can deliver for projects that are not large or complex. The challenge is being able to evolve from a simple project to a complex project. Disciplined Agile addresses some of the challenges that are faced when trying to deliver complex agile projects, including design and architectural challenges. The Scaled Agile Framework addresses the issue of how to deliver large agile projects requiring larger teams. Although a basic agile approach may not be the silver bullet for a complex project, there are agile techniques evolving to help with any project or program, regardless of size or complexity.
Ambler, S., & Lines, M. (2012). Disciplined agile delivery: A practitioner's guide to agile software delivery in the enterprise. Indianapolis, IN: IBM Press.
Beck, K., Beedle, M.,, van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., Kern, J., Marick, B., Martin, R. C., Mellor, S., Schwaber, K., utherland, J., & Thomas, D. (2001). Manifesto for agile software development. Retrieved from http://www.agilemanifesto.org/
Leffingwell, D. (2007). Scaling software agility: Best practices for large enterprises. Essex, England: Addison-Wesley Professional.
Leffingwell, D. (2011–2015). Scaled agile framework (blog). Retrieved from scalingsoftwareagilityblog.com
Leffingwell, D. et al. (n.d.). Agile teams abstract. Scaled agile framework (blog). Available at http://www.scaledagileframework.com/agile-teams/
Roles on DAD teams. (n.d.). Disciplined agile delivery. Available at https://disciplinedagiledelivery.wordpress.com/roles-on-dad-teams/
© 2015, Robert Tarne
Originally published as a part of the 2015 PMI Global Congress Proceedings – Orlando, Florida, USA