It is easy for a team to transition to agile/lean software development: Pick a good pilot project, get some training, re-arrange the workspace, learn the process, and maybe use a coach. It has been done thousands of times. It is easy, but all too often there is no benefit for the organization. The goal is not making teams agile but making the organization agile. This is harder. It involves building a transition plan around the organization so that everyone—customers, organization, and teams—will succeed.
A successful transition plan focuses on the complete set of work that is required to turn an idea into a product that customers can use. It will minimize the time and effort required. This requires looking at opportunities in an organization, management, team agility, and technical practices that will have the greatest impact on removing impediments and improving the flow of work.
Where to Start? Where Do You Want to Go?
The starting point for most journeys is having an idea of where you are going. As obvious as it sounds, many organizations are not clear about this. They know they want to “become more agile” but are not sure what that means. And much of the literature on agile does not help, focusing only on development teams. It seems natural to start introducing agile at the team level, but that is not the goal, not the destination; it is merely a way station on the journey of making the whole organization more agile.
Organization agility focuses on enabling a an organization to respond quickly to competitors, to internal insights, to changes in industry or technology, and so on. All efforts should lead to becoming more and more agile as an organization.
With organization agility as the goal, we can have meaningful measures. For example, the elapsed time between when an idea is first developed until it is implemented, delivered, and customers can begin to use it. We call this the “cycle time.” The shorter the cycle time, the better. Short cycle times let the organization penetrate the market faster and respond to competitive pressures more effectively. The value of this metric is that it focuses on the entire “value stream” for the product.
A value stream is the whole set of work that is done to create a product, from idea to delivery of value. Ideas arise from interactions with customers who could be external or, in the case of IT, internal to the organization. Organization stakeholders and product managers create a list of product enhancements to be built by the development organization and delivered back to the customers (either external or internal for IT organizations).
Product development is just one part of the value stream. If you improve product development but some other step still takes too long or is blocked, then cycle time may remain the same. Reducing cycle time requires optimizing the whole value stream which is a key focus of lean: lean suggests that removing impediments to work (delays which cause waste) improves speed to market while at the same time raising quality and lowering cost.
Impediments to Organization Agility
There are at least four general categories of impediments to organization agility.
Unrefined enhancement requests. The organization side does not properly select, size, or prioritize their desired product enhancements. Yearly planning forces stakeholders to ask for too many product enhancements, which tend to be too large. Faced with long delivery cycles, they pile on requirements just to be sure they get what they anticipate they will need. Trying to accommodate all these requirements results in working on too many projects at one time. Too large, and too many, organization requirements make it difficult to prioritize. This means that some of the features the team is working on may not be the most valuable to the organization yet, and because they were already started, they have to be finished before the team can take on the next, more valuable feature.
Unrefined enhancement requests also result in delays because developers must go back to the customer to ask for more explanation and must wait while requirements are developed. We would say the feature was not yet “ready” to be worked on. Waiting equals delay.
Improper allocation of resources. To manage this crushing workload, teams are often isolated into “silos.” Unfortunately, this dilutes the efficiency of the organization even more. Developers end up waiting for others to start work or to share information. In an attempt to remain productive, more projects are started. But this just adds to their overwhelm, reducing efficiency and causing even more delay.
Lack of team availability. Team agility holds the promise of shorter development times: take a statement of need and build a system or feature to meet it and then deliver it quickly. Unfortunately, teams may not be available or are difficult to form. Just creating new trams without solving the cause of too many projects may the rest of the organization becoming even more overwhelmed than before.
Poor technical practices. Too often, code becomes complex and brittle and hard to maintain over time. With discipline, it doesn’t have to. The proper use of acceptance test-driven development (ATDD), (unit) test-driven development (TDD), design patterns, and refactoring can keep code robust over time. While these techniques have been around for a long time, they are still not in widespread use. Many organizations suffer from technical debt in their existing code.
Start Where the Problem Is
If you want to make a car go faster, where would you start? Maybe you would put in a bigger engine. But what if the problem is in the transmission? Or the parking brake is locked? Or your tires are completely worn through? Of course, you should start where the problem is. The transition to agile/lean is no different.
Starting with development teams is like starting with the bigger engine. If impediments lie elsewhere, then even if the individual team gets better, overall agility might become less! Even worse, if an agile pilot project ends up causing headaches for the organization, people will turn against the agile initiative.
It is much better to start where the problems are. Here are some ideas that can help address these impediments.
If any of these are not present…
Then focus on this challenge area…
The product management organization maintains a list of prioritized product enhancements.
Product enhancement requests are broken into the smallest pieces that can be released.
The development organization is never overwhelmed by enhancement requests.
Enhancement requests are prioritized individually; they are not grouped into larger packages.
Product portfolio management. Create a prioritized list of smaller enhancements, each of which can be released when completed.
Product enhancement packaging. Keep the number of product enhancements being worked on small enough to not overwhelm your teams.
Cycle time. Focus on delivering value on a regular basis with short cycle times so that the organization can re-evaluate its plans and priorities regularly. Focus on shortening the cadence of product releases.
Effective teams work well together.
Only people with truly specialized skills are working on multiple teams.
The organization is free of “silos” that separate resources from each other.
Management clearly describes the order in which to build and deliver product enhancements incrementally.
Teams can get feedback from the organization and customers quickly, when they need it.
Value. Management must create an environment that supports the creation of value quickly.
Impediments. Many impediments to teams are due to forces outside the team’s span of influence.
Management must act intentionally to remove these impediments on behalf of the team.
Teams are able to deliver software incrementally.
Teams build only what they are requested to build without adding new features in anticipation of the future.
Code is tested virtually at the same moment that the code is built.
Teams write acceptance tests before writing any code.
Flow. Have teams learn an agile process that is based on flow (that is, the elimination of delays). It could be based on iterations or on kanban.
Cadence. Create a cadence of work that delivers value to customers and gets feedback from them at regular intervals.
Work-in-process (WIP). Ensure that the team keeps the amount of WIP as small as possible.
Test first. Ensure that the teams write acceptance tests for a story before writing any code for that story.
Development teams spend most of their time writing code and working with customers to analyze and refine requirements.
Teams easily understand how to introduce a feature into existing code.
Testing and debugging takes place throughout the iteration. Testing and debugging are not a major focus at the end of a development cycle.
There are relatively few errors detected during integration testing.
Test-driven development (TDD). Help teams learn to do this engineering practice.
Continuous build. Have teams use continuous build and integration.
Design patterns. Help teams learn the proper use of design patterns.
Start Where You Can
The good news is that the transition does not require great organizational changes before seeing improvement. You can start where you are. Here are two effective and relatively painless approaches that can be used.
- Limit work-in-process. Limiting the amount of work-in-process (WIP) in those places where people or teams are clearly working on too many things in parallel. This naturally frees up the parts of the value stream that get clogged up and overwhelmed with work and, therefore, create delays. It will have a ripple effect to unblock the flow of development. This will not seem natural at first, but it does work.
- Make product managers your allies. If your company works on many large projects at the same time, you can enroll product managers into modifying the way they manage the portfolio of products. Not only will this focus the organization on building the right products, but it will also lower the amount of work that is almost certainly overwhelming development teams.
You have many more options available to you today than we had in the past for deciding where to start the transition to agile/lean. Increasingly, there are organization leaders and management who recognize the need to become an agile organization. They may have been trained in lean thinking; they may have been exposed to the Toyota Production System and the Toyota Product Development System. They are more likely to see the need to develop strategic plans for the transition to lean-agile software product development. Involving leadership to create a top-down vision with a bottom-up implementation is always better. Developing this top-down/bottom-up approach requires careful, strategic thought. Consider using a coach who is experienced in working with senior leadership. Such a coach can bring to bear the experience of others to create a strategic plan that is likely to succeed and will have the proper organization justification.
In your transition to agile/lean software development, it is critical to keep focused on the goal: organization agility. Creating more agile teams is good but only as long it helps the entire value stream deliver product to customers more quickly and sustainably. Work on those changes that will produce the greatest results overall. This involves looking at all four of these critical areas:
- Team agility
- Technical skills
- Goldratt’s Rules of Flow by Efrat Goldratt-Ashlag. 2023