Disciplined Agile

Lean Portfolio Management: Maximizing Value Across Many Programs

One of the biggest challenges of a large enterprise is to choose the set of projects that will realize the greatest return on investment (ROI) possible. To achieve this, organizations use some form of product-portfolio management. This activity is usually beset by shortcomings such as delays and staleness by the time the products are released. Here we present a better approach based on lean thinking.

Here are the main points in this article.

  • The product portfolio should consist of minimum business increments (MBIs) to enable quick delivery of small product enhancements.
  • Use MBIs to obtain maximum business value across the entire portfolio.
  • Teams should be organized so that they can work efficiently on product enhancements.
  • Minimizing work-in-process (WIP) improves efficiency while reducing risk.
  • Increasing customer value must encompass all product lines.

Portfolio Management

In smaller organizations, there's normally one or two projects going on at any given time. Then it makes sense to focus most improvement resources on bettering the product-development process. The choice has already been made about programs.

By the time an organization is large-scale, however, a bigger payoff comes from selecting a mix of products and release scenarios that creates the most value for the enterprise. This is called “portfolio management.”

Portfolio management applies to both product-development organizations and IT organizations. Each type of organization has its own set of challenges, but the general approach, which we describe here, works for both. You must adapt it to your own situation, of course.

Project Portfolios

Virtually all large organizations have a portfolio of projects that they must manage. The mix of these projects is designed to maximize the return on their software-development investment. Ideally, the most important projects would be selected and then given to the development teams; but the larger the portfolio, the harder it is to administer.

Project Portfolios Are Idea Inventories

Administering the project portfolio involves a planning life cycle. The organization uses the life cycle to identify the sources of greatest ROI and then defines a plan to achieve it. It is common to use a yearly project planning cycle. An unintended outcome of the annual cycle is an ever-widening gap between what delivery organizations are working on and what the business needs. The wider the gap, the staler and the less relevant the projects become. The annual cycle becomes a sinkhole into which ideas descend.

Think of project ideas as a type of “work-in-process” (WIP). As long as an idea is in process, the business does not realize value from it. The quicker you can turn it into a real product, the sooner it has value.

What causes excess WIP? Planning teams start out with great intentions. They develop tight business cases and project visions during the project vision “season.” Program managers lead value-driven discussions about the capabilities and features they need to respond to market opportunities or competitor threats.

Then they enter the technical analysis season. All that high-bandwidth communication between business and development degrades while technical organizations take time to analyze the requirements. Work drops into various technical silos (QA, mid-tier, UI, and the like). Technical integration is pushed off farther and farther. It becomes a downward spiral—people need to be kept busy while they wait for answers to planning questions; the busier they are, the longer it takes them to answer other questions.

Business is savvy about these delays. If they anticipate long delivery cycles, they may end up piling on poorly prioritized requirements (the kitchen-sink effect) in the hope of getting some work that is useful. The more work they give, the larger the portfolio grows. And the larger the portfolio, the harder it is to administer. And the harder it is to administer, the more delays there are. And so on. It is a vicious cycle.

Should We Avoid Delays by Batching Project Analysis?

To address these delays in analysis, technical organizations often batch together various business requirements that seem to involve similar technical issues. Then, the technical analysts, who are always in short supply, can focus on those common issues at one time. That seems to be more efficient for the technical resources, but it significantly delays overall delivery and decreases the efficiency of the organization as a whole.
When you batch requirements together, you hold up the release of the most important one(s) in the batch until the least important one is ready. If there is a delay in one part of the bundle, the whole package is delayed.

The obstacle comes from letting projects drive planning, focusing more on minimizing cost and maximizing resource utilization rather than focusing on speed, reducing delay, and realizing ROI quickly.

Should We Avoid Delays by Increasing Release Frequency?

One approach to narrowing the gap between idea and delivery—between concept and consumption—is to increase the frequency of a product’s releases. This is good in that it gives the business a predictable schedule of changes. And it gives the appearance of rapid delivery. But if there is still a 12- to 18-month gap between idea and delivery, there is still significant delay. The ideas are still stale.

Lean Portfolio Management

Lean thinking offers a way out of the vicious cycle that can come with managing a large product portfolio. We start by thinking about the relationship between the portfolio’s needs and the development team’s needs. There is a pipeline and a feedback mechanism between the two. The goal is to promote a fast-flexible-flow of work while selecting projects that return the greatest value to the organization. Lean thinking says to focus on sustainable speed by delivering the most important aspects of a system first, by minimizing work-in-process, and by limiting the amount of work to the organization’s capacity. Lean addresses the risk in software development by creating feedback and focusing on value rather than by the classic method of planning everything up front.

Lean directs us to select smaller projects to work on whenever possible. This does not mean picking small projects from the batch that has already been identified; rather, it means that, early in the planning, projects are defined as small as they can possibly be. This ensures that essential projects are being worked on all the time. Projects that are too large run the risk of bundling together features that are less important to the organization overall.

This approach delivers value to customers more quickly by prioritizing which business features to work on based on business value and then managing the project in a visible portfolio. We call this lean portfolio management.

Why It Works

The lean portfolio-management approach allows stakeholders and clients to identify and prioritize features that create the highest ROI for the business. The lean organization is structured so that cross-functional teams can review and break down both business features and system dependencies to build minimal marketable software solutions. As opposed to the traditional approach, which is task-based and resource-driven, the lean portfolio approach is a results-based, validation approach. Status reviews are not based on tasks completed but instead on validation of technical results.

This solves the dilemma that delivering sooner seems to require more bodies or lowered quality. Lean thinking says that concentrating on delivering the most important features of a product maximizes both team efficiency (by eliminating task switching and waiting times) and team effectiveness (by working on the most important parts of the product). By focusing on small, marketable features, with the goal of getting completed software all the way through the system, an agile team immediately improves efficiency by avoiding the costs of multi-tasking and its subsequent thrashing. Furthermore, critical projects are not slowed down by working on less important projects in parallel. Additional benefits are achieved because agile methods quickly expose impediments to rapid delivery that are normally hidden in large Waterfall projects, where processes to transform handoffs create the illusion of control.

It Is OK to Plan Releases

On the other end of the spectrum, some agile practitioners believe teams should stay focused on the current work iteration and never look ahead (or not too much ahead). This attitude has led to many failed agile teams. While lean agrees that we not to look too far ahead, it recognizes that the decisions that need to be made early still need to be made early.

The lean product portfolio enables priorities to be set, and elaboration of details to occur, at the right, responsible moment. Value is realized incrementally. As more is learned about requirements, that knowledge is fed back into the planning for future increments.

Thus, agile methods allow a learning organization to emerge, which results in predictable estimation of features described at the capability level. These features can be deconstructed in advance of the iteration in which they are implemented by establishing lean flow that is conceptualized as the planned release of features.

With Existing Systems, Plan to Use Incremental Delivery

Building and delivering incrementally works when converting existing systems, too. Customers may say that they “want everything we had in the old system.” But that means they haven’t done an analysis of which features have been valuable. It is extremely unlikely that every feature in the existing system is truly valuable. By building in steps, business value drives and it allows the development team to focus only on high value, required features, and not to waste time converting the system’s unneeded features (or worse, bugs). It also allows the business to embrace market changes and opportunities that arise during the conversion project.

The Benefits of Lean Portfolio Management

This section describes benefits that lean portfolio management provides to a business.

Speed and Quality

A development organization that is truly in sync with its business customers positions itself to help identify minimal marketable features and is structured to be able to release them quickly. Organizations successfully making the transition to agile/lean discover that attempts to deliver quickly expose delays that are impediments to both effective and efficient product development. This enables the organization to adjust and to remove these delays and so deliver value more quickly. When the entire enterprise is focused on speed, market opportunities can be leveraged, and threats bypassed; the result is rapid savings and realized profits. To accommodate this, the development team must begin to see delivery and quality as sustainable activities that are constantly improved by short-cycle feedback loops. Once time-to-market becomes the focus, actions taken to reduce delays cause quality to go up since these delays are the cause of quality problems. The shorter cycles also reduce the development cost by eliminating wasteful steps.

Another benefit of focusing on speed is that the team’s understanding of what they are building increases and they can therefore avoid building what they don’t need. This creates less-complex systems because they are smaller—again, resulting in higher quality. Creating smaller, high-quality modules creates an upward spiral of competitive strength by enabling future changes to be more easily manifested.
This does require that the technology organization understand agile design patterns and test-driven development. These practices allow teams to create change-tolerant architectures that give confidence to make aggressive design changes, since the architecture supports them, and suites of automated regression tests allow verification that nothing existing is broken as changes are implemented.

Line of Sight to Business Needs

Think of the lean portfolio as a set or container of capabilities, defined at a high level, that the business requires to implement its market or response strategies. These capabilities map loosely to capability statements called out as business solutions in the project vision statement or a typical project charter. Call these capabilities “business features.” The lean portfolio gives the entire business and development organization a focal point, in which line of sight to the business needs is established for all work undertaken, with the goal of minimizing work-in-process and completing each capability as quickly as possible.

The lean portfolio can be tracked for reporting, but the real value to the business organization is that it creates a visible representation where business features are listed to establish both priority and technical effort required (which continually must be traded off).

Minimizing Work-in-Process

Building in smaller chunks has the side effect of minimizing work-in-process and enabling the formation of well-defined, agile teams that work on one feature after another. When they complete one, they pull the next one from the prioritized list of features. This enables teams to work on—and complete—one product or enhancement at a time, which ensures that they are always working on the highest-priority product enhancements.

Smaller chunks make it easier to manage key resources that must be shared across teams. If the work is smaller, there is less contention for these precious resources, so thrashing is minimized.

Minimizing Interruptions

Interruptions are also more easily handled when working in small chunks. When team members are involved in many projects at once, it is tempting to interrupt the team when a manager has an urgent task come up—just to get the task done—when there is no clear break in work activity, or the breaks are too far in the future.

When teams are working on smaller chunks of functionality, it is more likely that the manager’s urgent task can wait until a team is done with its current work. This avoids forcing the team to multi-task, which increases thrashing and lowers efficiency.

The Approach

The basic approach for lean portfolio management starts with a breakdown of business features. We will use these business features to transition from a collection of project visions to lean portfolio management. Instead of batching up capabilities into a large project, lean organizations can deliver minimal marketable feature sets. This requires cross-functional, continuously integrated agile teams pulling prioritized work based on capacity.

Taking all the business features for all the projects, we can get a plan for our product development.

Select those business features that will return the greatest value.

Developing a business feature from another product line when it will return more value to do so if our understanding of the world and our projects does not change, the team keeps to the schedule and develops the next most important business features as originally planned.
At some point, a business feature of another product may become more important than some feature in the current plan. Business conditions change all the time; something is learned that gives it more urgency. Because we are using short release cycles, it is easy to modify the plan and give that new business feature a higher priority so that it will be worked on next.

This even works for new features that were not part of the original plan. Just insert the new feature and give it higher priority.

The point is to focus on building those functions that are most needed and most important, and to avoid building features that are less useful. The iterative approach of focusing on the most valuable business capabilities enables us to take advantage of the Pareto rule: Find the 20 percent of the work that provides 80 percent of the value.

With this approach, it is straightforward to create a prioritized product backlog for each team. Each team has a product backlog that lists its features across the top and the story breakdown under each feature. The priority of each feature on the team’s backlog must align with the priorities of the overall portfolio. The priorities are shown graphically: Higher-priority features appear on the left and lower priority features on the right. 

Shorter Planning Cycles

You can still realize significant value without going that far by focusing on the time delays that occur in the regular annual planning cycle.
A typical annual planning cycle runs from July 1 to December 31. This means that we spent the last six months of this year collecting ideas and developing plans for work that begins next year. Then, in January, any new ideas that come along go into the hopper to wait until the next planning cycle, next July 1. If you have worked in IT for any length of time, this is not surprising.

But what is the result? It is that the ideas we are working with are likely stale. To compute the average amount of time it takes from when an idea enters the planning cycle until it is completed, add the following:

  • The average time it waits for planning.
  •  The time it takes for planning.
  • The average time it takes for completion.

Assuming everything gets done in a year, each of these steps takes 6 months, which means that the entire time from idea to delivery averages 18 months.

Ideas Planning Development

Timing and delays of quarterly planning cycle age of the items waiting to be planned is only 1.5 months. The planning cycle is three months, and with a quarterly development cycle, the average time until implementation is 1.5 months. Quarterly planning, even with the same development methodology, reduces the delay between idea and implementation from 18 months to 6 months.

Estimating and Tracking Progress

Traditional portfolio management often tracks progress against a plan rather than against value created. Progress means finishing various engineering phases. How often have you seen projects that show “green” status through requirements, analysis, design, and build stages—successfully finishing each phase on time—but then become “red” overnight when it comes to release?

In lean thinking, the most valuable indicator of status is working software. Building smaller, complete pieces of functionality simplifies the principles of continuous integration. Furthermore, working software exposes otherwise unseen problems that will not reveal themselves until too late in the development process. Avoiding long integration cycles and hidden snags is a great way to prevent waste.

By creating a lean portfolio of business features, businesses can focus on prioritization and can clearly see business value versus cost. This is accomplished because teams learn how to estimate with just enough accuracy for businesses to determine the best value returned from effort. An effective practice for institutionalizing this estimation skill is by using story points (Cohn 2005). The portfolio view simplifies budget decisions because funding centers can be determined by what percentage of features (along with costs) are staged in the portfolio. Once Agile teams establish their velocity, planners can create accurate release schedules that give clear time-to-market for business goals.

Try This

Here are some exercises to try. These are best done as a conversation with someone in your organization. After each exercise, ask each other if there are any actions either of you can take to improve your situation.

  • Think about the projects you have worked on in the past.
  • Was there a subset of these projects that could have been released without all the features of the project that was released?
  • What would have happened had you released the product enhancements in smaller increments?
  • Are there reasons other than the product itself (for example, cost of release) that prevent partial releases?
  • What should you do about these?
  • How many projects at a time does the typical person in your organization work on?
  • What does this accomplish and what does it cost?
  • Do key people work on even more projects than a typical person?

Related Reading

  • Aral, Sinan, Erik Brynjolfsson, and Marshall W. Van Alstyne. 2008. What Makes Information Workers Productive.
  • Beaver, Guy. February 11, 2008. Knocking Down Silos: Transitioning the Enterprise to Agile.
  • Cohn, Mike. 2005. Agile Estimating and Planning. Upper Saddle River, NJ: Prentice Hall.
  • Poppendieck, Mary, and Tom Poppendieck. 2003. Lean Software Development: An Agile Toolkit. Boston: Addison-Wesley.
  • Reinertsen, Donald G. 1997. Managing the Design Factory. New York: Free Press.
  • Shalloway, Alan. 2008. “Lean Anti-Patterns and What to Do About Them.Agile Journal.
  • Townsend, Patrick L., and Joan E. Gebhardt. 2007. How Organizations Learn: Investigate, Identify, Institutionalize. Milwaukee, WI: ASQ Quality Press.