Moving to agile in a waterfall world
a story of agile adoption at Kroger
Reedy Feggins, PMP, CSM
Like many other traditional brick-and-mortar companies, The Kroger Co. has sought ways to improve its ability to delivery software solutions better, faster, and more cheaply as a means of staying ahead of its competition. Along the way, Kroger had cultivated a deeply rooted waterfall development process due to the constraints of their more complex project environments; as a result, Kroger has struggled in its transformation to adopt agile practices. This case study will share the company's experiences in transitioning from a waterfall approach to become a more agile organization — with full knowledge that a unilateral adoption of agile practices would not be effective in this environment. Kroger's experience illustrates how a hybrid approach (one that brings together a combination of traditional and agile practices) that focuses on key business drivers and a pragmatic agile adoption framework, can reap tangible results within the context of an existing waterfall process. This case study will also share details of the agile pilot that was conducted to validate the benefits of using disciplined agile practices within Kroger. It will discuss how a methodical, real-world agile approach enabled Kroger to accelerate its software delivery cycles by 18.5% while using less than 12% to 13% of the projected budget, per release, all while significantly increasing product owner and customer satisfaction.
“Ambition is a very pretty thing; but sir, we must walk before we run.”
1851, G. Borrow Lavengro II. ii.
This truism applies to many areas of life and is highly appropriate for large organizations when attempting to adopt agile within a traditional software development environment. This case study explores what transpired when The Kroger Co. began the transformation journey with a service-provider with agile experts that assisted them in developing an enterprise agile transformation program that would fit their existing corporate culture. The program would pilot a disciplined agile delivery framework that would provide pragmatic, realistic measures to evaluate the effectiveness of agile practices on the company's development projects.
The Kroger Co., headquartered in Cincinnati, Ohio, USA, is one of the nation's largest grocery retailers, with fiscal 2011 sales of US$90.4 billion. Spanning many states, with grocery and multi-department stores, convenience stores, and mall jewelry stores, Kroger operates under nearly two dozen banners, such as City Market, Dillons, Food 4 Less, Kroger, and King Soopers.
Kroger's Software Development Environment before Agile
Software development is a critical part of Kroger's business model today and, in many regards, has become viewed as a competitive advantage for the company. Over a four-year period, Kroger embarked on a strategy to improve its software processes, to move away a silo-tiered IT organization to a service-tiered organization, in which infrastructure and enterprise-wide IT resources were grouped into a set of shared services (Exhibit 1).
Exhibit 1 – Service-Tiered IT Organization
In addition, the application development function was further re-organized by functional area (grouping project management, business analysts, architecture, and software development teams into discrete teams) in which line managers were replaced by resource managers responsible for staffing. Finally, all capital projects were required to adhere to more rigorous project controls. Unfortunately, these changes failed to achieve the desired results at Kroger. After a four-year journey, teams were struggling with collaboration, product quality had not improved, and project delivery time frames continued to average 30 to 41 months (see Exhibit 2).
Exhibit 2 – Typical Kroger IT Project Schedule
Admittedly, the company's waterfall development approach worked well for projects that had well understood timelines, domain space, and technology, but often failed to meet business expectations in newer, innovative areas. In general, Kroger's application development teams struggled to deliver value quickly, and most projects ended up costing more than expected and consumed unplanned resources.
Why Kroger Chose Disciplined Agile
Kroger's journey of agile adoption started long before they ever engaged a service vendor for help. Experimenting with agile, several IT resources obtained ScrumMaster certifications and started piloting agile practices, but they ran into a variety of implementation difficulties, which resulted in Kroger's leadership defaulting back to their traditional development approaches as the “least risky option.” They cited several failed attempts to adopt agile practices in the past due to a lack of:
- appropriate transparency and visibility within the agile projects
- inability of agile teams to “stick to a plan”
- any meaningful measures/evidence showing that agile projects outperformed waterfall projects
Regardless of these initial results, some members of the software development community felt confident that agile practices could indeed be successful at Kroger, given the right level of planning and modification based on the company's unique requirements. Knowing that the service vendor had itself been undergoing an agile transformation initiative, the Kroger team sought the counsel of this vendor's services organization for assistance with its agile adoption.
Working with Kroger, the service vendor evaluated its software development practices to identify scaling factors that could be preventing the successful introduction of agile practices. They determined that practices such as Scrum and XP could be effective on agile projects but would require significant modification. The service vendor's Agile Scaling Model was used as a reference point to determine how to effectively adapt and scale classic agile practices. During an assessment of Kroger's environment, the vendor discovered that the following scaling factors affected most of the large development projects within the company (Exhibit 3):
- The teams were relatively small but geographically distributed across various locations;
- The company had specific project/governance disciplines that must be followed;
- Project environments required the support of many operating platforms, including legacy environments;
- The product domain was innovative and rapidly changing;
- The organization was highly distributed, involving multiple internal organizations and independent third parties; and
- The organizational structure was quite rigid and steeped in existing procedures and practices.
Exhibit 3 – Scaling Factors at Kroger
Kroger's Enterprise Agile Adoption Strategy
Enterprise transformations are difficult at best in large enterprises such as Kroger, and they typically require a gradual and phased adoption strategy that permeates the organization over a period of time. Armed with this organizational insight, Kroger realized that they would not be successful with a haphazard approach to their agile adoption. Instead, Kroger put in place a deliberate and strategic agile adoption plan based on a hybrid delivery model that applied an iterative and incremental approach to agile transformation based on a common set of guiding principles:
- Maintain a focus on business value/results
- Drive the vision for adoption from the top down and implement from the bottom up
- Adopt agile practices iteratively in waves
- Adapt quickly and embrace new practices as projects and teams mature
- Strive for quick-wins and modify accordingly based on lessons learned (retrospectives)
Kroger's development team divided its multi-year agile transformation program into multiple waves, each lasting approximately 9 to 12 months. Each wave of its agile evolution would require commitment and action at multiple levels within the organization (see Exhibit 4).
Exhibit 4 – Agile Transformation Roadmap
Wave 1 Objectives
During Wave 1, the Kroger team set several goals: (1) assess the organization to identify the current state (current software practices, scaling factors, and organizational challenges that could impede their agile adoption); (2) define an initial agile framework to pilot in conjunction with their traditional process; (3) create a set of criteria, or decision matrix, to identify agile-friendly projects; and, (4) select a pilot project that would implement the new delivery framework using metrics to determine its effectiveness in delivering business value quickly. In subsequent waves, it was planned that the service vendor and Kroger would use feedback from Wave 1 results to update and adjust Kroger's adoption strategy, agile framework, and decision matrix as necessary, and would conduct additional pilots to expand the program in different areas of the business.
Agile Practices Employed
Given the organization's previous issues with agile and, more specifically, Scrum, and given the complexities inherent in Kroger's development projects, Kroger decided to implement a Disciplined Agile Delivery framework and apply an Agile Scaling Model, two methodologies that were created and espoused by Scott Ambler, Senior Consulting Partner for Scott Ambler and Associates. Disciplined Agile Delivery (DAD) is defined as:
“…an evolutionary (iterative and incremental) approach which regularly produces high quality solutions in a cost effective and timely manner via a risk and value driven life cycle. It is performed in a highly collaborative, disciplined, and self-organizing manner within an appropriate governance framework, with active stakeholder participation to ensure that the team understands and addresses the changing needs of its stakeholders to maximize business value provided. Disciplined agile delivery teams provide repeatable results by adopting just the right amount of ceremony for the situation which they face.”
The Agile Scaling Model (ASM) is a contextual framework for effective adoption and tailoring of agile practices to meet the unique challenges faced by a software or system delivery team of any size. There are a total of scaling factors that address a wide variety of agile bottlenecks, including geographic distribution, regulatory compliance, technical complexity, and others. Using these scaling factors as a guideline, the service vendor conducted an assessment where the following scaling factors were identified: geographic distribution, enterprise discipline, problem domain, project complexity, and organizational structure. Based on the existence of these scaling factors, the team implemented the following agile practices (Table 1):
|Discipline Agile Delivery Practices||Description Scaling Factors|
|Risk-value Life Cycle||Expand to encompass software delivery phases as opposed to just the Construction phases supported by Scrum.|
|Release Planning||Iterative Development||Core practices also found in Scrum|
|Whole Team||Team Change Management||To help overcome the organization structure factors related to team collaboration.|
|Shared Vision||User Story-Driven Development||Core practices also found in Scrum.|
|Business Value Focus||A measure to ensure working software is planned for, developed tested, demonstrated, and delivered at the end of each iteration.|
Table 1 – Scaling Factors
Risk-value Life Cycle
Scrum is an agile project management framework that works in conjunction with other agile practices to provide teams with the following capabilities: time-boxed iterations, product backlogs, user stories, sprint plans, daily meetings, sprint demos, and sprint retrospectives so teams can build and deliver small chunks (iterations) of “potentially shippable code.” DAD expands the Scrum life cycle by incorporating the entire software delivery phase, not just focusing on the development organization. DAD acknowledges that the development team is dependent on other critical enabling organizations (verification testing, integration testing, IT operations, and so forth) in order to get its software into the hands of customers. When implementing its agile approach, Kroger had to make several modifications to “pure” agile practices to accommodate the company's constraints: project funding model, existing organizational structure, resource allocation, co-location, vendor relationship, and the release the project was in when they began the agile pilot.
Although Kroger's first wave of agile adoption was deemed as a success, it wasn't without its challenges. Moving to a disciplined agile delivery framework was a big adjustment for the pilot team. Culture wars ensued in which some team members wanted to continue with “business as usual” (i.e., maintaining traditional practices), whereas other team members wanted the flexibility to use Scrum techniques' “out of the box” with minimal coaching. Change is painful, and all of these issues had to be overcome gradually.
Like any large organization, Kroger continues to struggle with ongoing agile adoption issues, including:
- Organizational Structure: Kroger has a three-tiered service model (see Exhibit 1), which has constrained their ability to create “cross-functional” agile teams due to the strong established roles. The company is still working through organizational changes to support an agile process.
- Project Funding Model: The front and back ends of Kroger's development life cycle still practice Waterfall processes, which limits the ability to do rapid development iterations. For example, approval of projects occurs twice a year, and projects must come to within +/‒ 5% of total funding to gain approval, which necessitates that ‘all’ requirements and design activities are completed. Kroger recognizes the need for a more flexible funding approach to fit with a development model of more frequent development iterations.
- Internal Governance: Kroger's project management office (PMO) was not initially involved in the company's agile transformation and resisted it, but there is a recognized need for the PMO to be more engaged as a facilitator and enabler of an agile process across the software delivery life cycle.
- External Governance: Kroger is required to conduct end-to-end testing of their solutions and has, at times, submited those activities to third parties for validation before the new solution is promoted to production. This governance must be incorporated into the overall agile process to get software quickly into the hands of internal and external customers.
- Team Dynamics: During the retrospective meetings, the team struggled with understanding the value of estimating using story points and iterations that included the external team (people who are not parts of the agile pilot core development and test team). Success with DAD will mean the incorporation of an expanded group of stakeholders who are responsible for overall software delivery.
- Estimating Effort: Kroger is refining its agile practices in the areas of estimation, including grooming the product backlog, conducting sprint planning as story points, converting points to hours, and using agile measurements, such as product owner acceptance, velocity, total defects, and release burn-down charts.
- Use of Agile Planning Tools: The pilot team expressed frustration that there were “too many tools” involved in getting their jobs done; the tools the team did have access to were not being used effectively. The service vendor and Kroger piloted the use of an enterprise agile project management tool to help manage the product backlog and improve team collaboration. The Kroger team continues to struggle with what is the right set of enabling tools to manage their internal agile development and test activities as well as those with an external third-party development vendor.
Kroger's first wave of agile adoption achieved several benefits as a result of an enterprise adoption framework and the incorporation of DAD practices.
- The increased collaboration between the product owner and the pilot team led to higher business satisfaction, greater levels of organization trust, better overall planning, and an accelerated software delivery cycle by 18.5% of working software (iterations every 2.5 months instead of every 30 to 41 months).
- The pilot team was able to predictably use between 12% and 13% of its projected project budget, per release — all while significantly increasing product owner and customer satisfaction. The pilot project team delivered software that was closer to the product owner's satisfaction and released higher quality products while receiving continuous and frequent feedback from their key stakeholders and customers.
Kroger's story is a classic example of a company once steeped in waterfall processes and is now achieving incremental success and tangible results by incrementally adopting agile practices. Despite the real and ongoing technical and cultural challenges, the pilot reported measurable, positive outcomes in the delivery of working software, which has had a measurable impact on driving additional revenue.
In Kroger's case, the organization simply had too many complexities to adopt Scrum in its purest form. By adapting agile practices in line with Kroger' scaling factors (organizational complexity, governance, functional roles), Kroger was able to attain significant improvements in product quality, speed of delivery, and increased customer satisfaction that were unattainable in their previous attempts at agile adoption. For Kroger, a disciplined agile approach was the only path toward success.
“Enterprise agile adoption is a marathon, not a sprint.”
Gerald Smith, PMP, CSP
Ambler, S. (2009). Agile Scaling Factors. Retrieved from https://www.ibm.com/developerworks/mydeveloperworks/blogs/ambler/entry/agile_scaling_factors?lang=en_us
Ambler, S. (2009). Disciplined Agile Delivery. Retrieved from https://www.ibm.com/developerworks/mydeveloperworks/blogs/ambler/entry/disciplined_agile_delivery?lang=en_us
IBID. Disciplined Agile Software Development: Definition. Retrieved from http://www.agilemodeling.com/essays/agileSoftwareDevelopment.htm
Smith, G., and Feggins, R. (2012). Agile Metrics — Measuring Successful Agile Transformations. Agile Alliance from http://submit2012.agilealliance.org/node/14820.
© 2012, Gerald D. Smith, PMP, CSP
Originally published as a part of the 2012 PMI Global Congress Proceedings – Vancouver, BC, Canada