Accelerating outcomes with a hybrid approach within a waterfall environment

Abstract

This paper describes how large projects use a hybrid agile and waterfall approach to reach the goal of rapidly developing a working software solution.

Many large projects using a waterfall development approach begin with risks associated with an aggressive schedule, vague scope, and resource constraints. Using a hybrid approach can provide benefit, generating a win-win for the project team and the client.

The following areas are addressed:

• Project profile
• Common considerations and criteria associated with selecting the applicable development approach
• Selected hybrid approach
• Details about how the hybrid approach was implemented, including team structure, stakeholder engagement, and project status tracking

The paper closes with a recap of considerations when using a hybrid approach and the benefits that can be achieved by leveraging a hybrid model if a project is not ready for full agile development.

Introduction – Accelerating Outcomes with a Hybrid Approach within a Waterfall Environment

Many projects are launched with aggressive schedules, vague scope, and resource constraints, which raise implementation risks. Agile methodologies have been used effectively to mitigate these risks. However, some projects may not be well suited to adopt full-fledged agile techniques. Some clients face regulatory deadlines that require functionality to be deployed (with certainty) by a specific date; others do not have the cultural landscape established within their organizations to fully benefit from agile.

In these cases, a hybrid of agile and waterfall techniques may be the better approach. The project is managed in a more typical “waterfall” fashion - where scope and requirements are defined up front - but the development and testing are broken down into iterations similar to scrum “sprints” used in agile methodologies. Sprint planning is conducted to prioritize the work of the sprint, and sprint reviews are conducted with the stakeholders to facilitate acceptance of the work accomplished as well as to identify issues as soon as possible in the development cycle.

This paper describes how large projects have used a hybrid agile and waterfall development approach to manage the risk associated with an aggressive delivery schedule and to reach the goal of rapidly developing a working software solution, generating a win-win for the project teams and for the business users.

Project Profile

The focus of this paper is on a composite of large-scale public-sector projects with aggressive deadlines to meet federal requirements. The broad scope of these multi-year projects included:

• Interfaces with multiple systems
• Ability to accommodate high volumes of end users
• Data conversions
• Training and implementation
• Technology infrastructure hosting
• Security
• Business intelligence
• Operations and maintenance

The critical elements of the project scope and the need to work closely with stakeholders to confirm progress throughout the journey resulted in selecting an effective development approach that combined waterfall and agile methodologies. This approach helped the team to meet an aggressive go-live deadline, and to set the stage as a leading practice for similar Deloitte projects.

Selecting an Effective Development Approach

There are many things to consider when choosing an effective methodology for project delivery. Agile methods can provide accelerated delivery of value, the flexibility to re-prioritize scope to meet evolving requirements changes, and enhanced communication between business and technical teams. While many organizations seek the benefits agile can provide, some are not well prepared to make the changes needed to be effective with agile. This is frequently the case in large enterprises where significant legacy and associated integration requirements exist. But there are often opportunities to move toward a hybrid approach, employing agile techniques to improve project outcomes while still maintaining the organizational controls and structure seen in traditional waterfall organizations.

Comparing Waterfall and Agile

Waterfall approaches consist of a set of predetermined phases. Work progresses sequentially through each phase, and the final software is delivered at the end of the phases.

Exhibit 1 – Waterfall Development Approach

Strengths of the waterfall approach include:

The waterfall approach also has drawbacks, including:

Agile approaches promote “adaptability” over “predictability,” measure progress by operational software (vs. % complete of work), and focus on iterative/incremental software delivery.

Exhibit 2 – Agile Development Approach

Strengths of agile include:

An agile approach enables the solution to evolve with adherence to predefined timelines and budget constraints in order to provide high-quality value in the shortest amount of time. In contrast, the waterfall methodology adheres to careful scope definition and delivers to plan.

Exhibit 3 – Agile vs. Waterfall

Pure agile methods are appropriate for projects where the scope is unclear or is expected to undergo frequent change. Projects that tend more toward the use of agile are generally:

And adopting agile comes with challenges, including:

The Hybrid Approach

Projects frequently opt for a hybrid approach, blending the preferred traits of both waterfall and agile development methodologies to decrease risk and increase stakeholder feedback and impact throughout the project. The model leverages five phases - planning and initial requirements and design, iterative agile sprints, and quality assurance (QA) and deployment.

When selecting the hybrid approach, the project team looked at the solution process much like creating a picture designed from puzzle pieces.

Exhibit 4 – Hybrid Development Approach

Agile sprints were used in lieu of traditional waterfall development and test phases. Each sprint addressed activities for additional design elaboration, development, and testing, including some UAT. Using a hybrid approach allowed the project to compartmentalize the different areas of the solution while enhancing collaboration between the delivery team and the client - especially during the development sprints where early and frequent feedback from the client is critical for confirming the solution throughout the project - verses waiting until the final QA phase.

Implementing the Hybrid Approach

Planning and Initial Requirements and Design Phases

The project begins using waterfall for planning and conducted Joint Application Development (JAD) sessions during initial analysis and design with requirements being revisited during future agile sprints. Requirements and design are developed up front to provide a solid foundation for the project, and to promote a clear understanding of scope. During this phase, the team works with the client stakeholders to define and refine the requirements and design for the to-be solution. Roughly 80% of the requirement and design decisions are finalized during this phase, leaving about 20% to be worked through during the iterative sprint phase. Future sprint tasks were defined based on requirements that turned into logical work units and how they interact. Complexity and predecessors were also assigned at this time to help with parsing out work for the upcoming development sprints.

Iterative Sprints

Iterative development sprints were scheduled for three weeks each over a seven-month period. The iterations are planned based on defined scope and priorities, and at a high level consist of repeating finalize design + build + test activities. While agile projects use “time-boxed” sprints, our approach recommends that iterations are “scope” bound, meaning that the planned scope of one iteration is completed before the next iteration can start. This is done to confirm that dependencies are well managed and committed scope is completed on schedule.

During each sprint, individual system tests are run and when they are completed, integration testing is performed based on the related system and use cases. For example, when testing an invoice you should test the login process as well as test how the data moves through the invoice process user interfaces. The development teams release a working build each sprint and each build is tested although end-to-end testing is done in the later UAT phase. Each sprint results in potentially shippable, thin slices of functionality. The “partial UAT” during each sprint represents how the client was involved with the testing process during each sprint. The client reviewed and accepted integration test scripts that were created during each sprint and will again later be executed during UAT. The rapid and collaborative pace of design and development during this phase is used to surface concepts promptly, and allows for adapting based on feedback in a responsive way.

QA and Deployment

Dedicated test cycles are planned after the iterations to thoroughly test and shake defects out of the solution prior to the final go-live. Testing cycles during this phase include functional, integration, performance, usability, and UAT testing. Once the QA phase is completed, the solution is deployed.

Team Structure

The recommended structure of the project leadership team is depicted in Exhibit 5. The Project Manager has overall responsibility of the project and is the primary contact with the client, with the Account Manager designated as executive advisor and the Deputy Project Manager accountable for day-to-day management and reporting, acting as the lead not only for the PMO but also for the Development, Test, and Training and Deployment leads. Despite the hybrid approach, the organization chart is typical to many large projects.

Exhibit 5 – Project Team Structure

The PMO created a work plan aligned to the over-arching waterfall phases with the sprints defined instead of traditional “development and test” phases. Guidelines defined for each sprint included:

• Sprint timelines should be adhered to
• Tasks that are not completed should be deferred to a later sprint
• Unit test scripts should be created and executed as development tasks are completed
• Tools for tracking sprints should be updated frequently and kept current

The roles and responsibilities for the agile development and test sprints are depicted in Exhibit 6. The Deputy Project Manager held the role of Scrum Master. The responsibility of the Deputy Project Manager is to be the voice of the stakeholders and prioritizes requirements for each iteration backlog, while also facilitating the Scrum process.

Exhibit 6 – Agile Sprint Roles and Responsibilities

Having strong project management experience was instrumental to the project management office (PMO) and the development and test leads who helped manage the work due to the fact that planning (or re-planning) and design for the next sprint took place during the current sprint. Although development and test worked simultaneously during the sprints, they focused on their individually assigned tasks.

Stakeholder Engagement

Delivering a large, complex project in a manner that addresses quality and schedule expectations requires frequent, formal and informal communication among team members. Each team lead had a designated business user subject matter expert (SME) counterpart that they communicated with directly to enhance a collaborative working environment. Stakeholder engagement throughout the process provides continuous feedback on build, quality assurance and release readiness.

Tracking Sprint Progress

Burn down charts like the example shown in Exhibit 7 were used for each sprint to track the “burned” or “used” effort expended against the amount of effort estimated for the set of tasks within a sprint. If the effort is not getting “burned down” as planned, then the tasks related to the remaining effort should be reviewed during the scheduled sprint meetings and re-planned for a future sprint as needed.

Exhibit 7 – Burn Down Chart Example

Sprints were also tracked in a “Sprint Task Progress” chart as depicted in Exhibit 8. The projected overall timeline helped with staffing the appropriate resources for each sprint. There was a tendency to push back on more complex tasks during early sprints that resulted in exponentially more effort expended for each subsequent sprint.

Exhibit 8 – Sprint Task Progress Chart

The PMO created the sprint tracking reports for project leadership on a weekly basis. Daily sprint meetings were led by the Development and Test Leads to track progress of each task and keep the agile-related tools updated as the data fed into the weekly PMO reports. Pre- and post-sprint meetings were held with the PMO and project leadership to plan future sprints and review lessons learned from each.

The Rubber Meets the Road

What Worked Well

Using the hybrid approach allowed the project team and the business users to see the realization of the big picture that they envisioned throughout the project versus waiting until the end. Much like putting a puzzle together, as the sprints progressed the overall picture became clearer to everyone. Additional areas that worked well for the project included:

Challenges

With the hybrid approach, the project also encountered some challenges whose resolution fed into developing leading practices for similar projects, and included:

Summary

Considerations for Selecting an Effective Approach

There are many organizational implications to adopting agile, and to introducing agile approaches in a hybrid situation. Consider the following when assessing whether agile is a good fit for the organization, or which agile techniques might be incorporated into a hybrid model:

Benefits of a Hybrid Approach

With the appropriate staff that has strong project management experience, along with dedication of your business user SMEs to work closely with your team, there are benefits to using a hybrid development approach including:

As used in this document, “Deloitte” means Deloitte Consulting LLP, a subsidiary of Deloitte LLP. Please see www.deloitte.com/us/about for a detailed description of the legal structure of Deloitte LLP and its subsidiaries. Certain services may not be available to attest clients under the rules and regulations of public accounting.

This publication contains general information only and is based on the experiences and research of Deloitte practitioners. Deloitte is not, by means of this publication, rendering business, financial, investment, or other professional advice or services. This publication is not a substitute for such professional advice or services, nor should it be used as a basis for any decision or action that may affect your business. Before making any decision or taking any action that may affect your business, you should consult a qualified professional advisor. Deloitte, its affiliates, and related entities shall not be responsible for any loss sustained by any person who relies on this publication.

Copyright © 2013 Deloitte Development LLC. All rights reserved.

Member of Deloitte Touche Tohmatsu Limited