Balancing agility with conformance on complex government programs
Complex government transformation programs are subject to many of the same challenges encountered on commercial programs of similar size and scope. Government programs face additional challenges that are unique to their environments, such as federal regulations that are continuously evolving as legislation changes. Furthermore, complex programs are never implemented in a static environment. Rather, they are developed in dynamic environments where solutions are implemented across several domains of change over multiple years. During that time, business needs inevitably change, which effectively alters the resulting software requirements, oftentimes, before the software application under development is completed. These complexities introduce a high level of risk that must be addressed efficiently in order for the program to meet its contractual objectives.
Embracing change is the cornerstone of agile software development best practices. The agile methodology, when used appropriately in the public sector, can be an effective means of adapting to change. The goal of this paper is to demonstrate the value of applying agile methods to complex government programs. This is based on experience implementing large scale programs within the public sector. The commercial software industry has proven that agile techniques can both mitigate risk and improve the quality of the product. Agile methods can help a project team adapt to dynamic business requirements. The iterative nature of agile development creates a feedback mechanism that enhances the understanding of business requirements and how the new system can transform the business for the better. The analysis contained herein intentionally omitted consideration of contractual terms and acquisition regulations and their associated constraints. Until federal acquisition law changes, we must seek the right balance of approaches.
Applying agile software development methods to complex programs in both the private and public sectors has proven to be a value added endeavour. The U.S. Chief Information Officer, Vivek Kundra, is keenly aware of this notion as evidenced by his paper entitled, 25 Point Implementation Plan to Reform Federal Information Technology Management (Kundra, December 2010). In this paper, he recognizes that through the use of iterative requirements definition and modular development, large scale information technology programs are able to increase their probability of success while effectively reducing their exposure to risk. The iterative concept is at the core of the agile methodology, where the idea of developing software functionality in time-boxed iterations allows teams to create a continuous and effective feedback loop with the customer. In so doing, this allows the project team to quickly adapt and respond to change during project execution, and thereby, improve the availability and quality of the work products.
Striking the appropriate balance between the various agile software development methods with respect to the underlying contract deliverables is not an easy undertaking. The argument, however, to invoke these techniques is strengthened proportionately with the scope and complexity of the program being considered. Large government programs that employ the discipline to leverage the agile paradigm can benefit by being able to adapt to an inherently uncertain and complex environment. The extent to which a complex government program can successfully implement agile methods in the dynamic federal environment will enable all the stakeholders to mitigate their exposure to risk by providing prototype functionality sooner and improving the quality of the deliverables by exposing product features early and often. This will ultimately lead to enhanced final work products and improved customer satisfaction through a high level of collaboration.
This paper demonstrates how agile concepts and methods can be evaluated and implemented within a complex government transformation program. The agile methods that were applicable were identified and implemented effectively, allowing the program in the case study to achieve the goals of reducing risk and increasing customer satisfaction.
Anatomy of Complex Government Transformation Programs
In the private and public sectors alike, large scale business transformation programs that include custom software development are inherently complex and take an extended period of time to implement successfully. The additional challenges that exist within the government environment impact both complexity and duration, resulting in more exposure to risk. Typical challenges faced by public sector programs specifically include: large organizational scale, dynamic regulatory requirements, strenuous conformance requirements, and acceptance of business transformation. Each of these challenges can impact the complexity and duration of the engagement, resulting in an increased exposure to risk.
In order to describe how various agile techniques can be considered and applied to large federal transformation programs, it is important to appreciate the complexity of the synchronization required across the multiple domains of change. These domains include business process transformation, software applications, technology, and data. Due to the size and scope of these programs, periodic oversight is typically required from various federal audit agencies.
This level of oversight and federal compliance can lead a program to adopt additional checkpoints and rigor throughout the development life cycle, which can be costly for the program. The key is for the government users to effectively define the functional requirements early in the software life cycle. This allows for all the stakeholders to develop a common understanding of the requirements within the collaborative project team. A frequent reality of software-based projects is that the functionality of the final work products does not meet the expectations of the user. This is often due to a gap between the expectations of the user and the requirements communicated to the project team.
Overview of Agile Concepts
In this paper, we assume a fundamental understanding of the concepts of agile development. The Manifesto for Agile Software Development (http://agilemanifesto.org) reads as follows:
We are uncovering better ways of developing software by doing it and helping others do it.
Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan.
While there is value in the items on the right, agile methods value the items on the left more.
In the Federal Information Technology (IT) environment, the shift of emphasis on the items from the right to the items on the left is more challenging than in the commercial environment. Until federal acquisition law changes and major rework is done to the Federal Acquisition Regulation (FAR), programs must seek to balance these approaches in order to achieve the benefits from the items on the left, while maintaining conformance with their respective contract.
Waterfall methods are best suited for projects in which the business requirements are close-ended and clear. Agile methods are best suited for projects in which the requirements are open-ended and not adequately defined by the business community (Hotle, 2008). As described in the prior section, complex programs typically have close-ended requirements as defined by the contract. In addition, the degree of technical and business process transformation is high, suggesting that the program is very complex.
There are six essential characteristics of an agile approach to solving software development problems:
- Iterative Development - Software developed in short iterations allows the team to be adaptive and embrace changes.
- Incremental Delivery - Incremental delivery of portions of the system under development enables end user and customer review and feedback. Often users and customers do not completely understand their requirements, and this feedback validates that understanding.
- Value-Driven - A value-driven approach ensures that the solution remains focused on the highest valued product features. Ultimately, this ensures the best return on investment. A value-driven approach ensures that an initial, valuable product is delivered as early as possible, where enhancements are rolled out with each subsequent release.
- Lean Development - The goal here is to eliminate waste. Waste is defined as anything that does not provide value as seen from the perspectives of the end users or customers. For example, excessive documentation has diminishing returns and doesn’t always contribute to a better product (even if it is required by law).
- Collaborative - Agile approaches require more collaboration between the customer and the development team in order to keep the project properly targeted and focused. Significant benefits can be realized from employing an iterative and incremental development methodology in which a high level of collaboration exists between the stakeholders and the project team.
- Quality-Focused – Quality-focused means delivering deployable product increments with an equal focus on testing, quality assurance, and documentation.
Application of Agile Approaches
The human element of the agile methodology is important to consider (Reinersten, pp 263–265). There are three key human dimensions to agility in client engagements and these are:
- Continuous collaborative engagement
- Rapid execution with a feedback loop to allow for adapting to uncertainty
- Co-located groups with the authority to make Subject Matter Expert (SME) level decisions
Ideally, the iterative process utilizes face-to-face communications, however, agile can successfully be delivered in a virtual and distributed environment. Typically, co-located groups are not feasible for the whole team or for entire project durations. With this in mind, the practical solution, at a minimum, is to engage the key SMEs for that particular iteration. This helps to both shorten and improve the feedback loop. As an alternative, video and teleconferencing can be very useful to review an iterative prototype. For example, Defense Connect On-line (DCO) is a powerful prototyping and collaboration tool.
Ultimately, the purpose of these iterative reviews is to engage the client in meaningful conversations with reference to the latest prototype, thereby validating the business process and software to date. This is an improvement over other methods, such as specification reviews, that typically do not adequately convey the “look and feel” of the “to-be” software functionality. In order for these iterative reviews to be effective, it is helpful to set them up at predictable intervals with respect to each iteration.
Balancing Agility and Discipline
Agile software development methods can be successfully applied in combination with an underlying traditional methodology, resulting in a hybrid approach. A pragmatic rule of thumb when approaching a situation in which there is a need for agility and traditional management disciplines is to balance and apply them appropriately. Established methodologies, such as SEI CMMI® and PMI’s A Guide to the Project Management Body of Knowledge (PMBOK® Guide) readily support using agile methods. In fact, the latest CMMI-DEV publication v1.3 has added guidance for organizations that use CMMI in agile environments.
Applying a combination of core project management principles and agile within complex government programs is best. To be clear, the agile methodology is not, in any way, a replacement to core program management principles. Rather, agile methods should be considered as complimentary to these principles. For example, a time-boxed iteration may expose additional scope for a particular project. This would then need to be managed within the existing change management process described in the program management plan.
Barry Boehm and Richard Turner define a set of criteria and a functional process to help organizations plan how to balance agility and discipline. The custom method defined in the book provides a risk-driven, spiral-type approach for balancing agile and plan-driven methods. It is important to recognize that there is no universally applicable approach. Rather, the details of each program will require careful consideration as to how to best apply the approach within the context of the unique situation.
A recent report by the Defense Science Board Task Force regarding the Department of Defense (DoD) policies and procedures for the acquisition of IT projects proposes that “a new acquisition process for information technology should be developed, modelled on successful commercial practices, for the rapid acquisition and continuous upgrade and improvement of IT capabilities. The process should be agile and geared to delivering meaningful increments of capability in approximately eighteen months or less, increments that are prioritized based on need and technical readiness” (DSB, 2009).
Exhibit 1- A New Acquisition Process for Information Technology
Agile Project Definition in Large-scale IT Program Management
A major root-cause identified by Boehm and Turner in recent troubled DoD software programs is directly attributable to poor expectations management (Boehm and Turner, 2003, P4 Paper). Managing client expectations is paramount to effectively implementing an agile methodology within a complex government program. The earlier the customer is aware of its roles and responsibilities with respect to the agile method, the better. We recommend that collaborative roles and responsibilities be formally documented in the Project Definition in order to clearly define the expectations that are fundamental to the success of implementing an agile approach. The Project Definition has multiple sections; those that are most relevant to defining agile methods include the approach, the critical success factors, and the client responsibilities. In Federal IT Programs, the roles and responsibilities must also be contained in the contract directly or by reference to clearly establish the legal duties per the FAR.
Agile Scope Management in Large-scale IT Program Management
Agile methods can help with scope management on a large-scale IT program. This is due to the added value of the key stakeholders being continuously engaged in the life cycle process (PM Network, 2011). In many projects the stakeholders are only included in the early stages, such as requirements definition, and the later stages, such as acceptance and signoff of deliverables. Oftentimes, when stakeholders are asked to accurately specify their requirements, they do not fully understand what is required to meet their business needs. Moreover, they frequently are not aware of the latest capabilities of the “to-be” software application and how it can potentially streamline their legacy business processes. Furthermore, the requirements may change as business needs evolve during project execution. Throughout a project’s life cycle, requirements can change due to learning more about the business process as well as the software application being implemented. It is important to note that the underlying contract needs to address this under an amplified changes clause in the FAR. (FAR, 52.243)
There are key agile success factors that can be used as a gauge for measuring how well the program is adopting agile methods. Those factors include the following:
- Early and continual involvement of the client, including user representation
- Co-location of joint project team where possible
- Active stakeholder engagement
- Empowered joint project teams with authority to make SME level decisions during iterations
- Assurance that collaboration is fostered between customer and development team throughout the project
Case Study: Balancing Agile Methods into a Waterfall Life Cycle
The journey toward continuous improvement led to an evaluation of the applicable agile software development methods. The evaluation began with the statement, “real systems have no top” (Meyer, 1997). That is, once requirements are defined and a high level design is drafted, according to a waterfall methodology, the design is elaborated through functional and then detailed technical design. The real challenge begins once the developers begin coding and uncover the additional complexities involved in introducing a change into a complex integrated application. The more complicated the change being introduced is, the higher the likelihood that unforeseen complexities will be discovered during project execution.
As the process of discovering these complexities is deferred in a typical waterfall life cycle, both schedule and budget pressures increase as the complexities are addressed. The challenge then becomes a method to introduce ways of identifying the inherent complexities and their associated defects earlier in the development life cycle.
As the program management and various project teams in the case study investigated ways to work more productively and to increase the quality of their work, they focused on the following principles of Lean Software Development (Poppendieck, 2002):
- Eliminate waste
- Amplify learning
- Decide as late as possible
- Deliver as fast as possible
- Empower the team
- Build integrity and quality in
- See the whole
Eliminate Waste - Value Stream Analysis
The team performed a value stream analysis on their development life cycle, focusing on the flow of work and the steps that add value. This analysis intentionally omitted consideration of contractual terms and acquisition regulations. Particular attention was paid to steps and techniques that help to identify defects in design or code as early in the life cycle as possible when they are most cost effective to address.
When operating under a waterfall model, there are various handoffs to diverse project roles. Additionally, there are more handoffs across work streams; business analyst to training developer, training developer to trainer. Also, the work of the development team has the potential to impact work in other domains, such as data migration. Each change has to be synchronized across domains. Furthermore, on a large program, there are a number of stakeholders that evaluate a change and provide input to the process, such as the sustainment organization, the architecture review board, and configuration management.
An exercise in value stream analysis helped the project team in the case study to identify ways to eliminate or reduce handoffs and waiting. By engaging various work streams earlier in the design process, and maintaining a highly collaborative approach throughout the development and testing phases of the life cycle, the work flowed much more efficiently. The team’s developers were brought into the design phase, and business analysts worked closely with them during the development and unit testing phases.
Amplify Learning - Iterative Prototyping and Peer Reviews
Detailed technical documentation of the deployed application is a requirement of the program. In a strict waterfall life cycle, development doesn’t start until detailed technical specifications are created and reviewed. This approach results in developers waiting to start development and thus, delays the discovery of potential design defects hidden by the complex, integrated nature of the application. By deemphasizing detailed technical documentation as a predecessor for development, and instead, focusing on prototyping for design, the team benefited by learning what is feasible from a design perspective and was able to flush out defects much more quickly. By implementing the agile method of time-boxed, iterative prototyping during design and development, the prototypes themselves created a feedback loop with stakeholders across domains and work streams. This enabled the team to continuously learn how the design was meeting the expected requirements.
The prototype events also allowed the stakeholders to get continuous feedback on how the requirements are enabled by the application and how the change impacted their domains. With regard to the requirement for detailed technical documentation of the deployed application, this documentation was viewed as part of the product and was developed based on the developers’ working papers, created during the development effort.
Peer reviews are a formal process used by the program to help identify defects in the design or code. The return on investment by using peer reviews comes from identifying defects early in the life cycle before they are identified as test or production defects. The program in the case study implemented a formal peer review process. Peer reviews should be a collaborative effort between the project team and sustainment organization that is also monitored by the program’s quality assurance organization.
Decide as Late as Possible
As iterative prototyping gained traction throughout the solution delivery organization, prototyping for design found its way into the design phase for the more critical and complex changes. The number of iterations required became a function of the complexity of the requirements and associated changes. During design, prototypes were intended to be light-weight prototypes with minimal coding and were demonstrated to stakeholders in order to validate that the requirements had been met successfully. The more complicated and integrated changes required more iterations to achieve agreement among the solution team and stakeholders on the correct path forward to completing the design and makig the application ready for production.
Deliver as Fast as Possible
In order to maintain a high pace of development and to keep the feedback loop fresh, the teams handled critical and complex changes by targeting prototype iterations of two to four weeks. At the outset of each prototype, the project team in the case study would set expectations with stakeholders and SMEs as to what they would see at the end of the iteration. The most critical and complex requirements were targeted for completion first, while less critical requirements were deferred to later iterations.
Specifically, each development iteration had both “must include” requirements as well as “stretch-goal requirements.” After the prototype session, the feedback was incorporated into planning for the next iteration. This agile approach of time-boxed iterations enabled the team to identify integration challenges and design defects as early as possible. In addition, integrated prototype milestones helped the group to “see the whole” and “build integrity” within the solution.
Empower the Team
To address the most critical requirements with the most challenging schedules, dedicated teams of business analysts, developers, trainers, and SMEs were created to work together from start to finish. The following guidelines were established at the outset of the project:
- Each team member needs to be aware of all the deliverables (software, process documentation, training materials, etc.) and be prepared to contribute to the creation of these products, regardless of role.
- Each team member is dedicated exclusively to the team effort for the duration of the effort in order to eliminate waiting during handoffs
- Time-boxed prototype iterations are established with a scheduled in order to keep the key stakeholders engaged and maintaining a effective feedback loop
- The team constantly adds value to the product
- Business analysts need to be able to think like developers
- Developers need to be able to think like business analysts
- Business analysts need to be prepared to develop and deliver training
- Training developers and trainers become experts in the process and product by being a part of its development and prototype delivery
Areas for Continuous Improvement
One of the principles of the Agile Manifesto (Beck et al, 2001) is that at regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. In agile terms, this is called a retrospective. We have found that as we train our project teams in agile methods and the goals of iterative development, that the language the teams use to refer to their work has changed. As the teams progressively understand more about agile and iterative development, the team members developed a curiosity become even more agile. The project team in the case study eventually adopted a continuous improvement culture when addressing the following constraints to becoming more agile:
- Agile is focused on the early and continuous delivery of valuable software. Continuous delivery of software into the production baseline has the potential to disrupt the organization’s existing business processes or operations. As the software changes, the documentation of the solution must change and users must be trained and ready to accept the change. The project teams are getting better at using prototype events to engage multiple work streams. As the business transformation work streams are engaged earlier in the development life cycle, they can begin their work sooner. This results in better synchronization between the deployment of working software and the organization’s ability to accept the change.
- Agile is focused on welcoming changing requirements, even late in development. In a complex government transformation program, what is even more evident than changing requirements is how the business understands the evolving product changes and how it satisfies their requirements. As agile methods gain momentum within project teams, the teams are continuously improving using prototype iterations to create a feedback loop and adapting the solution to the requirements.
- It is an expectation that for a project to become truly agile, the key stakeholders must collaborate throughout the project. In fact, the more face-to-face interaction between business and development, the better. The breadth and geographical dispersion of the business organization, combined with the expense of travel costs to the project site, limit the ideal face-to-face interaction between SMEs, business analysts, and developers. The project team was, however, able to successfully collaborate by utilizing web-based prototype demonstrations at regular intervals. As previously noted, the use of DCO allowed for effective demonstrations of prototypes that would have been otherwise limited by travel budgets. This has enabled the team to expand the feedback loop to more frequently engage stakeholders that would have been otherwise left out of the loop.
- Peer reviews are a target area for continuous improvement, as there are social aspects to peer reviews that can enhance the return on investment of the process. Agile approaches to peer reviews utilize the momentum of the actual software development process. The project team in the case study emphasized a continuous collaborative process as opposed to formal checkpoints. This approach required a cultural shift within the program organization to consistently achieve a collaborative environment. As a result, there proved to be a greater return on investment as opposed to merely a checkpoint in the life cycle. The overall goal for the peer review process in the case study was to not only produce a greater return on investment with regard to the identification of defects, but also to amplify learning and collaboration within the context of the entire program.
The iterative nature of agile methods can be used to both adapt to dynamic environments, as well as help clarify requirements. This is especially important within the context of large public sector programs with multiple domains of change and a variety of key stakeholders with competing priorities. Striking the appropriate balance between the various agile software development methods with respect to the underlying contract deliverables is not an easy undertaking. The argument, however, to invoke these techniques, is strengthened proportionately with the scope and complexity of the program being considered.
It is important to note that there is no “one-size-fits-all” way to implement agile methods. The program managers and clients alike need to appreciate the complexities of their particular engagement and apply the agile methods where applicable. Even within a program, it is likely that the more complicated projects will call for agile methods more than others. Overall, it is our opinion that applying iterative and lightweight concepts of agile software development methods to large government transformation programs can prove to be an effective means toward reducing exposure to risk while at the same time increasing the probability of success. However, the agile methodology is not, in any way, a replacement to core program management principles; rather, agile methods should be considered as complimentary to these core principles.
In order to fully realize the benefits of the agile methods, there are fundamental legal, regulatory, and cultural changes within an organization which need to occur. Until these changes are realized, large complex programs should consider a “just enough” (Hotle, 2008) approach to applying agile methods that work within the context of the contract deliverables and conformance requirements. To reiterate, this paper has not addressed the changes to federal acquisition law, regulation, and contracting policy that must be made to permit these methods.
The cultural changes begin with the education of both the project team and key client stakeholders as to how the agile methods should be implemented on the program, as well as the importance of a high level of collaboration. Eventually, the stakeholders will begin to see the added value from the agile methods in the resulting work products. Small successes in using agile methods can lead to traction within a program and may, ultimately, lead to an expansion of agile as a paradigm to promote continuous thoughtful evaluation within a program. As a result, this may lead to an improvement in both business processes, as well the associated software development.
Large government programs that employ the discipline to leverage the agile paradigm can benefit by being able to adapt to an inherently uncertain and complex environment. The extent to which a complex government program can successfully implement agile methods in the dynamic federal environment will enable all the stakeholders to mitigate exposure to risk. This will ultimately lead to enhanced final work products and thereby a greater likelihood of customer satisfaction.
Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler M. et al. (2001). Manifesto for Agile Software Development Retrieved from www.agilemanifesto.org
Boehm, B., & Turner, R. (2003, June), Using risk to balance agile and plan-driven methods, Retrieved from http://faculty.salisbury.edu/~xswang/Research/Papers/SERelated/Agile/r6057.pdf
Boehm, B., & Turner, R. (2003, November), Observations on balancing discipline and agility, Retrieved from http://agile2003.agilealliance.org/files/P4Paper.pdf
Carnegie Mellon University CMU/SEI-2010-TN-002, (2010, April), Considerations for Using Agile in DoD Acquisition [Electronic Version]. Retrieved from http://www.sei.cmu.edu
Carnegie Mellon University CMU/SEI-2008-TN-003, (2008, October). CMMI® or agile: Why not embrace both! [Electronic Version]. Retrieved from http://www.sei.cmu.edu
Dolfi, J., (2011, January). PMI Presentation to CSC PMPG, PMI Pulse of the Profession 2010 Study Results. Retrieved from http://www.pmi.org/~/media/PDF/Home/Pulse%20of%20the%20Profession%20White%20Paper%20-%20FINAL.ashx
Hotle, M. (February 2008). Waterfalls, products and projects: A primer to software development methods, [Electronic Version] Retrieved from http://www.gartner.com
Hunsberger, K. (2011, February). Change is good. PM Network, 25(2), 50–53.
Meyer, B, (1997). Object oriented software construction, (2nd Edition), New York: Prentice Hall.
OSD 13744-10 - 804 Report to Congress, (2010, December), A new approach for delivering IT capabilities in the DoD, [Electronic Version]. Retrieved from http://dcmo.defense.gov
Poppendieck, M., & Poppendieck, T. (2003). Lean software development: An agile toolkit, Addison-Wesley Professional
Reinersten, D. G. (2008). The principles of product development flow: Second Generation Lean Product Development (1st ed.), Celeritas Publishing (2009).
Report of the Defense Science Board Task Force on DoD Policies and Procedures for the Acquisition of Information Technology, (2009 March)[Electronic Version], http://www.acq.osd.mil/dsb/reports/ADA498375.pdf
TechAmerica Foundation’s Commission on GTO-21, (2010, October). Government technology opportunity in the 21st Century, [Electronic Version]. Retrieved from http://www.techamerica.org
Tremlett, J. (2009) CSC LEF Grant Project paper. Agile software development. Retrieved from CSC LEF site – for internal use only.
Vivek. K., U.S. CIO, (2010, December), 25 Point Implementation Plan to Reform Federal IT Management, [Electronic Version]. Retrieved on from http://www.cio.gov
© 2011, Peter Zaleski, Christopher Bostian, Christopher McDyer
To be published as a part of 2011 PMI Global Congress Proceedings - Dallas/Fort Worth