Application of hybrid agile project management methods to a mission-critical law enforcement agency program
Software development methods have evolved and improved over the years. There are a number of distinct approaches and methods available to base application software development on. More recently, agile methods have been gaining momentum, obtaining critical mass and endorsement from well-established information technology (IT) organizations among others. A brief history of the evolution of the methods is presented in this paper. The advantages and disadvantages of the various approaches are briefly examined, and the agile concepts are further explored. This paper also addresses some of the current challenges with pure agile methods that were encountered in the case study and exhibits a tested and proven discipline based on some of the traditional methods working in concert with a practical agile implementation. The tradeoff between the traditional methods and those unique to agile techniques is the premise of the case study discussed in this paper.
Software development techniques and methods have been around for decades. From those deployed in the “real time” systems of the 1960s, such as those enabling the command instructions wired into the Apollo’s guidance computer, to the 1970s and the era of software engineering: structured analysis and design, top-down programming, and waterfall approaches. The 1980s brought along the fourth generation languages, and the emergence of the personal computer! The 1990s brought us the rigorous processes: the do-it-right-the-first-time mindset as epitomized by six-sigma, ISO-9000 compliant processes, and the CMM levels of process maturity. The 21st century, so far, has been about the rapid response to consumer needs and quick-to-market solutions—technologies used, for example, in developing social networks, blogs, and “smart” handheld electronic devices. Software development techniques are maturing and going through another life cycle evolution. Agile development methods have expanded rapidly this decade, and different flavors of agile methods have emerged; yet, all the agile-based methods share a set of common fundamental principles:
- Individuals and interactions over processes and tools,
- Working software over comprehensive documentation,
- Customer collaboration over contract negotiation, and
- Responding to change over following any pre-set plan.
While agile methods in their “purest form” have worked well for a number of organizations, it can be argued that it has been challenging to implement them in environments with disparate teams developing large software applications. Contractual considerations, for projects being developed for external organizations, introduce added complexities for pure agile based approaches.
This paper presents an approach to developing a large complex software application that is based on a hybrid method, or a “practical” agile approach. This phased release approach is based on Microsoft’s® Solution Framework (MSF) and its “scrum” instantiation. The case study being presented here is based on managing an application development project using this hybrid-agile approach.
In the following sections, traditional “waterfall” project planning and controlling, as well as agile project planning and execution, are discussed. This includes reviewing the pros and cons of each approach, leading to the decision on devising a practical approach which combined aspects from the two distinct approaches to deal with the technical and environmental challenges of the project that is the subject of the case study.
Defined vs. Empirical Approaches
The “Defined” Approach
The Waterfall model, also referred to as a “traditional” project management approach in this paper, is based on a “defined” approach. It is a sequential software development model in which development is flowing steadily downwards through the phases of requirements analysis, design, implementation, testing, and maintenance. This model was further developed and enhanced from its original applications, which included large government projects, into iterative models that include approaches such as the Microsoft Solution Framework (MSF) and Iterative Application Development (IAD), with feedback from each phase influencing subsequent phases.
Various (modified) Waterfall models are widely used by large software development organizations, including NASA and the U.S. Department of Defense. Analytical methods supporting these models include the PERT planning process that relies on the following:
- Identify the specific activities and milestones,
- Determine the proper sequence of the activities,
- Construct a network diagram,
- Estimate the time required for each activity,
- Determine the critical path, and
- Update the PERT chart as the project progresses.
Exhibit 1: PERT chart
These defined approaches can provide a degree of predictability and enable contract constructions based on analytically measured metrics and milestones that can be monitored and reported against. However, for such a systems development process description to be “defined” adequately it would have to fit into in the following categories:
- A detailed, complete set of requirements as specified from a user’s perspective must exist.
- A detailed, complete description of all inputs, outputs, and transformation processes to fulfill the requirements must exist.
- A detailed description of the skills and capabilities of the resources must exist.
- A detailed account of tasks and schedules for completing the final product must exist.
While there are other conditions that would further enhance the application of the defined approaches, the four conditions noted previously are critical to the successful application of the model. However, empirical approaches provide an alternative solution to the challenges inherent in application development projects where some of the fundamental technical/business/and environmental requirements may not exist.
The “Empirical” Approach
Empirical process control models are elegantly simple. They employ feedback mechanisms to monitor and adapt to the unexpected, providing regularity and predictability. The diagram below depicts a typical scrum approach:
Exhibit 2: Emprical process control model
I is the input (requirements, technology, team) for building a product increment
Process is typically a 30-calendar day increment, called a “sprint”
C is the control unit that monitors the scrum progress at daily scrum meetings
O is the product increment built during each iteration
In most practical situations, “change” is an integral part of the system, application, and the adapted approach for the solution being worked on. Iterations accommodate and provoke early changes. Brilliant ideas emerge over time, they need the time and space to develop, and most often they come from interactions of engaged people. Agile models are based on the end user and the product owner being an integral part of the project team.
Agile methods also allow for early risk discovery and mitigation, improved progress tracking and predictability, and more manageable complexity and higher quality code with fewer defects through early and regular process improvement. As a result, customer and developer confidence tends to be higher, and the products and solutions more closely match and align with customers’ desires and expectations, although it should be noted that business user training will often be required initially before any gains in confidence can be realized. Also, predictability is compatible with learning; and a system and approach that allows for short iterative development cycles, based on retrospective and learning, can lead to higher productivity, lower overall investments and increased profitability.
It should, however, be noted that agile is a much disciplined process, and it works best when the project team is comprised of highly skilled and disciplined (technical) individuals. Other challenges in having a successful experience with agile approaches include a paradigm shift (project management and project resources). Also, measurements and indicators are not easily understood by customers accustomed to traditional methodologies.
As previously stated, the contracting aspects add another degree of complexity when this approach is adopted for projects and application solution delivery provided by outside vendors. While the “product” should still be defined at the project inception, the various “deliverables” and the “milestones” may not (and probably should not) be defined at the outset, which is the antithesis of “sound” contracting practices. Also, some of the fundamental traditional project management principles would differ and conflict with the tenets inherent in agile projects.
To sight an example, consider the “change management” aspects of traditional projects and compare to that of one based on an agile method. While stringent change control management is required in the former approach, from contract preparation through to the actual delivery, the latter approach embraces and encourages changes that would lead to satisfying the clients’ real needs and requirements. Hence, the contract should be structured in such a way that while it protects the clients’ interests in setting boundaries and encapsulating a product definition, with specified schedule, budget, and solution “scope,” it is also flexible to allow for changes to occur that would enable the project team to deliver a product that best fits the business and technical requirements of the client as they evolve during the applications development process.
While the sprints are planned in such a way to conclude with a working portion of the system or product being developed (i.e. a tested demonstrable cross section of the final product), to further support the project’s burn rate and the accomplished results, clients’ (especially those new to agile methods, and those going through the journey to transition to the agile world) often require more insight into the status and measurement indices. The more recent trends include the addition of objective measurement metrics during project execution. These include (simplified) earned value management for application development efforts, and for status monitoring, controlling and management. While these techniques do not follow the strict agile methods they still provide an accurate means of assessing and quantifying project performance and progress.
These techniques also provide greater realism and accuracy in project metrics, allowing more accurate forecasts, based on trending assumptions around project burn-down charts and velocity. It should be noted that these techniques are most reliable during individual sprints, and that the extrapolation across the entire project development life cycle does not always bear accurate results in agile methods.
Applying a Practical “Hybrid” Approach
This section describes the project’s setup, including the team model and technical and environment challenges, before describing the approaches adopted in addressing the issues.
The following is the top 10 list, describing the circumstances, issues, and challenges:
- Disparate team model: team of 22 with 11 resources working offsite,
- Fixed-fee contract,
- Client’s lack of experience and understanding with agile methods,
- Team comprised of highly technical and competent resources, as well as some novice and less experienced individuals,
- Lack of technical ownership and “go to” person for decision making,
- Client was not very engaging and remained hands-off throughout the first few months,
- Lack of comprehensive project requirements,
- Unclear roles and responsibilities,
- Lack of end-user involvement, and
- Lack of clear customer expectations and success criteria.
While some of the challenges and concerns seemed to suggest agile methods as a viable solution, others didn’t quite align with what agile processes and environmental requirements would point to for a successful outcome. The following section describes the mapping of the above issues with the approach adopted on the project.
The overall approach was based on scrum blended with elements of formal project management techniques based on the MSF model. In this section the specific approach taken to address the top 10 list are described.
- Disparate team mode: team of 22 with 11 resources working offsite
Scrum works best when the entire team is collocated. To mitigate this, daily scrum calls were established and they included the entire project team. While these were not strictly the prescribed scrum standup meetings, they successfully facilitated communications around daily progress and impediments and created a team environment and sense of ownership.
- Fixed-Fee contract
In order to mitigate the risk of having an ill-defined scope and a lack of formally documented requirements, an agile-like development approach that consisted of time-boxed development releases was chosen. In this time-boxed approach, no traditional scope was defined and no contractual commitments were made with regard to what would be delivered from the backlog; only that functionality would be selected from the backlog which would then be developed and delivered within the development release defined by the timebox. The scope of each development release was determined during an MSF-like planning phase where the product backlog was reviewed and prioritized, and elements selected from the backlog that could be accomplished within the time-box by the resources available.
- Client’s lack of experience and understanding with agile methods
In this case, the customer did not have prior experience with true agile methods. In fact, this customer, being a law enforcement agency in a large metropolitan area, had little to no expertise in developing IT solutions, nor did they have the time or desire to fully participate in any sprint-related development activity as would normally be expected. They did, however, have a good concept of what they wanted see in the solution despite a lack of any documented requirements. By including them in the pre-scrum planning phase for each sprint, they were able to provide items for the development backlog and they were able to prioritize the items in the backlog for inclusion in the next release (development scrum) while also limiting the time required of them.
- Team comprised of highly technical and competent resources, as well as some novice and less experienced individuals
The inclusion of experienced resources has presented a good training opportunity for the less experienced ones on the project. The scrum teams have been structured in a way to support a level of pair-programming, enabling ongoing support and mentoring throughout the sprints. The sprints are planned with this approach and its impact on burn-rate and velocity in mind. The resources’ areas of focus would also rotate, whereby they would assume responsibilities over the various areas of the application, thereby gaining product awareness and learning the application end-to-end (vs. specializing only in certain specific parts of the overall solution). This approach has improved the teams’ productivity over the development life cycle, and has increased the team’s overall competencies in technologies adapted on the project.
- Lack of technical ownership and “go to ” person for decision making
As noted previously, the customer for this project is a large, metropolitan law enforcement agency focused solely on law enforcement and the prevention of terrorist and other violent acts. As such, the customer’s chain of command had little time or interest in decision making for an IT development project, even one designed to provide the enforcement and prevention tools they said they desperately needed. By implementing the product backlog to capture items needing decisions, and holding short “planning phase” sessions in advance of each sprint aimed at prioritizing the backlog, we were able to successfully “delay” the need for most decisions until the pre-scrum planning phase, and also get the necessary decisions made during the planning phases as part of the prioritization and selection process.
- Client was not very engaged and remained hands-off throughout the first few months
At the outset of this project, it was recognized that customer involvement would be minimal and difficult to obtain. By breaking the development into smaller releases (sprints), it was hoped that the impact of the lack of customer involvement could the minimized by having them focus of the planning phase before the start of each sprint. This proved to be a very effective technique to minimize the customer’s involvement while still getting them to prioritize their needs and desires from the backlog and allowing them to select the functions that would be developed within the constraints of each time-boxed sprint. At the end of each sprint, the customer was then called back to validate and accept the results of each release. It was during this short, post-sprint “deployment” phase for each release that they got to see how their application would work. This generated much enthusiasm for the project and “primed the pump” for the generation of new ideas (i.e., requirements) to add to the backlog for future releases, and also made the customer much more enthusiastic and willing to participate in the planning phases for subsequent sprints.
- Lack of comprehensive project requirements
At the start of this project, there was no defined set of specific requirements for this application. Furthermore, the customer being a large, metropolitan law enforcement agency focused solely on law enforcement and the prevention of terrorist and other violent acts and unfamiliar with IT development project methodologies, they believed that had neither the time nor the inclination to spend time writing a set of detailed requirements. They did, however, have a good concept of what they wanted see in the solution despite a lack of any documented requirements. By including them in the scrum planning for each sprint, they were able to provide items for the development backlog and they were able to prioritize the items in the backlog for inclusion in the next release (development scrum).
- Unclear roles and responsibilities
As with any IT projects involving extended teams of developers, some working remotely and some being onsite, further challenged by unclear requirements and project deliverables, identifying clear roles and responsibilities at the project kickoff was not feasible. Over the initial few weeks of the project, given the constraints, the project organization and the resources, the team identified and fine-tuned the optimal structure for the project team. This included role clarity, and guidelines and expectations during the project and sprint lifetime (keeping the integrity of the sprint backlog intact while providing flexibility in assigning individual tasks). This enabled the project’s Communications Management plan document to be completed at the close of the first sprint.
- Lack of end-user involvement
End-user involvement is a critical success factor for any IT development project, especially with regard to defining requirements prior to development and user acceptance testing after development. However, because the eventual users of the system being developed are full-time law enforcement officers focused on their law enforcement jobs, getting their participation was very difficult. As noted above, this was successfully mitigated by implementing a short MSF-like planning phase prior to the start of each sprint to prioritize the backlog (requirements/ envisioning) and select the components to be developed during the sprint (planning). Also, at the end of each sprint, the customer was called back to validate and accept the results of each release (user acceptance testing/ stabilizing). This generated much enthusiasm for the project and made the customer much more enthusiastic and willing to participate in the planning phases for subsequent sprints.
- Lack of clear customer expectations and success criteria
Because there were no clearly defined requirements at the start of the project, there were also no clearly defined success criteria or expectations beyond the development of a system to support the agency. While this is well and good as far as it goes, it is not a sufficient measure when applied to a project team, especially one contracted to perform the development. To address this, specific expectation and success criteria were determined for each sprint as part of the function prioritization and selection process performed during the planning phase for each sprint. At the end of the project, overall success was then determined by the degree to which the success criteria were met for each sprint. This enabled expectations and success criteria to be “customized” and mutually agreed between the customer and the project development team for each sprint.
Phase 1 of this project concluded very successfully with the implementation of the first three releases. Customer satisfaction with the development team, the development approach, and the application are very high. Enthusiasm from the customer for the application continues to grow and a follow-on contract to continue development using this method, with some changes to the approach to alternate “bigger, longer” releases with smaller, shorter releases (see Lessons Learned) resulted. Phase 2 recently kicked off and is off to a strong start.
The hybrid agile approach used on this project was very successful. Nonetheless, some key lessons were learned and will result in some changes to this approach for the next phase of the project, and future customer projects.
1. Customer’s awareness and buy-in in the approach selected for project delivery is critical in these scenarios, especially if the organization is new to agile concepts and methods. Additional upfront discussions and enhanced customer training and expectation setting would have resolved some of the issues during the initial few weeks of the project when the team is still self-organizing and planning the initial sprints
2. The project team must also comprehend the differences between the (hybrid) agile approach and the traditional methods, this includes some of the key concepts:
• understanding the concepts of the team assigning work to itself (vs. the project manager assigning work to individuals),
• the differences in “leading and coaching” to “project managing,” and
• the required flexibility in assuming different roles spanning the various technical areas of the solution.
As with the customer training and discussions mentioned previously, there could have been more preparation and emphasis prior to project kick off on these key areas to raise awareness.
3. Customer involvement is critical to success. The development team’s access to key customer decision makers needed to be improved. As a result, monthly executive sponsor meetings will be implemented for phase 2.
4. The customer end-users needed more time to assimilate the new functionality delivered with each release. To accommodate this, as well as to allow some “tweaks” and enhancements to what was delivered in earlier releases, the sprint cycle for phase 2 will be altered to allow for alternating mini-releases and major releases. The major releases will focus on development of new functionality as selected from the prioritized backlog, and will be developed within a larger time-boxed sprint of 6–8 weeks. The mini-releases will focus on changes and enhancements as selected from the backlog to previously developed functionality, and will be included in a smaller time-boxed sprint of 2–4 weeks.
The following is a list of the main take aways from the case study:
- MSF and agile can be merged into a hybrid approach,
- Set up the environment before starting (customer and team alignment with the proposed approach),
- Pre-sprint planning phase is critical to success,
- The Product Backlog is the repository for requirements and changes,
- Customer / product owner must drive and own the content of the product backlog, and
- Manage changes through the product backlog.
The agile concepts were merged with more traditional methods leading to a successful project delivery. The lessons learned will enhance the process to be implemented in phase 2.
Nikravan, B. (2004). Application of time-boxed iterative methods to a large enterprise insurance solution development. Microsoft Consulting Services internal whitepaper.
Poppendieck, M. (2008). The software development pendulum—why agile? Agile Conference, Vancouver, BC.
Rothman, J. (2005). Hiring for team fit, Cutter IT Journal, 18(7).
Schwaber, K. (2003). Agile project management with scrum, MSPress.
© 2008, Bijan Nikravan & Douglas Melanson
Originally published as a part of 2008 PMI Global Congress Proceedings – Denver, Colorado, USA