Although Agile software development techniques are considered to have ‘potential’ and Agile philosophies are not fundamentally at odds with the key knowledge areas of A Guide to the Project Management Body of Knowledge (PMBOK® Guide), Agile is typically overlooked in favor of the more established PMBOK® Guide techniques. Reluctance to choose Agile techniques over more conventional project management procedures and tools seems to stem primarily from a lack of understanding of when and how PMBOK® Guide and Agile techniques align and diverge. With reference to an experience of an Agile implementation on a large-scale public sector project, this paper identifies commonalities and differences between the two approaches in six key knowledge areas of project management: Integration Management, Scope Management, Time Management, Cost Management, Quality Management and Communications Management. Strategies for adapting Agile to successfully meet project needs are also described.
The relationship between Agile software development techniques and those based on the Project Management Body of Knowledge (PMBOK® Guide) is an area that has been under-researched. The goal of this paper is to strengthen our collective understanding of how Agile techniques can be used to satisfy knowledge areas of the PMBOK® Guide, and it is hoped that this will in turn help organizations to make an informed decision about which approach is best suited to any given project.
In this paper, the merits and drawbacks of Agile- and PMBOK® Guide -based approaches are discussed in the context of a reference project carried out at the government of Canada in 2007-2008. Like many information technology (IT) projects, the reference project faced management and client acceptance challenges. Through a responsive process, the client and integration services vendor adapted Agile techniques to meet project governance and management requirements as well as to deliver a system that met all major requirements and user expectations. The overall success of the reference project demonstrates the viability of Agile as a management technique.
The paper briefly describes the core elements of an Agile technique called Scrum. It then discusses the use of Scrum on the reference project, examining how the technique was used to satisfy some of the PMBOK® Guide requirements that were mandated by the government of Canada’s Treasury Board Secretariat’s (TBS) Enhanced Management Framework (EMF), Treasury Board, (CTB, 2007), and how the client’s concerns were successfully managed using Scrum.
Scrum is a project management framework, not a methodology. It produces the following clear management artifacts:
- Product Backlog: Living document that is added to over the life of the project and product. It is a master list of all functionality desired in the product, along with initial estimates of the relative complexity of features.
- Scrum Meetings: Daily15-minute, time-boxed, stand-up meeting where each member of the team answers three questions: 1) What did you do yesterday? 2) What will you do today? and 3) Are there any impediments in your way?
- Iteration Burndown Chart: List of tasks that defines a team’s work for an iteration, or sprint. Each task identifies who is responsible for doing the work and the estimated amount of work remaining on the task on any given day during the iteration. Burndown charts show the trend of work remaining across time in an iteration.
Other characteristics of Scrum are that it strives for transparency in status reporting and it uses a fixed iteration for releasing software (typically 2-4 weeks). This exposes underlying problems and limitations while also providing numerous opportunities for the client to review the capabilities of the system as development progresses. As with other techniques, customer involvement is important to the success of the project. In addition, teams must be self-managed and the scope of the project is subject to negotiation at regular intervals. Some of the main strengths and challenges associated with Scrum are summarized in Exhibit 1.
Reference Project Overview
The project presented here was part of a larger program for reviewing and updating the web presence of a large Canadian government department in 2007-2008. The program mandate was to analyze the existing website, interview stakeholders and develop governance structures, guidelines and content development strategies. The program also developed requirements for the new web content management system (WCMS), established and executed the implementation plan for the WCMS. The project discussed here covers the implementation project.
When the implementation planning began, there was some uncertainty as to how best to proceed. The government of Canada had a preferred WCMS – a commercial off-the-shelf (COTS) product that all WCMS projects were mandated to use – but the client had little knowledge of how to configure and deploy it. The vendor had extensive expertise in installing, configuring and using the product, but the optimal project management strategy had not been firmly established. All government IT projects had to satisfy the requirements of the TBS EMF, but the framework had few practical examples that could be used for guidance. The vendor proposed that a Scrum approach be used to manage the project, feeling that Scrum offered a good fit with the scale of the project (roughly $2 million) and the high engagement level of the client. The iterative mechanisms within Scrum also allowed the vendor to demonstrate the capabilities of the COTS WCMS product incrementally, thus satisfying a key requirement that the client be capable of eventually supporting and administering the system once the vendor had finished the project.
PMBOK® Guide Knowledge Areas and Scrum Approach
The six knowledge areas that were investigated as part of this paper are outlined below, along with details of how the Scrum approach explicitly or implicitly addressed them.
Project Integration Management
Project integration management speaks to the premise, planning and co-ordination of project activities. The key elements of this knowledge area are the Project Charter and preliminary scope. In other words, a PMBOK® Guide approach would define project scope, schedule and cost before any development begins.
Scrum does not have much to say about planning and co-ordination. According to Schwaber (2004, p. 68), “Scrum projects require less planning than typical Gantt chart based projects because those working to deliver the expected benefits provide visibility into their progress at the end of every sprint”. Scrum proponents feel that by only providing a vision of the project and a prioritized feature list, project sponsors will be satisfied as long as they are actively involved. According to Sliger (2006), project plan definition is a co-operative exercise between the client and vendor, resulting in “several different envisioning and planning exercises done on an iterative basis”.
In the reference project, project integration management procedures were not explicitly defined before an Agile approach was suggested. The project was initially planned out according to a limited program schedule (planning horizon of about 6 weeks). This schedule resulted in a series of small contracts being issued to establish requirements and initial scope; however, no formal Project Charter was approved before significant IT development resources were dedicated to the solution. In fact, the Charter was not approved before the end of active development of the technical solution. The main reason for this Project Charter delay was that the client did not see the cost/benefit (in terms of time and effort) of establishing a formal Project Charter that met EMF guidelines before the analysis and implementation began. This did not prevent the funding of the initial phase, but it still had an impact on the project. For instance, without a Charter, roles and responsibilities were not formally defined until much later into development. The approval of the budget of the project was much more tenuous and managed month to month. While the financial aspects eventually fell into place, there was a concern after requirements definition, before the start of development, that the project might not get funded. However, as mentioned above, the material gathered and presented to project sponsors after initial requirements were developed was sufficient for approval. While the absence of a well-written Project Charter did lead to unnecessary downstream difficulties, it did not initially affect the launch of the execution process.
Against this backdrop, Scrum actually brought process and discipline to scope and planning areas. Using the requirements developed from the initial analysis phase, the development team conducted a requirements review with the client where each requirement was assessed for business value and high-level acceptance criteria. This review allowed the development team to do two things. First, the team met to assess and estimate the required development effort for each feature. At completion they had some confidence that they were estimating the required effort to complete the feature to meet acceptance criteria. Second, the team could build an ordered list of features that would represent the priority given to individual features and supporting activities.
The product backlog (Exhibit 2) formed the basis for estimating the total cost of implementation and set the stage for the development of the project work breakdown structure (WBS).
Another key difference from PMBOK® Guide, as recommended by Griffiths (2004), was that the WBS was based on features to be developed rather than tasks. It established the ‘pattern’ that each iteration would follow, including the following tasks: 1) iteration planning meeting; 2) acceptance criteria development and sign-off; 3) feature development; 4) user acceptance test; 5) documentation; 6) iteration review meeting; and 7) Scrum process review meeting.
These seven elements were repeated over five iterations. The approximate time box for each iteration was 20 business days. The development team and the client used the initial estimates to propose an assignment of features to iterations that kept each iteration to approximately the same size of effort while preserving the priority by business value.
The product backlog also allowed for work assignments and responsibilities to be indicated clearly, which was essential given the mix of client and vendor development personnel.
All told, Scrum brought sufficient rigor to meet the client’s expectations for this area. From a project integration management perspective, the reference project confirms an observation by Griffiths (2004) that a PMBOK®-based project approach can co-exist with the Scrum approach. In addition, it shows that an informal Project Charter may be sufficient to obtain project sponsor approval. This is independent of whether or not Scrum is used as the eventual development methodology.
Project Scope Management
Project scope management, under the knowledge areas of the PMBOK® Guide, is at the greatest odds with Scrum. In theory, the PMBOK® Guide appears to be closely aligned with Scrum’s philosophy around scope management, citing Turner (Turner, 1992, p. 94; PMI, 2004, p. 103), who said “Project Scope Management includes the processes required to ensure that the project includes all the work required, and only the work required, to complete the project successfully.” However, the PMBOK® Guide adds further restrictive clarification: “Project scope management is primarily concerned with defining and controlling what is and is not included in the project” (PMI, 2004, p 103). The “Tools” section of the Project Scope Management chapter leaves little doubt that scope is to be managed and controlled with centralized tools and decision making processes (PMI 2004, p 121), but as noted by Griffiths (2004), the tools that have been implemented to support scope management (e.g., WBS, change request management systems) are cumbersome to update and quickly become out of date.
The reference project used Scrum’s core toolset for managing scope changes. It used the product backlog and high-level acceptance criteria to prioritize and schedule feature development. At the start of each iteration, the client and vendor co-developed detailed acceptance criteria for each feature to be developed. After each iteration, the client verified that the features met the acceptance criteria. The client also had the opportunity to reprioritize the next set of features from the product backlog and could modify the requirements of developed or undeveloped features on the understanding that any change in effort would result in additional costs. Details of how these cost adjustments were managed are in the ‘Project Cost Management’ section.
The reference project did reveal a significant short-coming in the Scrum approach that is not present in PMBOK® Guide -based projects: the absence of an initial overall architectural document. This made the client uncomfortable about the scope tradeoffs that were being made during the implementation. To address this shortcoming, the vendor produced a scope discussion document to describe the options for the key features of the system. It provided a ‘gold/silver/bronze’ option that outlined the costs and benefits of multiple design approaches for a feature.
The reference project differed from a more traditional project approach in the following ways: First, in place of strict change control, there were three separate opportunities to adjust scope at every iteration. Second, instead of keeping the scope document up to date, the project did not adjust the initial WBS, principally because it was purposely written to guide the overall direction of the implementation. Third, in place of documented system architecture up front, it addressed this client concern by writing an option document to justify the direction and scope tradeoffs that were made during the course of implementation.
In general, Project Scope Management cannot be performed using the PMBOK® Guide approach in a Scrum implementation. After the initial burst of activity around setting up the project WBS and associated documentation, Scrum processes will ensure that the initial plan will become out of date, unless a low value activity is performed frequently to update the WBS. Or, the WBS can be written as a guide, instead of as a detailed list of tasks and schedule. According to Sliger (2006, Part 2 ¶3), “The PMBOK® Guide practices of scope definition, work breakdown structure (WBS) creation, and scope verification occur iteratively in Agile”. This iterative approach ensures that a detailed task list and scope originally defined at the planning stages will be significantly different by the project’s end.
Project Time Management
With regard to time management, Scrum- and PMBOK® Guide -based projects share similarities in activity definition and resource estimation/duration. However, for the remaining sub areas, the approaches differ in that Scrum mandates that the project team should collaboratively determine appropriate sequencing, scheduling and schedule control. Note that the project team includes development and client staff who share responsibility for the project’s timely completion.
In the reference project, the client determined the product backlog sequencing and the list of features to be included in the final solution. The development team determined the estimates of the final solution and, crucially, they then implemented the project. This allowed them to quickly assess the impact of scope changes on effort estimates, since they knew the assumptions that had been made. This estimation exercise occurred over two days at the start of the project where the initial requirements and acceptance criteria were discussed. The estimates were then reviewed at the start of every iteration to verify their accuracy, based on the results from previous iterations.
The team used a Scrum artifact called a burndown chart (Exhibit 3) to ensure that each iteration completed the committed functionality in the required 20-day window. Daily updates reflected each team member’s assessment of how many hours remained on assigned tasks. Task ownership was self-assigned according to interest/specialty. The initial time assigned to a task was based on the initial estimate plus any clarifications that changed the initial scope of the feature. Day by day, the team could see how the actual time remaining contrasted against the baseline. Each iteration followed an ‘S’ curve of accomplishment, and at times team members felt they did not make enough progress on activity completion at the beginning. Reasons for the slow start were addressed in subsequent iterations. For example, it was identified that the iteration estimate did not take into account the number of review meetings occurring at the start of every iteration. As a result, tasks were explicitly added or modified to ensure the chart captured all project-related activities.
Lastly, scheduling and schedule control were handled using PMBOK® Guide methods by comparing the product backlog to the baseline at each 20-day iteration. By committing to only 20 days worth of features at a time, the project team and sponsors could easily chart performance. When one iteration went over the plan, it was immediately clear that the project team would have to address how to make up the time. Along with the burndown, the entire project team could easily track the daily, monthly and schedule variances.
Scrum- and PMBOK® Guide -based approaches are similar in their activity definition and estimation processes. However, the distributed schedule sequencing, control and task assignment activities of Scrum are not compatible with the centralized approach implied by the PMBOK® Guide.
Project Cost Management
This represents a key interface between a project and existing financial controller functions in an organization. It is difficult for Scrum to actively alter these processes because they are often in place for policy or regulatory reasons. PMBOK® Guide would seem to have the upper hand in this knowledge area because the artifacts and processes generated through Project Integration, Scope and Time Management are all inputs into project costing and budgeting. Moreover, many IT projects are ‘fixed price’, but according to Poppendieck & Poppendieck (2003, p 171) such contracts encourage self-serving behavior by the customer and defensive behavior by the vendor. These constraints are challenging for a Scrum project to overcome without losing the essence of trust and client/vendor co-operation that is the cornerstone of Scrum’s development philosophy.
The reference project was subject to fixed-price constraints and significant project controller overhead to ensure that task authorizations (TAs) against the standing agreement met budgetary and control requirements. Initially, it seemed that Scrum could not be used because of this restriction. However, the vendor proposed a one-to-one mapping between each iteration and TA. That is, each TA covered only the features to be developed for one 20-day period. Because the product backlog included initial effort estimates, the client could see how the contracts traced back to effort, plus some overhead. At the start of each iteration, a new TA representing a revised effort estimate and new feature list would be signed. At the end of the iteration, the client would determine if the vendor had delivered on the feature list with the requisite quality and the vendor could determine if development costs incurred in the project met profitability requirements. At the end of each TA, the vendor or client could adjust the terms and scope of the subsequent TA to reflect new understanding of technical and business requirements. As the project continued, the client could see the business value in the emerging system, while the vendor understood more about the client’s processes and technical environment. Ultimately, this enabled the vendor and client to behave as part of one team, committed to the overall success of the project.
In this area, the PMBOK® Guide and external financial control requirements take precedence, but the reference project showed how the existing cost structure implied by the contract could be used to support Scrum. At any time, if the client or vendor felt they were not getting enough value for their investment, they had the option to either switch to a more traditional method or walk away from the other party. Fortunately, such extreme options were not exercised because both parties agreed that the project brought acceptable and measurable business value at a reasonable price.
Project Quality Management
In this area, Scrum and PMBOK® Guide are aligned. Both see the importance of quality planning and assurance in meeting client expectations and overall success, and both know that negative quality outcomes occur if the implementation team is forced to meet requirements through overtime or if the final quality inspection process is rushed due to schedule overruns (PMI, 2004, 180).
The reference project used the following quality measures to insure the project met client business needs: 1) detailed acceptance criteria were developed after every iteration kick-off meeting to create clear development targets; 2) the client was responsible for system testing after every iteration to confirm it met the acceptance criteria; 3) the iterative development process and the prioritized product backlog meant that the most important features were developed early, and therefore subject to repeated (if indirect) testing as new dependent features were added to the system; and 4) the first features developed were for release maintenance to ensure that customized code could be reliably transferred between development, test and production environments. These elements helped insure that system quality was repeatedly evaluated.
PMBOK® Guide and Scrum both emphasize quality goals as important elements of a successful project. The tools and techniques may vary across projects, but the steps and procedures outlined in the PMBOK® Guide Project Quality Management knowledge area are applicable to Scrum projects.
Project Communications Management
Here, Scrum and PMBOK® Guide again diverge. PMBOK® Guide (2004, 235) sets out various communication types/purposes and discusses stakeholder management planning with respect to communications. Scrum does not have this level of formalism, but communication is still critical. Scrum has regularly scheduled meetings within an iteration. For example, without daily scrum meetings, development team members might lose touch with each other’s progress. Without iteration kick off meetings, client expectations could not be fed into scope and acceptance criteria changes. Without iteration review meetings, clients and stakeholders could not review project progress. Finally, without regular status meetings with the project executive, project-specific issues could not be raised and reviewed. Where Scrum differs from PMBOK® Guide in this area is that the type and layout of communication is not fixed. It may be enough to capture the essence of discussions in a whiteboard session, or it may be necessary to draft a formal proposal to express choices to be made on a technical issue. In a given situation, the best communication tool and approach varies.
In the reference project, the most regularly attended and successful meetings were weekly status meetings with the project executive. Throughout the project, client and vendor management and architects met regularly to discuss progress, identify risks, log decisions and set action items. The meetings were informal (but documented) and served to keep track of project progress. Less successful were the daily Scrum meetings, which were in fact only held sporadically. This was surprising, given the importance of the meeting in Scrum methodology. The main explanation for this half-hearted approach to Scrum meetings was the fact that the whole team was located in a single room and was highly communicative. It was quickly apparent when issues arose and members took collective ownership to solve difficult problems. Also, team progress was visible through regular updates to the iteration burndown chart. Finally, a modestly successful meeting type was the iteration review meeting. These meetings with the client and end-user enabled them to review and understand the system as it was developed. They offered clear and meaningful direction to development and helped to make the users comfortable with the final system.
The reference project showed that this PMBOK® Guide knowledge area was at best an optional part for a project of its size. Scrum allowed communication forms and feedback opportunities for issues to be resolved without having to be escalated up to the project executive status meeting.
The reference project presented here demonstrates that several key PMBOK® Guide knowledge areas are wholly or partially supported by the Scrum framework, including Integration Management, Time Management, Cost Management and Quality Management. However, there is little support offered by Scrum to Scope Management and Communications Management as outlined in the PMBOK® Guide. When evaluating the appropriateness of Scrum for a project, an organization should determine if the lack of support for Scope and Communications management impedes the project’s overall success. For reasons outlined above, the reference project did not significantly suffer as a result of these limitations. In fact, Scrum brought an appropriate amount of structure and process to the project, and was ultimately a key reason for the project’s success.