Applying project methodology in agile development

Abstract

Agile methods and Scrum provide a huge possibility to improve productivity, quality, and customer value. An increasing number of customers prefer vendors working with Agile methods, as they involve the product owner throughout the development, provide feedback on achieved results, and help prioritize features for continued development.

Most product-developing companies working in an Agile way have a solid tradition of running their product development in projects.

This paper is about Scrum and how you integrate your projects, portfolio, and organization with it.

The first part of the paper discusses Scrum, what it is, and the huge positive impact it has had on productivity and quality. The second part discusses the challenges regarding Scrum with projects and portfolios in a traditional organization and how you can mitigate these challenges. The third part presents a practical example of how Scrum is fit into traditional project management methodology. The fourth and last part provides guidance on the implementation in an organization.

Scrum Boosts Productivity

Motivation

People using Scrum have experienced huge productivity gains. Many say that you can hardly avoid doubling productivity with Scrum; simply focus on implementing only high business value (which is exactly what Scrum is about). Other statistics say that over 60% of the requirements change during the development phase, which calls for an iterative approach like Scrum.

Scrum improves quality with continuous testing. Scrum improves usability and customer value with feedback on every iteration.

Scrum at a Glance
Scrum is an iterative, incremental process commonly used in conjunction with Agile software development. It is a lightweight framework with relatively few roles and artifacts.

Scrum is ideal for projects with changing requirements. You deliver the product feature by feature, not the whole product at once, thus Scrum embraces change.

Exhibit 1: The Scrum process

(Source: Mountain Goat Software, http://www.mountaingoatsoftware.com/Scrum)

Work is structured in iterations, called sprints. Each sprint typically lasts two to four weeks. An important thing about the sprint is that no outside influence can interfere with the team until the end of the sprint. The team can maintain a flow with a delimited scope.

The product backlog is a compilation of all the features required, including functionality, technical architecture, known bugs, and so forth. The product owner is responsible for keeping the product backlog up to date and prioritized.

The sprint backlog is more detailed, containing information about how the team is going to implement the features for the upcoming sprint. Features are broken down into tasks. As a best practice, tasks are normally estimated at 4 to 16 hours of work.

The progress of all team members is monitored through a short daily Scrum meeting (15 minutes).

Three questions are answered by every team member:

1. What did you do since the last meeting?
2. What are you doing until the next meeting?
3. What prevented you from doing work?

It is reported by the team, for the team.

The first two questions allow the Scrum Master to monitor the process. The Scrum Master can address problems quickly, since there is daily visibility of the work performed. Just as important is the group dynamic aspect; this process clearly requires commitment to do what you set out to do the day before. It gives everyone insight into what everyone else is doing, which motivates and tightens the group. It is also rewarding, since you get instant appreciation when you tell the team you have accomplished what you set out to do.

I think the third question was designed to give the Scrum Master the information needed to enhance teamwork, that is, to remove impediments. That is fine and very important, but from my experience, just as important is the fact that question 3 provides a natural opportunity to seek help from your fellow team mates. Most often someone else in the team has had a similar problem and knows a solution. Yes, the worker would probably have asked for help anyway, but my experience says that he or she would have waited a day or two.

Having a common work area, often known as war rooms, enables teams to work together and share information. If that is not physically possible the next best thing is an electronic war room accessible over the Intranet.

Exhibit 2: The task board

(Source: Mountain Goat Software, http://www.mountaingoatsoftware.com/task-boards)

A task board shows everyone's work in a sprint, and it is updated continuously throughout the sprint. Each task is represented by a card which is moved from left to right according to progress.

Exhibit 3: Burndown charts

Burn down charts are useful follow-up tools. They visualize progress for the team and others.

The product owner is responsible for what is developed, that is, the business value.

The Scrum master is responsible for the process and for removing any impediments.

The Scrum team is self-organized and cross-functional.

My Journey

By now I am a strong believer in Scrum, but I can easily identify with those who are sceptical when they first hear of it. I know the mental hurdles, because I have been there myself. That is why I want to describe my own journey from Scrum sceptic to Scrum believer.

I was a traditional project manager and saw Scrum as something fluffy, fuzzy and unpredictable. I mean, you develop it piece by piece and the team is self-organized. How much control is there?

My reality was often a struggle with the product owner, the team, and management. The product owner took every opportunity to increase the scope by bold interpretations of our requirements. Developers took their chances to gold-plate just for professional satisfaction. On top of that, I had management who demanded that every deadline be met.

How could I have even a slight chance to control my projects if things were planned along the way, in sprints? How would I ever be able to measure our progress, even report it to management? Would I lose the little control I had when the team became a self-organized Scrum team? In short, I was terrified. We started working, using all this Scrum stuff.

To make a long story short, I experienced a productivity not compared to anything I had seen before, much better quality and higher usability. In addition to this I had a happy team around me that was truly accountable for their work. After 12 months we (I had two Scrum teams working in parallel) celebrated a very successful release.

First of all: Was this a coincidence? I don't think so, because this is reported from so many Scrum teams. Secondly: Why?

There are many reasons and some of the most significant are these: With Scrum you focus on the most important features as you take them from a prioritized list—there is no waste. You get additional focus and flow because the team must not be disturbed during a sprint. With focus and the correct priorities comes results, and management is happy, the Scrum master is happy, and the team is happy. The annoying change management discussions are replaced by a prioritized backlog together with a very much involved product owner. Change is embraced. Much to my surprise, I also got more control than I ever had thanks to the daily meetings and burndown charts, which are excellent for follow-up. My experience with Scrum is that everyone is pulling in the same direction with focus and energy and the reason for that, in my opinion, is that Scrum is the commonsense approach. When you force a less natural process on a development team, a lot of energy gets wasted.

The message I want to give is: Trust me, Scrum works, and in particular, don't be discouraged if it takes a while to fully grasp the benefits. Give Scrum a chance.

Scrum and the Knowledge Areas
A Guide to the Project Management Body of Knowledge (PMBOK® Guide) recognizes nine knowledge areas:

• Project Integration Management
• Project Scope Management
• Project Time Management
• Project Cost Management
• Project Quality Management
• Project Human Resource Management
• Project Communications Management
• Project Risk Management
• Project Procurement Management

The practices have their counterparts in Scrum, but when and how they are executed is quite different. My experience is that many are even better handled in Scrum than in a typical, traditional project. Very much so, since all the knowledge areas become integral parts of the everyday work. Risks are discussed on the daily meeting, quality is controlled by continuous testing, and scope is controlled by continuous evaluation. The process itself is evaluated after each sprint at a sprint retrospective. You may ask yourself how often traditional project evaluation reports are actually written—or read, for that matter.

PROPS V4 (Semcon, 2006) recognized a tenth knowledge area:

• Project Value Management

This important area ensures that the customer business value of the project is fulfilled. Fulfilling value does not necessarily equal fulfilling all the requirements. By having close contact with the product owner a Scrum team stands a considerably higher chance to meet expectations on value.

These are just examples to assure you that Scrum is just as much compliant with the knowledge areas of the PMBOK® Guide as any other methodology. For deeper information I strongly recommend part II of The Software Project Manager's Bridge To Agility (Sliger & Broderick, 2008).

Challenges

Scrum is ideal for software development by a team of five to nine people, but in an organization there are other things to consider. I have identified six challenges. Let us investigate them one-by-one.

Project Selection

An organization typically has many project ideas in the pipeline. It has a project approval process in order to prioritize and decide which to start. That requires an initial opportunity analysis where Scrum and traditional projects can be objectively compared. In an early stage you may not have an entire Scrum team at your disposal. Instead this first activity may be handled in a more traditional way. Note that Scrum is iterative; that is, you don't do all your planning up front. An analysis at this stage is by nature at a high level. If, on the other hand, the team is available, you can use the first sprint for the opportunity analysis, after which the business can make a go/no-go decision.

A Project Is Often More Than the Development of Software

How do I handle the entire project? Other parts are often included, such as changes to the organization, training users, construction of a new assembly line, and marketing and sales activities. All these parts are often not handled by Scrum teams. The challenge is to control and coordinate the entire project, including all dependencies.

As manager of the entire project, you need to define how information is communicated to the team and you need to define what follow-up information you will need from the team. Ideally, you can use the information already at hand in the backlog, task board, and burndown charts, and furthermore, attend some of the Scrum meetings.

You have tollgates in the project management model to consider. One of them may require an approved architecture specification as input. Let the product owner put that in the backlog and you will be notified once it is done.

You have milestones and project dependencies to consider. An example is the technical writers, who may reside outside the Scrum team and who want to be informed when the Scrum team has raw material ready for them. You just add another backlog item, as in the previous example.

Large Software Projects

The software task itself may be too much for one Scrum team. What you need is more Scrum teams working in parallel. In Scrum the mechanism for handling that is called Scrum-of-Scrums (Exhibit 4).

Exhibit 4: Scrum-of-Scrums

This model works across cross-functional Scrum teams. The teams are linked by a Scrum-of-Scrums, where Scrum masters meet regularly in a meeting, very much like a daily Scrum meeting.

A more traditional approach would be an Agile development program that consists of a large number of small development increments, coordinating (release planning, dependency planning) at the program level.

Monitor Your Program or Portfolio

Being responsible for a portfolio or a program means that you have to monitor the health of each project on a higher level. You must handle dependencies and make business decisions on which projects to stop, prioritize, and so forth. To be able to do this, you need comparable reporting of status and metrics from all projects. This goes for traditional as well as Scrum projects. A guideline here is that you should not impose the old reporting style on Scrum teams by default. Instead, you should use this opportunity to consider what reports are of real interest. If the Scrum team already uses an IT tool for Sprint burndown and Release burndown, then important data is already in place.

Auditing for Compliance with Standards

Some formal auditing (e.g., FDA regulations or ISO 9001 standards) will require documentation. Let the product owner put items in the backlog to work these into your sprints. Again this is an excellent opportunity to investigate the real need. Your Scrum team may unnecessarily loose productivity if the need is not mandatory.

Handover of the Final Product

The handover to the receiving organization may include compatibility tests, raw material for the product launch, training of help desk personnel, and different signoffs. This takes place after the sprint, where you have a potentially shippable code. A suggestion is to introduce a final sprint with backlog items for these things according to the same principle as when we added items for tollgate decisions.

An Example

Integrate Scrum with the Traditional Project Methodology

Here is a practical example of how you integrate Scrum with a traditional project methodology, without cramping your Scrum team(s). The reason to keep existing processes is to make the change to Agile an evolution, not a revolution. Also, by keeping the steering function/gates, we can monitor our projects or program on the portfolio level in similar ways (using the same interface as for traditional projects).

Introducing Scrum for developing software in projects, we however need to change the way we steer and manage the project. From a steering perspective we need to change the definition of the gates, from authorizing the project to proceed with the next phase, to something more suitable for Scrum (see Exhibit 5).

The ongoing Scrum team signals readiness for tollgates, metrics, synchronization points etc. This is accomplished by making backlog items as described in the previous section.

As we in Scrum iterate through the development phases, the need for traditional project phases becomes more or less obsolete. The project management work is based on the rolling wave principle.

Exhibit 5: Scrum integrated with a traditional project management model

Implementation

Now we know the benefits of Scrum, the benefits of the more traditional approach, and we know how we want them to work together. What remains is the implementation, making it all happen. There are several hurdles, and here are a few tips on how to proceed:

• Buy-in from management. Without that, no new work process stands a chance. The increased productivity and quality are easy to sell. You need commitment that the team cannot be disturbed during a sprint, that planning is done in iterations, that the product owner is empowered and accessible for the team, to mention a few things.
• Buy-in from the organization. Scrum might create a slight culture clash with new roles, iterative planning, self-organized teams, and so forth. If the advantages are not communicated well, obstacles will seem to be everywhere.
• Create a concept of how Scrum shall work in your organization. Determine how your organization shall handle the six areas discussed in the section “Challenges.”
• Scrum Training. We need training for teams, also for managers and executives, because it is important that everyone knows the principles, practices, and terminology in depth.

A suggestion is to start with a pilot project. With a successful pilot the management and organizational buy-ins are going to be much easier. Another suggestion is to get help from outside advisors, such as an experienced Scrum coach, to ensure a successful pilot.

Final Words

First, consider the reward. You will have Scrum in place, boosting productivity, quality, and customer satisfaction, and at the same time, be able to handle the synchronizations and so forth needed in a large organization.

Second, accept the work needed to get this in place. It is crucial to get it right. You may use the guidelines in this paper, but be prepared: changing a work process always takes time and effort.

When all this is in place your new work process is real or implemented, as this condition usually is called. Then it's time for you to benefit from the best of two worlds.