Disciplined Agile

Common Challenges for New Scrum Teams

This article describes common challenges for teams that are just starting with scrum and some common solutions to them.

When looking to improve your practices, you can't just drop one without considering its affects on the whole. There is a disciplined process that must be followed. See how to improve or change practices for making changes to help any of the following challenges.

Common Challenges

There are challenges that most every team new to scrum face. As a coach, we can anticipate these challenges and be prepared to help the team address them.

Challenges with Quality of Iterations

Challenges for Coaches

Events

Writing Stories

Organization of People and Teams Working Together

Addressing the Challenges

Having Several Incomplete Stories at the End of an Iteration

Some people suggest going to longer iterations is the way to solve this. But that merely hides the problem. Going to four weeks is almost certainly a bad idea. In fact, if you did weekly iterations, you would find yourself doing better because it creates even greater focus. It is better to ask, "Why are you spilling over tasks/stories?" Here are reasons we have encountered.

  • Stories are too big and hence have too many tasks (which are also likely too big).
  • Too many stories are being opened.
  • The team is running mini-waterfalls in an iteration.
  • Too many stories are committed to for the iteration which adds pressure to get them started.
  • You have too many interruptions.
  • There's unplanned work because of poor quality.
  • People writing code are not collaborating sufficiently with people testing the code.
  • Development teams are not collaborating sufficiently with shared services teams.
  • You haven't written acceptance tests prior to starting code (they do not need to be automated).

Here is the bottom-line advice.

  • Don't have stories that take more than three days to do.
  • Don't open too many stories.
  • Make all of your work visible.
  • Use a kanban-style daily coordination meeting.

Using test-first methods can also help this problem because it makes for smaller, well-defined stories.

Having Stories on the Backlog That Have Not Started

Having stories on the backlog that you haven't started by the end of an iteration is not the same as having incomplete stories. With non-started stories, the only real challenge is that we are over forecasting during planning. This error does not really cause any waste unless there are dependencies involved. Some teams are sometimes tempted to have longer iterations when this happens but that is typically a poor idea. It is better to get your progress known more quickly with shorter iterations.

Too Many Stories Get Started Early

When too many stories get opened the amount of Work-in-Process goes up and the level of collaboration across the team goes down. These are both bad things. This typically results from a tendency to prefer to start instead of a focus on finishing. The solution is to create agreements among team members to first look to help people out before looking to start something new. See manage work-in-process (WIP) by focusing on finishing.

Testing Is Not Complete at the End of an Iteration

While some practices or objectives may vary by the context the team is in, this is not one of them.  When more than one or two tests are not complete it is an indication that there are one or more of several problems present. These include:

  • Different people are doing coding and testing. In Scrum these should not be separate roles. But if you do have people doing coding and testing it is critical that they do this working together or at least with high collaboration.
  • The iterations are done as mini-waterfall cycles. This delays feedback. At a minimum have those writing the code help those testing the code finish before starting new work when testers get behind.
  • Testing takes too long because of a lack of automated tests. Automate the tests.

These can usually be solved by using acceptance-driven development using behavior-driven development. ATDD/BDD does not have to be a dramatic adoption but can be started in phase. See How to start with ATDD using BDD.

Other practices that will help are:

  • Automated testing
  • Continuous integration
  • Small stories
  • People writing code and people testing code working together

Running Iterations as Mini-Waterfalls

Agile is not about running shorter waterfall cycles. It's intended to build small pieces of value quickly. This both cuts out the delays during development as well as enabling quick feedback. To do this at the team level requires that features be decomposed into small, well-defined stories. The best way to do this is with a combination of acceptance test-driven development (ATDD). You don't need to even write the tests, but if you write specifications in a Given-When-Then format, they enable you to write small, clear stories. All teams need this ability from your first iteration on. It is therefore suggested if you are looking for initial training make sure your scrum training includes writing your own stories.

Having Frequent Spill-over

Going to longer iterations is typically not a good idea. If your motivation is you have incomplete stories at the end of your iterations, read having incomplete stories at the end of an iteration. If you are completing stories that have been started but have several left on the backlog it may be you are just over-committing the number of stories.

Another reason often given is that the testing cycle at the end of the iteration takes too long. If that is the case, read testing is not complete at the end of an iteration.

Some teams complete there 1- or two-week iterations but fear the overhead of the iteration is too high. But the iteration planning and review should take about half the time for a two-week iteration as for a four-week iteration. The faster feedback should also save some time. 

Getting People Interested in Test-First Methods

Many people think that you should start "test-first" with test-driven development. But TDD is often difficult to do with legacy code. The best way to start is with creating acceptance tests with behavior-driven development (BDD) or acceptance test-driven development (ATDD).

Scrum Teams Working on Different Iteration Lengths

A common situation in companies is having many teams do Scrum but having some using a different iteration length than the others. For example, most might be using two-week iterations while a couple are doing three-week iterations. Most of the time this slows down the other teams since they have difficulty coordinating with those using longer iterations.

A useful question to ask teams who are not doing two-week iterations is, “Why are you using that iteration length?” And then listen: What are the reasons they have for doing this? Then think with them about this.

For example, suppose their answer is, "We can't get our stories small enough to fit into a two-week iteration" or "The overhead of two-week iterations is too high for us." They are thinking about their own team. This is natural when teams are being told they should all just choose their own way of working, that scrum is a team practice.

Then, you can have the conversation about the larger scope, to see the whole system, to help them think about “optimizing the whole.” How the team is part of a system that is trying to realize business agility quickly, predictably, sustainably and with high quality. That teams focus should be on how they contribute to this.

So, while it may not be in their team's local interest to go to two-week iterations, it may help everyone if they are on a common cadence, everyone working in two-week iterations.

Not Being Able to Write Small Stories

While the story format of "As a < type of user >, I want < some goal > so that < some reason >" is not a bad place to start, it is not always possible to use it to create small stories. Note that this discusses what the user wants. But goals and reasons don't necessarily describe behavior. A much better format is "Given I am in <this situation>, when this <event> occurs, then I want this <behavior> to occur." The "as a" method is nice in that it gives us goals. But it is not easy to get to small stories with it. To do that, a lightweight use of Acceptance Test-Driven Development and Behavior Driven Development should be used.

Acceptance Criteria Is Not Well Known for Many Stories

One of the biggest challenges in organizations is understanding what is really needed. The question "how will I know I've done that?" is not often asked. Rather teams act as if they understand what is needed. Or they go the other way and ask for detailed requirements that will ensure an understanding. Unfortunately, it is virtually impossible to get that much detail into a written document without also making the document unusable.

Stories need to have a well-defined acceptance criteria.

Difficulty Getting Cross-Functional Teams

Cross-functional teams are great. Try to achieve them. When multiple teams are present, it gets harder and harder to actually manifest them. While it is good to attempt to get this immutable rule in Scrum, we've found it's not always possible or viable. Subject matter experts are often in short supply. However, the intention of cross-functionality can often be achieved. See Form Team.

How to Improve or Change Practices

Here is a four-step process for making a change in a practice.

  1. Are you having challenges with the practice because it is being done poorly? If yes, then inspect and adapt and see if you can do it better. If no, continue.
  2. Is there something else in the organization that is causing us this problem? If yes, then see how to fix that or at least influence the fixing of it. If no, then continue.
  3. Is the ecosystem that the team finds itself in causing the problem? That is, are people not collocated when they need to be or are required skills missing? Can you improve on this? If yes, do so. If no, continue to see if another practice that works within this ecosystem will work better.
  4. What else can we do that meets the same objective of the practice? If there is something else you can do, then try that. If not stick with the practice until you learn more. 

Now, consider if the change is better.

  • There is a set of underlying principles that can provide an indication if a change will improve things. This is always in theory to some extent, because even if a change will improve things if made, there are often side effects caused by people not adopting the change that work against it. We therefore must always be diligent and validate any change we make.
  • The measure to use is the value stream impedance scorecard (VSIS). In a nutshell, the VSIS indicates how much resistance the system will impose on work being attempted. It is based on what improves total value manifested. Lowering this resistance usually results in more value manifested. 

April 2023