The Phases of ATDD
The practice of doing analysis via the discussion of acceptance criteria and writing the results in the form of acceptance tests is known as both Behavior-Driven Development (BDD) and Acceptance Test-Driven Development (ATDD). The intentions of both are the same, the biggest difference is in the formatting of the test specifications. If teams are having difficulty breaking features into small stories, the Given-When-Then format of BDD may be the preferred starting point.
ATDD using BDD is one of the first things all companies should adopt, whether they have already transitioned to Agile, are desiring or starting to transition, or are even staying with a traditional or Waterfall approach. It is such a critical discipline that we’ve built our team-level approach for people wanting to use Scrum, the Net Objectives approach to Scrum.
Starting with BDD/ATDD is both easier and safer than starting with traditional Test-Driven Development (TDD). TDD is often difficult and risky to do with legacy code. Not only does BDD/ATDD allow software development organizations to better meet stakeholder expectations, it also results in better overall code quality. This is because considering tests prior to writing any code is the first mantra of design patterns. It results in better design while helping avoid extra code. Other benefits include better tester-coder alignment, coordination, and collaboration.
BDD/ATDD can be adopted in stages. These are discovery, where we discover what the requirements are, specification where we create scenarios for what we’ve discovered automation where we automate the scenarios.
The first part of discovery virtually no extra work and the rest of discovery takes very little and all return great value. As one goes deeper into full implementation, even more value is achieved. The important thing is to realize that you start where it is best for you.
Discovery is the beginning phase of ATDD. It is where we initially discover what the requirements are. As we go deeper into ATDD additional discovery will be made. There is no implication that this discovery phase is absolute or cannot be revisited. Rather it is just the start of the process.
Level 0: Ask the Question “How will I know I’ve done that?” and consider tests prior to writing code
At a minimum, developers should both ask the question “How will I know I’ve done that?” while also considering how they will test their code prior to writing it. This doesn’t take any extra work. It’s merely changing the order in which steps you are going to do take place. Even if they don’t document anything, this mere consideration of behavior is a kind of design. This is actually the first mantra of design patterns and greatly increases testability. Doing this as a practice takes virtually no additional effort. Instead, one should think about it as a rearrangement of the order in which the workflow is done occurs. Instead of:
- Consider what is needed
- Consider how to validate what is needed <—
We are putting step 4 up above step 2:
- Consider what is needed
- Consider how to validate what is needed <—
We call this “Level 0” because it doesn’t require any extra work.
Level 1: Use test specifications as an analysis tool and to validate the requirements
The next level is for developers to validate their understanding with the BA/PO/Customer. This small documentation is useful for the obvious reasons that it will help when anyone has to come back to maintain the code. But the real value is the quick feedback provided to the developers on their understanding of the requirements.
Level 2: Have Business Analysts, Product Owners, customers, developers, and testers write acceptance criteria together
One of the biggest advantages of BDD/ATDD is that it changes the nature of the conversation between whoever is representing the customer, the developers, and the testers. It is a great way to uncover assumptions that will cause waste. And assumptions are killers to clarity. Many times we make assumptions without realizing they are assumptions (something we don’t know and we don’t know we don’t know it). In these cases, we don’t know the right question to ask to discover the mistaken understanding. In Level 2, getting the different roles to talk to each other and create acceptance tests, these hidden assumptions tend to get discovered.
Specification is taking what we’ve discovered and converting them into scenarios. There are several ways to do this. We tend to prefer Given / When / Then because it enables breaking big stories into small stories in a straightforward manner.
Level 3: Use Given / When / Then as a format
BDD is often thought of as being defined by GWT. But the two are not synonymous. While tool support is available for GWT, it is not required to get massive benefit. GWT means:
- Given a <situation>
- When an <event>
- Then I want this <behavior> to occur
Note that GWT specifically ignores how to implement the behavior.
While the ubiquitous “As a < type of user >, I want < some goal > so that < some reason >” is a good place to start, it is often difficult to create small stories with this. GWT can be used to slice up any feature, no matter how complicated, because the “Given” and “When” can be made more and more specialized.
Although a lot of teams think they need to improve code quality with TDD, the reality is that starting with acceptance criteria can reduce bugs by up to 95%. It also is the first mantra of design patterns (design to behavior).
BDD’s (behavior-based development) GWT enables people to get down to small stories (every time) as well as setting up for automated testing. The “As a …” can still be used as a description of the first story/feature to provide the “why” of what is being asked for. But after that first case (or soon thereafter), it’s better to go with GWT.
Level 4: Put acceptance criteria into a test-tool harness
In this level, acceptance tests are implemented in a test tool or harness as soon as they are captured. This is not difficult. Once the acceptance tests are defined (which is the more involved part), writing the acceptance tests is easy, especially if you are using a test tool or harness. While you’ve got them, you might as well capture them!
Automation is the final stage and highly recommended. The largest investment in time is in discovery and specification. While automating the tests is not necessary, if you’ve gotten this far this final investment is well worth it.
Level 5: Automate the tests
Some test harnesses make this automatic. However, even if you have to do some work here, it will pay itself back quickly by reducing regression testing. Think of all the places where tests currently don’t exist and manual regression testing slows you down. Don’t keep adding to that pile.
Be clear that BDD/ATDD is not tool-driven. It is a thought process and an essential part of Agile Product Management. The fact that it can be automated is just an additional advantage.
How long it takes to adopt ATDD
There is no set time frame for adopting ATDD. Much of it depends upon how much energy the team is intending to take. You should do level 0 (asking the question “how will I know I’ve done that?”) all the time. It doesn’t slow things down and is almost always useful. Level 1 doesn’t take a lot of time either. However, as you go up the higher levels each story takes more time to write tests for. We advise doing as many of these as you can but recognize that at first things more time – so you may not be able to do all of them this new way.
The increasing return for step-wise investments. Consider what level is appropriate for your group. Consider the ROI for each level’s benefit of BDD/ATDD. While it may be difficult to start with a full adoption of BDD/ATDD, virtually every development group should start at Level 2 or higher. It’s really not that hard.
BDD/ATDD is so important that we’ve integrated levels 0-3 into the course, Scrum with Agile: Requirements:Achieving Sustainable Agility.
We are happy to talk with you in more detail to let you know-how.
Acknowledgment: The phases of discovery, specification and automation come from Discovery: Explore behavior using examples by Gaspar Nagy and Seb Rose, a book we highly recommend as well as input from Liz Keogh.
Systems Thinking and How It Can Be Applied to Frameworks and Methods
What is systems thinking?
Systems thinking is considering a system to be an interrelated and interdependent set of parts which is defined by its boundaries and is more than the sum of its parts (subsystems). Changing one part of the system affects other parts and often ripples through the entire system. While the specific impact itself may not be predictable, there are many patterns of behavior which are.
The goal of systems thinking is to systematically discover the system’s dynamics, constraints, conditions and elucidating principles (purpose, measure, methods, tools, etc.) that can be discerned. With this knowledge improvements can be suggested and then run as experiments, knowing that seemingly simple changes often will create unknown side-effects.
A quick overview of systems thinking
Consider a single-engine airplane. What part of it is responsible for flight? It’s a pretty silly question, actually. While you need an engine for power, wings for lift, controls to direct it, a place for the pilot, and other things, not one of them is responsible for flight. They are all responsible. If you improve one of them (e.g., make a stronger engine) you may actually damage the ability of the plane to fly (e.g., you may now not have large enough wings to provide you with enough lift). This illustrates the first lesson of systems thinking – systems work as a whole and improving one part may degrade the performance of the whole.
There is another aspect of systems thinking related to this. Consider identifying the fifty best cars in the world. Now, consider which of these cars has the best transmission, which one has the best engine, which one the best seats, and so forth until you have the best parts of the best cars. Now, let’s take it a step further and get these best parts of the best cars and try to put them together. They won’t fit. You’ll just have a pile of junk. But let’s say you somehow got them to fit- you still wouldn’t have a working car. This illustrates the second lesson of systems thinking – it is more important to see how the parts of a system fit together than what the parts themselves are.
Some of the lessons from these two insights are:
- never assume a local improvement in a system will be a system wide improvement
- when making a change to a component in the system, you need to attend to how other parts of the system relate to the component
Systems thinking and complexity
Systems thinking tells us to attend to the relationships between the components of a system more than the components themselves. The challenge is as the number of components increase the number of relationships between them increases exponentially. And, when these relationships have to do with human behavior they are only somewhat predictable. But this complexity does not mean that we cannot make reasonably accurate predictions about the effects of change in a system. Nor that we can’t learn how to make better predictions when we are wrong. While it is true that one cannot have certainty on what affects a change will make, there is a fair amount of predictably available.
The approach to take is to make predictions based on the laws of development present and on the known relationships and current state of the organization. We can make a change, but know we may be missing some relationships and possibly be misinterpreting others. We therefore consider the change an experiment. One that we think will work. If we don’t get an improvement we have likely found out about a relationship we weren’t aware of or a relationship that we have to be careful of. In either event we’ve learned something and our next “experiments” should be better.
Systems thinking suggests attending to how people will react to the suggested approach
Systems thinking is a foundation of FLEX. In order to consider the system as a whole we must look at what we want the system to achieve – quick realization of business value, predictably, sustainable and with high quality. When we get challenges to this we want to look and see why and what can be done about it.
However, looking at the work mechanistically is insufficient. How the organization will react to any changes must be considered by any approach you take. Their reaction is part of the system. The natural laws of software development are now pretty well known. The challenge is not just what are the correct practices as much as how to get people to adopt them.
Our approach must be understanding our challenges, the intentions they are blocking and considering different solutions to eliminate the challenges. Many of these challenges are in fact how people react to change. We cannot just say “follow this” without attending to how people react to that mandate. With apologies to Nike, you can’t just say “just do it.” The greatest areas of unpredictably in a system is how people will react – not the laws of product development.
If the approach you are undertaking doesn’t incorporate this systems thinking perspective, expect difficulties and resistance. If your approach requires that people need to be a certain way (e.g., commitment, focus, openness, respect and courage), you must ask yourself “what will happen if they don’t have these qualities?” You should not be surprised if you don’t get the results the approach purports. In this situation you should also ask “how can I change the system so as to encourage these values?”
FLEX recognizes that it becomes part of the system when used to improve it. Hence it considers how people learn and change in its guidance. When applying FLEX, it is important to see how people react to it. Some of this is in the ability of the coach to interact with people (see What to say when someone just doesn’t get it) in the Appendix for more). This is one of the reasons that FLEX is designed to be used in a step-wise manner attending to the culture of the organization.
Applying Systems Thinking to Frameworks
A system is a network of interdependent components that work together to try to accomplish the aim of the system. A system must have an aim. Without the aim, there is no system. Edwards Deming
There are several ways systems thinking can be applied to frameworks. These include:
Beware of taking part of a framework and applying it. While starting with part of a framework can be a good way to start, taking parts of a framework may have unintended consequences. Systems thinking tells us that not attending to the relationship this part has with others may cause problems. This is not to say not to do this, but to be aware of challenges it may cause.
Ensure the goal of the system is made clear throughout the organization. Alignment is not possible if people are working at cross-goals.
Everyone needs to do their work within the context of the intent of the system. While delegation is a cornerstone of Lean and Agile, the delegator must always make the desired outcome clear. This might even be that the delegatee must clarify things. People at all levels in the organization must attend to how their local actions will affect the big picture.
A system must be managed. It will not manage itself. Left to themselves, components become selfish, competitive, independent profit centers, and thus destroy the system. Edwards Deming
Challenging the Assumption That One Must Get Teams to Work First
This post was originally written 2014-03-15
As some of you may have seen, Ron Jeffries put forth a blog on SAFe that assumes something I don’t agree with – always get teams doing Agile before starting to scale. By Scale, I don’t mean making projects larger, but rather having Agile extend across the entire project.
As Ron puts it “most of your teams may not really be Agile at all. Until all your individual teams can really do what Agile teams do, using all those XP practices that SAFe does wisely recommend, it’s not time to start directing them with a large process. It’s time to get them trained and experienced in doing software right. After they can do it, you’ll find that most of them don’t need all that top-down large process after all. The large process is trying to compensate for the problem rather than fix it.”
While I think there is merit to this thinking, I do not agree the best way to achieve scale is always by getting the teams working well. As with most things, an approach depends upon the situation you are in. If the types of projects you are working on don’t require teams of more than 30-50 folks, I’d probably take Ron’s approach, because SAFe is likely too big for them. But many projects are too big or complex to unwind. In this case, looking at the big picture and getting teams to deliver in 2-week cycles may be the right approach.
The bottom line for me is that 1) I agree you want to solve a problem and not merely work around it, 2) look to see what the best way to solve the problem is. If it’s there is no appreciation for agile, perhaps getting the teams to work in Agile methods is better. But if the root cause is how to get multiple skills in a siloed organization to work together, taking on an entire train may be best.
Part of Agile thinking requires us to avoid getting trapped in familiar/common solutions. I am reminded of Bucky Fuller’s quote:
“I am enthusiastic over humanity’s extraordinary and sometimes very timely ingenuity. If you are in a shipwreck and all the boats are gone, a piano top buoyant enough to keep you afloat that comes along makes a fortuitous life preserver. But this is not to say that the best way to design a life preserver is in the form of a piano top. I think that we are clinging to a great many piano tops in accepting yesterday’s fortuitous contrivings as constituting the only means for solving a given problem. ”
In the same way, we shouldn’t assume SAFe will solve our problems, we should also not assume getting teams to work will solve our problems. I am a great believer in systems thinking. This means that most of our problems are due to the system we are working in. In large systems, local solutions are unlikely to get at the root cause. In those cases, SAFe may very well be the correct solution.
Here’s a related blog: The Implications of Systems Thinking and Complex Systems
what other D.O.G.M.A. is out there?
Dogma – Persistently working towards a goal without ever questioning the methods being used.
Persistence – Dogmatically working towards a goal without ever being committed to which methods being used.