During the Inception Phase the team should consider developing a test strategy so as to identify how they will approach verification and validation. This includes thinking about how you intend to:
- Staff your team
- Organize your team
- Capture your plan
- Approach testing
- Approach development/programming
- Choose a platform for test environment(s) platform
- Choose a platform equivalency strategy
- Test non-functional requirements
- Automate test suites
- Obtain test data
- Automate builds
- Automate test coverage
- Determine the intensity of testing
- Report defects
- Govern your quality efforts
Figure 1. The Develop Test Strategy process goal diagram (click on it for larger version)
Why is This Important?
There are several reasons why this is important. We want to ensure:
- We have sufficient skills within the team. Our testing strategy will drive whether we need people with the skills to write automated tests, the skills to perform specialized types of testing such as security testing and exploratory testing, test-first development skills, and so on.
- We have sufficient technical resources. We need to determine whether we have sufficient access to resources such as testing tools, test data, and testing environments.
- We build quality in. We want to build quality into the way that we work, rather than inspect it in after the fact. Important strategies to do this including preferring test-first or test-driven strategies over testing after the fact, coaching people in design and usability skills, testing throughout the entire lifecycle rather than testing at the end, and adopting a mindset that quality is everyone’s responsibility.
- We fulfill our quality needs. Our team may have regulatory compliance, governance procedures, and organizational standards around security and data that need to be addressed.
- We test to the risk. Our testing strategy should be driven by the risk that we face – the more complex the domain problem we face or the more complex the technology that we’re working with, the more robust our testing strategy will need to be.
- We reduce the feedback cycle between defect injection and defect identification. In the 1970s Dr. Barry Boehm, a computer science researcher, discovered that the average cost of fixing defects rises exponentially the longer it takes you to find the defect. Dr. Boehm continued researching this into the early 2010s and found, not surprisingly, that it holds true for agile as well as traditional teams. The implication is that we want to adopt testing and quality techniques that have a short feedback cycle.