Disciplined Agile

Develop Test Strategy

During the Inception Phase the team should consider developing a test strategy so as to identify how they will approach verification and validation. This includes thinking about how you intend to:

  • Staff your team
  • Organize your team
  • Capture your plan
  • Approach testing
  • Approach development/programming
  • Choose a platform for test environment(s) platform
  • Choose a platform equivalency strategy
  • Test non-functional requirements
  • Automate test suites
  • Obtain test data
  • Automate builds
  • Automate test coverage
  • Determine the intensity of testing
  • Report defects
  • Govern your quality efforts
2021 Project Management Institute Develop Test Strategy v5.2 Test Staffing Strategy Generalizing specialists Exploratory testers Behavior-driven development (BDD) analysts Test automation specialists Manual testers (from scripts) Test Teaming Strategy Whole team/embedded testers Parallel independent test team Independent test team External (outsourced) test team Level of Detail of Test Plan Outcome driven Lightweight overview Detailed specification None Test Approaches Black box Clear box Confirmatory Exploratory Stakeholder validation Test Intensity Life critical Business critical Product critical Project critical Development Strategy Test-driven development (TDD) Test-first programming (TFP) Test-after development Testless programming Quality Requirements Testing Strategy Accessibility Availability Concurrency Data privacy Internationalization Performance Resilience/reliability Scalability Security Usability Volume/rate Test Environment(s) Platform Strategy Cloud based Physical Virtual Test Environment(s) Equivalency Strategy Production equivalent Production approximate Disparate Choose Testing Types Accessibility testing Alpha/beta/pilot/canary testing Component testing Database testing Exploratory testing Functional testing (FT) Performance testing Prototypes Quality attributes (ility) testing Security testing Simulations Split (A/B) testing Story testing System integration testing (SIT) Unit testing (UT) User acceptance testing (UAT) User experience (UX) testing User interface (UI) testing Workflow/scenario testing Test Suite Strategy Multiple regression test suites Single regression test suite Manual testing Test Data Source Original production data Masked production data Generated data Engineered data Service/source virtualization Test Automation Strategy Continuous deployment (CD) Continuous integration (CI) Automated regression tests Scripts Manual testing Test Automation Coverage Multiple systems Solution Service/application programming interface (API) Component Unit User interface (UI) Defect Reporting Conversation Operational monitoring Agile management tool Bug/defect tracker Test management tool Quality Governance Strategies Nonsolo work Tool-generated metrics Automated code/schema analysis Quality guidelines Informal reviews Formal reviews Test case documentation

Figure 1. The Develop Test Strategy process goal diagram (click to enlarge)


Why is This Important?

There are several reasons why this is important. We want to ensure:

  • We have sufficient skills within the team. Our testing strategy will drive whether we need people with the skills to write automated tests, the skills to perform specialized types of testing such as security testing and exploratory testing, test-first development skills, and so on.
  • We have sufficient technical resources. We need to determine whether we have sufficient access to resources such as testing tools, test data, and testing environments.
  • We build quality in. We want to build quality into the way that we work, rather than inspect it in after the fact. Important strategies to do this including preferring test-first or test-driven strategies over testing after the fact, coaching people in design and usability skills, testing throughout the entire lifecycle rather than testing at the end, and adopting a mindset that quality is everyone’s responsibility.
  • We fulfill our quality needs. Our team may have regulatory compliance, governance procedures, and organizational standards around security and data that need to be addressed.
  • We test to the risk. Our testing strategy should be driven by the risk that we face – the more complex the domain problem we face or the more complex the technology that we’re working with, the more robust our testing strategy will need to be.
  • We reduce the feedback cycle between defect injection and defect identification. In the 1970s Dr. Barry Boehm, a computer science researcher, discovered that the average cost of fixing defects rises exponentially the longer it takes you to find the defect. Dr. Boehm continued researching this into the early 2010s and found, not surprisingly, that it holds true for agile as well as traditional teams. The implication is that we want to adopt testing and quality techniques that have a short feedback cycle.

More Information

Choose Your WoW!

The strategies/practices referenced in the goal diagram above are described, including the trade-offs involved and considerations for when (not) to apply them, in the book Choose Your WoW! A Disciplined Agile Delivery Handbook for Optimizing Your Way of Working.

If you want to succeed at enterprise agile you need choices, not prescriptions.

LEARN MORE