Disciplined Agile

Develop Test Strategy

This Inception phase process goal describes that the team should consider developing a test strategy so as to identify how they will approach verification and validation.

2021 Project Management Institute Develop Test Strategy v5.2 Test Staffing Strategy Generalizing specialists Exploratory testers Behavior-driven development (BDD) analysts Test automation specialists Manual testers (from scripts) Test Teaming Strategy Whole team/embedded testers Parallel independent test team Independent test team External (outsourced) test team Level of Detail of Test Plan Outcome driven Lightweight overview Detailed specification None Test Approaches Black box Clear box Confirmatory Exploratory Stakeholder validation Test Intensity Life critical Business critical Product critical Project critical Development Strategy Test-driven development (TDD) Test-first programming (TFP) Test-after development Testless programming Quality Requirements Testing Strategy Accessibility Availability Concurrency Data privacy Internationalization Performance Resilience/reliability Scalability Security Usability Volume/rate Test Environment(s) Platform Strategy Cloud based Physical Virtual Test Environment(s) Equivalency Strategy Production equivalent Production approximate Disparate Choose Testing Types Accessibility testing Alpha/beta/pilot/canary testing Component testing Database testing Exploratory testing Functional testing (FT) Performance testing Prototypes Quality attributes (ility) testing Security testing Simulations Split (A/B) testing Story testing System integration testing (SIT) Unit testing (UT) User acceptance testing (UAT) User experience (UX) testing User interface (UI) testing Workflow/scenario testing Test Suite Strategy Multiple regression test suites Single regression test suite Manual testing Test Data Source Original production data Masked production data Generated data Engineered data Service/source virtualization Test Automation Strategy Continuous deployment (CD) Continuous integration (CI) Automated regression tests Scripts Manual testing Test Automation Coverage Multiple systems Solution Service/application programming interface (API) Component Unit User interface (UI) Defect Reporting Conversation Operational monitoring Agile management tool Bug/defect tracker Test management tool Quality Governance Strategies Nonsolo work Tool-generated metrics Automated code/schema analysis Quality guidelines Informal reviews Formal reviews Test case documentation

Figure 1. The Develop Test Strategy process goal diagram.

Click the diagram to open the interactive DA Browser, where you can learn more about the decision points and options of this goal.

Why is This Important?

There are several reasons why this is important. We want to ensure:

  • We have sufficient skills within the team. Our testing strategy will drive whether we need people with the skills to write automated tests, the skills to perform specialized types of testing such as security testing and exploratory testing, test-first development skills, and so on.
  • We have sufficient technical resources. We need to determine whether we have sufficient access to resources such as testing tools, test data, and testing environments.
  • We build quality in. We want to build quality into the way that we work, rather than inspect it in after the fact. Important strategies to do this including preferring test-first or test-driven strategies over testing after the fact, coaching people in design and usability skills, testing throughout the entire lifecycle rather than testing at the end, and adopting a mindset that quality is everyone’s responsibility.
  • We fulfill our quality needs. Our team may have regulatory compliance, governance procedures, and organizational standards around security and data that need to be addressed.
  • We test to the risk. Our testing strategy should be driven by the risk that we face – the more complex the domain problem we face or the more complex the technology that we’re working with, the more robust our testing strategy will need to be.
  • We reduce the feedback cycle between defect injection and defect identification. In the 1970s Dr. Barry Boehm, a computer science researcher, discovered that the average cost of fixing defects rises exponentially the longer it takes you to find the defect. Dr. Boehm continued researching this into the early 2010s and found, not surprisingly, that it holds true for agile as well as traditional teams. The implication is that we want to adopt testing and quality techniques that have a short feedback cycle.

Important Questions to Consider

  • How do you staff your team?
  • How do you organize your team?
  • How do you capture your plan?
  • What approach should you take to testing?
  • What approach should you take to development/programming?
  • Which test environment(s) platform should you choose?
  • Which platform equivalency strategy should you choose?
  • How do you test non-functional requirements?
  • How do you automate test suites?
  • How do you obtain test data?
  • How do you automate builds?
  • How do you automate test coverage and reporting?
  • How do you determine the intensity of testing?
  • How do you report defects?
  • How do you govern your quality efforts?

January 2023