Taming the test strategy dragon

As you might know I have opinions about test strategies. I have been in the enterprise testing space the last 20 years. But I can make them as cumbersome and filled with legal constraints as the stakeholder feedback I get. I can write most flavors of test strategies, the legal ones, those who are plainly a way of working, and those that are all about configuring the testcase management tool. I know how to follow a template – even if the order and headlines are fixed. That is not the issue.

A test strategy always exist within a context, within an organization. I often experience it as a persistency vessel between two parties to elaborate the framing of the testing activities in a delivery. Often the test strategy is the output of elaborations and negotiations in the early phases of the delivery. The more legal and contract management teams are involved in the process, the more it becomes useless.

Templates for test strategy documents are often both explicit and directive. You must do this, and you have to write about that. The strategy document often becomes a process framework for test tool configuration, test data handling, and other supporting activities – not about decisions and purpose. The test strategies are often a document facing into the team, and not so much anchoring up to business goals. 

The explicit activities in a test strategy document often become a pointy cog in the wheel (or bottleneck in a lean vocabulary). The testing activities get all the blame for a dysfunctional delivery, and then even more rigor is added (written in project blood as checklists usually are). The rigor of checklists impacts the test activities with more controls and checklists. The document templates paint the testing activities into yet a corner. We must write better test strategies and stop writing overdone test documents. 

Goal-Aligned Test Strategies, 2022

If you need a wiki page on how your test management tool is configured – sure, create a “Way of working” markdown file. Or if you need place for a test type vocabulary – go ahead. And surely you should have a place for both the template “Definition of Ready” and “Definition of Done”. All of these are necessary, and more “how to”-statements. Not strategic decision making. Some might thing of a test strategy into explicit “selector strategies” or patterns for test cases or even how to structure information within the IDE. The old Page Object Model comes to mind, but while relevant to who, how and where and is less about why.

I guess some of it comes from the fact that organizational strategies are equally fuzzy and filled with hot air. The word “strategic” is surely overloaded and hyped. Often I wish people would read “Good Strategy – Bad Strategy” by Rumelt before even speaking the word “Strategy”.

If you ask me – I would recommend a strategic test strategy

Strategic test strategies are not theoretical. I want them as dark and to the point as my coffee. black. No sugar. The best illustration I have for the sweet-spot for strategic test strategies, is for them to address the “here be dragons space” on the illustration below – test strategies are to bridge the business objectives, strategic objectives and existing delivery patterns into a coherent prioritization and leveraging of resources.

I wrote my first book on the topic of applying “Wardley Mapping” into the testing domain. Wardley mapping is a visual strategizing approach, over the years I have samples to share – but it can be tricky to get do do an actual mapping of the strategic decisions needed. Sometimes a couple of principles will do.

• We prioritize automating our tests over one-off manual test cases
• All interfaces have mocks and stubs
• As this is a national-security class system, all DoD/DoR include security testing

Strategic decisions are decisions that “leverage resources” in the words of Rumelt. Or similarly, things that has a cost, but yet is a line we do not cross.

Call it the continuity line. It is short. Five things, not fifty. And there are three tests for whether what you have written is a line or a poster.

It has to be losable. If holding it never costs you anything, it is not a line. “We put the customer first” costs nothing, and nobody has ever had to turn down revenue to hold it. Name the thing you would say no to. If you cannot point at a deal, a saving or a launch you would give up rather than break it, you have written a description. Descriptions do not survive a bad quarter. This is the test almost everything in the drawer fails, including work I am proud of.
It has to be encounterable. Something a customer actually meets. A rule about how you say no. A response you always give, even when it is expensive. A cost you eat instead of passing on. A constraint you never engineer around. The element rules worked because they were physical and could not be fudged. The rest of the business has to reach the same hardness through behaviour.
It has to survive your absence. Take what you have written to somebody who was not in the room and has no history with you. Hand them three live decisions from this week. If they cannot make the call you would make, the line is still in your head rather than on the page, and it will not reach the people who are making those calls without you.

https://www.ceciliaweckstrom.com/the-tenon/the-rule-that-could-kill-a-good-idea/

If principles have no teeth and no link and no clear link to the business objectives then it’s merely a paper dragon dancing in a hurricane. (Mr. Bond)

To tame your test strategy, the homework is:

1. What are one or two key objectives for your stakeholder
2. What one or two thing do you prioritize and shift from one-off to best-practice?
3. How does it hold the continuity line?