[ First posted on jlottosen.wordpress.com, Oct 2023 ]
Spending your time writing long-form test documents is not an efficient way to communicate, nor will it get you the situation-specific support that you probably need. Don’t listen to the stochastic parrot – request information that is relevant to the situation at hand.
Recently I read a Slack thread on test plans, and one of the comments was that we should stop making things so wordy and reduce overexplaining everything. That reminded me of the time I wrote a 100-page test strategy that was very far from strategic – it was rather a specification of the how & the what, and less about why. Unfortunately, I had no choice in the content as it was part of a local public tender reply. To win the thing we had to provide a document called “Test Strategy” with items like:
- Start and Approval Criteria
- Listing of Applied Testing Types
- Milestones
- Risk areas
- Methods for testing coverage
- Test environments and test data
- Test tools
- Consideration of the Need for Test Automation
‘ve been around the testing block to recognize the old book this is coming from it’s older than 2008 BTW). While it seems helpful to elaborate on everything and the kitchen sink, it’s mostly busy work of empty calories. We have better ways of working than a big specification (of testing activities) upfront.
Don’t spend time writing a detailed Test Plan which will be out of date in no time and most people don’t bother reading (TL;DR). Instead, focus on communicating. The best plan doesn’t win. The best intelligence wins!
Andy Glover
The Stochastic Parrot
My bet for the future is that long-form template documents will soon be generated using large language models. The more you ask for the more will be auto-generated by a stochastic parrot [On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 Authors: Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell]
The better approach for customers of IT services is to ask for less text. Have the documents zoom in on the vendor’s perception of the client’s challenges. Yes, it will take more time and effort – but you can still score a short essay for the tender process.
Make shorter planning documents – and for strategies, make a map.
One response to “Wordy Documents Are Not Better”
Issue #106 : Software Testing Notes
https://softwaretestingnotes.substack.com/p/issue-106-software-testing-notes%5D
“ Long form test plan with detailed documentation has always been give extremely high importance in the testing world. But it not might be as effective in a world with LLMs as Jesper Ottosen suggest.”