We are usually focused on testing if it’s the right thing that is being build: Are the all the acceptance criteria covered, have the customer seen the agile demo – have we run the functional regression steps. All good an required things to do. These days we can ship functionality faster and faster from a functional perspective. I recommend quality engineers and lead testers to consider that there are at least four dimensions of quality, and when one is accounted for explore the others:
- Functionality including integrations
- Performance, resilience and scalability
- Governance, dependencies and security
- Proces compliance
if anyone of these fails the solution fails over all. In other words we have all the functional requirements tested and polished, but if we have a security incident the whole thing falls. I have covered aspects around governance previously, see below. After the jump let’s look into proces compliance.
Proces compliance
True story as of april 2026: We had a P1 on a major solution we are building. Someone, with all the best intentions, reset all the batch processing – in Prod. The business organisation was annoyed. Obviously. But what they where most annoyed about was not the technical flaw, but the communication and processes around the event. It would have mattered more to them, that we had told them about the resetting – both before it was to happen and after it happened. That we had owned it and involved them.
Which reminded me about this recent podcast (AB testing podcast 228) on among other things – validation and verification. Verification being that we built the right right – and validation being that we are building it the right way.
As many I have a awkward relationship to the process police, as if following the corporate policies to the letter makes all the difference. As aircraft check lists are written in blood so are corporate policies shaped by corporate blood and risk avoidance. Examples could include:
- If we are not CMMI / ISO27001 compliant, we we do not qualify as vendors for our corporate customers
- If we do not fill out the documents with this specific content, we are in breach of contract
- If we don’t do follow procedures around updating in production, we introduce failures for all
- if we are to update things based on findings, bugs and observations, are we involving the stakeholders adequately?
- If we work on a feature, have we delivered not only all the testing – but also updates to system specs?
The last point is something I’m helping testing in a another recent project delivery. I’m wearing the “quality control” (in life science quality management terms) of setting up meetings to control not the specific testing – but that it has happened and that the system specifications have been properly updated. Before we hand over to production operations.
Is it testing – oh, yes. Yes it most certainly is.
The is a system under test (the organisation or procedures), there is an expected outcome and steps to perform to prepare for the expected outcome. Often there is an elaborate body of knowledge – the test oracle – that can tell if things are passed or failed. And if we have to rerun the whole resilience exercise because of people problems – yes, that happened.
The four dimensions of testing above (yeah, let’s call it that). All advance somewhat in parallel, somewhat individually. If we can build succesful test automation, then surely we can build scalability automation. As practices they too follow a part from hand-crafted one-offs to repeated best practices. (see https://www.o2sn.dk/2026/03/29/return-of-the-darlings-pets-cattle-and-guids/ ).
And sure, we can build tooling to test for compliance processes. The test automation for this is, though, not your average playwright script. But perhaps more “compliance as code” that helps monitor that things are done the right way. Not just when auditors comes around but consistently through the life-time of the contract/agreement.
Let’s get our heads out of the functional bias and work with the organization as a whole.