Testing & QA
Releases should not be a gamble on how much anyone remembered to click. We build a test suite that runs on every change and fails in a way that tells you where to look.
Something that worked last month stops working, and nobody finds out until a customer does. So the release gets a week of manual clicking beforehand, which catches the obvious things and misses the one that matters. What fixes it is a suite that knows what the software is supposed to do, runs without being asked, and says something useful when it goes red.
Rerunning until it passes is not testing
The pipeline goes red, someone hits retry, it passes, the merge goes through. After that happens a few times the suite stops being a signal and becomes a toll. Flakiness is usually a design problem rather than bad luck: shared state between cases, real network calls, an assumption about timing. That part is fixable.
- Quarantine the flaky cases first, then fix them or delete them
- Replace shared fixtures and sleeps with data each test creates and owns
- Push cases down a level where a unit test proves the same thing
- A red build blocks the merge, so nobody learns to ignore it
Coverage counts lines, not behaviour
Coverage tells you which lines ran while the tests were running. It does not tell you whether anything was checked. A suite can execute most of a codebase and assert almost nothing, and a target set high enough will produce exactly that: tests written to touch code rather than to describe what it should do. The number goes up. The confidence does not.
So we ask two questions of a test instead. Does it state something the software is supposed to do, in language someone from the business could recognise? And when it fails, does the failure name the place to look, or does it only say that a page did not load? Coverage is still worth watching, mostly as a way of finding code nobody tests at all. It is a symptom, not a target.
What each layer is for
| Layer | What it proves | Where it runs | Who owns it |
|---|---|---|---|
| Unit | A rule behaves as written | Locally and on every commit | The engineer who wrote it |
| Integration | Code and its dependencies agree | On every commit, against real services | The feature team |
| Contract | Two services still fit together | In both teams' pipelines | Both teams, jointly |
| End-to-end | A whole journey still works | On merge, against a deployed build | The feature team |
| Performance | It holds under expected load | Before a release, and nightly | Engineering with the service owner |
| Exploratory | What a script would not ask | On a build, before sign-off | A tester using judgement |
Tools we test with
The runner matters less than what you point it at. Where you have a suite, we work in it rather than starting a second one; where the choice is open, we take the tool your team can maintain over the one with the better demo.
- Playwright
- Cypress
- Selenium
- Jest
- xUnit
- JUnit
- pytest
- Pact
- Postman
- k6
- JMeter
- GitHub Actions
- Azure Pipelines
What you are left with
Testing stops being an event. What should be true of your release once the work has landed:
A suite that runs itself
Every push runs the tests, and a failure blocks the merge. Nothing depends on somebody remembering to check before they release.
Failures that point somewhere
A red build names the behaviour that broke and the layer it broke at, so an engineer starts by fixing rather than by trying to reproduce it.
A regression pack without the week
The checks that used to take a room of people clicking through the same screens now run unattended, and they run on every change rather than only before a release.
Manual effort spent on judgement
Testers stop re-checking what a machine can check and go after the things a machine cannot: confusing flows, odd data, accessibility, and whether this is any good.
The questions we get
Not all of it, and aiming for that is how suites become expensive. Automate anything that will be checked more than a few times, anything that guards money or data, and anything a person finds boring enough to do badly. Leave exploratory work, accessibility review, and first looks at a new interface to people, because those depend on judgement rather than repetition.
Not sure which one you need?
Describe the problem in a paragraph and we will tell you which service applies.