DevOps
Releases stop being an event when the path to production is code, reviewed and rehearsed like anything else you ship. Pipelines, infrastructure as code, and a rollback someone has actually tried.
The release goes out on a Saturday because that is when there are fewer users to upset, and because one person knows the order the steps have to happen in. It mostly works. What it costs is everything around it: changes queue up for weeks waiting for the window, the staging environment stopped matching production some time last year, and nobody has tried the rollback since the script was written.
Rare releases are the risky ones
Deploying more often is safer than deploying rarely, for an unglamorous reason: a release that carries one change can be diagnosed in minutes, and a release that carries three months of changes can only be bisected. The ceremony that was meant to reduce risk is what concentrates it.
- Changes queue behind the window until every release is large enough to be frightening
- A rollback plan written down once and never rehearsed on a real environment
- Staging and production drift apart in ways nobody can list from memory
- Small improvements go unmade because shipping them costs the same as shipping anything
A pipeline will not change how you work
Automate a handover and you still have the handover. If the engineer who wrote the change cannot see it running, cannot read its logs without raising a ticket, and is not the one the alert reaches at two in the morning, then the pipeline has only made the same handover faster. What changes behaviour is the feedback arriving at the desk where the decision was made.
So the honest position: buying a pipeline buys you a pipeline. Whether your engineers get access to production telemetry, who carries the pager, and whether an operations team is still asked to approve a deployment it cannot evaluate are decisions we can argue for but not make. We will say which of those the tooling depends on. If none of them can move, the pipeline is still worth building, but it will buy you less than the invoice suggests.
What you are left with
None of this is exotic. The test is whether a new engineer, on their second week, can read the pipeline, recreate an environment and put a release back without asking anyone.
A pipeline you can read
Build, test and deploy defined in files next to the code, so the question of what happens on a merge has an answer anyone can look up.
Infrastructure reviewed like code
Networks, clusters and databases declared in Terraform or its equivalent, changed through pull requests. A change to a firewall rule gets the same reading as a change to a function.
Environments you can recreate
A test environment stood up from the same definitions as production and thrown away afterwards. Drift stops being something to manage when nothing survives long enough to drift.
A rollback you have rehearsed
Going back is a path in the pipeline that gets exercised, not a paragraph in a document. We practise it on a real environment before anyone needs it at speed.
What the pipeline runs on
Your existing stack usually wins. If the organisation already runs Azure DevOps, that is the answer, and a tool we happen to prefer is not worth the migration and the retraining. A case for replacing something has to be made in failures rather than preferences.
- GitHub Actions
- GitLab CI
- Azure DevOps
- Jenkins
- Terraform
- Ansible
- Docker
- Kubernetes
- Helm
- Argo CD
- Prometheus
- Grafana
- OpenTelemetry
- HashiCorp Vault
How the engagement runs
Nothing here starts with a greenfield pipeline. It starts with one service that already exists and one path to production that already works, badly, and makes that path repeatable before anything else is touched.
- 01
Watch a release
We sit through one of your deployments as it is done today, including the manual steps and the message somebody sends to say it is safe to proceed. You get the current path written down, with the parts only one person can do marked.
- 02
Pick one service
Something that matters enough to be worth improving and is small enough to finish. Its pipeline gets built end to end in your repositories, and you review the first automated deployment reaching a real environment.
- 03
Move the environments
The environments that service runs in get declared in code and rebuilt from those declarations, so the difference between staging and production becomes a diff rather than an argument. You can read the change in a pull request.
- 04
Rehearse the failure
A deployment gets rolled back on purpose, with your engineers doing it, while somebody times how long it takes and notes what was missing. What you see afterwards is the rollback procedure corrected by the attempt rather than written from imagination.
- 05
Spread and hand over
The pattern proved on one service becomes the template for the rest, with your engineers doing more of each one. The engagement ends with runbooks, dashboards and on-call procedures owned by your team, and the decision record explaining what we rejected.
What buyers press on
Because the problem usually is not the server. Pipelines fall out of use when they are slow enough that people work around them, or when a failure gives no clue which change caused it. So the first thing we look at is why yours is being bypassed, and sometimes fixing that is the whole engagement.
Not sure which one you need?
Describe the problem in a paragraph and we will tell you which service applies.