Inference Public

Platform engineering

Your build pipeline is a production system.

It holds access that can change the live service and it runs code written by many other people. When it fails, urgent fixes wait. It deserves an owner.

Engineering workstation with a small computer, laptop and connected storage.

Look at what a pipeline can do

A typical pipeline checks out source code, downloads third-party packages and shared actions, runs them, and then assumes a role that can change the live service. That is a great deal of privilege for a system that often has no named owner, no patching schedule and no recovery plan. If it is unavailable on the day an urgent fix is ready, the fix waits too.

Anything that can deploy to production is part of production

Because a pipeline can change the live service, it belongs inside the same boundary of care. Its dependencies, plugins and shared actions run with its permissions. It needs an owner, monitoring, patching and a recovery plan in the same way as the services it deploys, and it should appear by name in the service's risk register.

Replace stored keys with short-lived credentials

Long-lived cloud keys stored as pipeline secrets are valuable to an attacker. GitHub Actions and similar tools can exchange a short-lived OpenID Connect token for temporary AWS credentials through a trusted IAM role, which removes the need to store long-lived keys at all.

Access then depends on the role's trust policy and the workflow's permissions, so those need the same care the keys once did. GitHub's guidance is to condition the trust policy on the token's subject claim, which can limit the role to a specific repository and branch or to a named deployment environment with protection rules. A wildcard that admits every branch in a repository grants more than it appears to. With GitOps, the build itself only needs permission to publish an image and propose a configuration change.

Make what it produces trustworthy

Build each artefact once and promote that same artefact through every environment. Store images with immutable tags so that a version number cannot later point at different code, and deploy by digest. Our article on pipelines that produce their own evidence covers the records each stage should leave.

Decide in advance what happens when it breaks

The GOV.UK Service Manual accepts that an emergency fix may skip non-essential pipeline stages. The useful work is deciding beforehand which stages count as non-essential, who may authorise the fast path, how it is recorded and how you would deploy at all if the build service were unavailable. Test that route once and keep the result with the runbook.

Watch its speed and reliability

A slow pipeline encourages people to batch changes together, which works against the advice to deploy little and often. Tests that fail intermittently teach people to retry without reading the result. Track duration and failure causes, and treat an unreliable test as a defect with an owner.

Pipelines and AI agents

Pipelines are increasingly read and operated by AI agents as well as people, and build logs or repository content can contain instructions that nobody intended an agent to follow. I explored that separately in When the build log starts giving orders. The principle is that untrusted text should never be able to grant permissions.

Working document: a release runbook

A short runbook kept with the service lets anyone on the rota release, verify and roll back without relying on memory.

Illustrative template
FieldWhat to record
Release identityVersion, image digest, source commit and change record.
Pre-release checksThe pipeline evidence to review and who reviews it.
DeploymentThe steps, or the configuration change that triggers deployment.
VerificationThe essential user tasks checked after release and who confirms them.
RollbackWhat triggers it, how it is done, its expected duration and any data considerations.
Emergency routeStages that may be skipped, who authorises this, how it is recorded and when the follow-up review happens.
Pipeline unavailableThe alternative deployment route, who holds access to it and when it was last tested.

Read about platform engineering or discuss your path to production.

Further reading

GOV.UK Service Manual: deploying software regularly

GitHub Docs: configuring OpenID Connect in Amazon Web Services

Amazon ECR: image tag mutability

Related articles

A deployment pipeline that produces its own evidence.

What an embedded DevOps engineer needs to succeed

All insights

Start a conversation

Let’s talk about what you need to build.

Bring the service, the constraint or the engineering question. We’ll work through a sensible starting point.

Discuss a project