Canary, blue/green & rollback

Ship safely, revert instantly.

Even a reviewed, verified deploy can be broken in a way tests did not catch. Progressive delivery limits the blast radius of a bad release by not sending all traffic to the new version at once. The two classic strategies are canary — route a small slice of traffic to the new version, watch it, then ramp up — and blue/green — run the new version alongside the old and flip traffic over in one switch, with the old version kept ready to flip back.

Automated canary analysis

A canary is most powerful when the promotion is automated against real signals. Argo Rollouts and Flux’s Flagger watch metrics — error rate, latency, saturation — while the canary takes a slice of traffic, and they promote the release step by step only if the metrics stay healthy, or abort and roll back automatically if they degrade. The deploy becomes self-driving: it advances on good data and reverts on bad, without a human staring at a dashboard.

spec:
  strategy:
    canary:
      steps:
        - setWeight: 5
        - pause: { duration: 5m }        # observe
        - analysis: { templates: [{ templateName: error-rate }] }  # auto-abort if bad
        - setWeight: 25
        - pause: { duration: 5m }
        - setWeight: 100

Rollback is the real safety net

The point of all of this is a fast, reliable way back. In GitOps, rollback is reverting the commit that bumped the image — the agent reconciles the cluster back to the previous known-good state, and because that state is in Git, it is exact and instant. Blue/green rollback is flipping traffic to the still-running old version. Whatever the strategy, "how do we get back to the last good version in under a minute?" is the question every delivery design must answer.