CI/CD & Deployment Strategies: Blue-Green and Canary

Part 1: Continuous Integration (CI)

CI (Continuous Integration) means developers merge small changes into the main branch often (at least daily), and every change is built and tested automatically. The main branch is the shared line of code that everyone's work joins. A pipeline is the list of automatic steps that run for each change.

Lint is a tool that reads your code and warns about style problems and likely mistakes. Static analysis is a tool that checks code for bugs without running it. A unit test is a small automatic check of one function. A container registry is a server that stores built container images.

Good CI practices:

• Trunk-based development: keep branches short (hours to a couple of days) and merge them to main often. Branches that live a long time cause big merge conflicts (two people changed the same code in different ways).
• Fast pipelines: aim for minutes, not hours. Slow CI makes people save up changes and merge them in big groups.
• Keep main always ready to release. If the build breaks, fix it at once (or undo the change that broke it).
• Build once, promote the same artefact. An artefact is the file you build, such as a container image. The exact same image (named by its commit SHA, the unique ID of a commit, or by its digest, post 55) moves through test → staging → production. Do not rebuild it for each environment. "Promote" means move it to the next environment.
• Automatic tests at several levels: many fast unit tests (test one small piece), fewer integration tests (test pieces working together), and a few end-to-end tests (test the whole app like a user). This shape is called the "test pyramid".
• Security checks built in (post 52).

Part 2: Continuous Delivery vs Continuous Deployment

• Continuous Delivery: every change that passes the pipeline is ready to deploy to production at any time, but a person clicks the button.
• Continuous Deployment: every change that passes the pipeline goes to production automatically, with no manual step.

Both need strong automatic tests, monitoring and safe rollout strategies. We look at those next.

A typical pipeline:

Part 3: Deployment strategies

1. Recreate (stop everything, start new)

Recreate is a strategy that stops all the old instances and then starts the new ones. A rollback means going back to the previous version. The blast radius is how many users a bad change can hurt.

• ✅ Simple, and old and new versions never run together.
• ❌ Downtime. This is only acceptable for dev environments or systems with planned maintenance windows (times when users expect the service to be off).

2. Rolling update (replace instances gradually)

v1 v1 v1 v1  →  v2 v1 v1 v1  →  v2 v2 v1 v1  →  v2 v2 v2 v1  →  v2 v2 v2 v2

• ✅ No downtime, and no extra full environment is needed. It is the default in Kubernetes (post 56).
• ❌ Old and new versions run at the same time, so they must be compatible (APIs, database schema, which is the layout of the tables, and message formats).
• ❌ To roll back, you go backwards through the same process, which is not instant.

3. Blue-green deployment

Blue-green deployment is a strategy that keeps two identical copies of production and moves users from one to the other in one step. Run two identical production environments. Blue is live, green gets the new version. You test green, then switch all traffic at the router or load balancer (the part that sends users to servers).

• ✅ Instant switch and instant rollback (you just switch the router back).
• ✅ You can test the new version in an environment like production before it gets real traffic.
• ❌ Double the resources during the deploy.
• ❌ All users move at once, so a bug hits everyone until you switch back.
• ❌ Shared databases still need schema changes that work with both versions (see Part 5).

4. Canary release

A canary release is a strategy that sends the new version to a small group of real users first. A rolling update replaces old instances with new ones a few at a time. Send a small percentage of real traffic to the new version, watch the metrics, and increase gradually:

Step 1:   95% → v1     5% → v2 (canary)    watch errors, latency, business metricsStep 2:   75% → v1    25% → v2             still healthy?Step 3:   50% → v1    50% → v2Step 4:            100% → v2               ✅ done   At ANY step, if metrics worsen → send 100% back to v1 automatically

• ✅ Limits the blast radius. A bug affects only a small share of users, for a short time.
• ✅ Real production traffic tests the release. This finds problems that staging (a copy of production used for testing) never shows.
• ✅ Works well with automated analysis: compare the canary's error rate and latency with the baseline (the old version), and roll back automatically if the canary is worse.
• ❌ You need good traffic routing (a load balancer is a server that spreads requests across servers; a service mesh is a layer that manages traffic between services; a gateway is the single entry point for requests) and good metrics (numbers that show how the system is doing, posts 46–47).
• ❌ Old and new versions run together, so they must be compatible.

Canary tips:

• Choose what to watch: error rates, p99 latency (the time that 99 out of 100 requests beat), crash rates, and business metrics (checkout success, sign-ups).
• Bake time: wait long enough at each step to catch slow problems, like memory leaks (a program that slowly uses more and more memory).
• For big platforms, roll out in waves: one server → one AZ (availability zone) → one region → all regions. A bad change then never hits everything at once.

5. Shadow (dark) traffic

Send a copy of real production requests to the new version, but do not send its answers to users. Compare its results and speed with the current version.

• ✅ Tests with real traffic and no effect on users.
• ❌ Doubles the load, and you must prevent side effects. The shadow copy must not send emails, charge cards or write to real data.

6. A/B testing

A/B testing works like a canary, because it also splits traffic. But the goal is different. The goal is product experiments: "does version B get more people to buy (conversions)?". It is usually controlled by feature flags, and you measure business results over days or weeks.

Part 4: Feature flags, separating deploy from release

Deploying code (putting it on servers) and releasing a feature (letting users see it) do not have to happen together. A feature flag (or feature toggle) is a switch in your code that turns a feature on or off without a new deploy. The new code goes out turned off, and you turn it on step by step:

if (flags.isEnabled("new-checkout", { userId, country })) {  return newCheckout(req);}return oldCheckout(req);

Uses:

• gradual rollout: your own staff → 1% of users → 10% → everyone,
• targeting: by country, plan, device or beta testers,
• kill switches: a kill switch is a flag that turns off a broken or costly feature at once, without a deploy (post 44),
• trunk-based development: merge unfinished work safely, because it is hidden behind a flag.

Flag hygiene:

• Remove old flags after the full rollout. Hundreds of old flags become flag debt: confusing code and combinations that nobody tested.
• Give each flag an owner and an expiry date.
• Remember that flags themselves are changes. Turning one on for everyone is the same as a release, so watch it like one.

Tools include LaunchDarkly, Unleash, Flagsmith, OpenFeature (an open standard that works with any vendor), and systems that companies build themselves.

Part 5: Database changes without downtime

You can roll back code in seconds. Database schema changes often cannot be rolled back. (A schema is the structure of the database: its tables and columns.) During rolling, canary and blue-green deploys, old and new code run at the same time against the same database. So schema changes must be backward compatible, which means the old code still works with them. Use the expand and contract pattern (also called parallel change):

Example: renaming a column name to full_name

Every step is safe to roll back to the step before it.

Other rules:

• Never drop or rename columns in the same deploy as the code change. (To drop a column is to delete it.)
• Run large migrations online: in small batches, with tools made for big tables. This avoids long table locks (times when nobody else can use the table).
• Take a backup (or check that point-in-time recovery works, post 45) before risky migrations.

Part 6: Automated rollback and safety nets

• Health-based auto-rollback: if the error rate or latency goes past a set limit during rollout, roll back automatically. Nobody needs to wake up at 3 AM.
• Deployment windows and freezes: do not make risky changes during busy times (a big sale), or when the error budget is used up (the amount of failure your reliability goal allows, post 47).
• Deploy markers on dashboards: show exactly when each version went out. Then you can link an incident to a change quickly (post 48).
• One-click rollback: always know how to go back to the previous version, and practise it.
• GitOps: a way of running systems where the desired state is stored in Git, and tools like Argo CD or Flux make the cluster match it. Deploying means merging a pull request, and rolling back means reverting a commit (undoing it).

Part 7: Measuring delivery performance, the DORA metrics

The DORA research programme (DevOps Research and Assessment), described in the book Accelerate, found that high-performing teams are both faster and more stable. It uses four key metrics (numbers you can measure):

The key idea: speed and stability are not opposites. Small, frequent, automatic changes that you can observe are safer than big, rare, manual ones. (Later DORA reports added more measures, such as reliability, and stopped using fixed "elite" levels. But these four remain the core.)