Docker Compose: Running a Whole App Locally

In lesson 2 you typed long docker run commands with --network, -v, -e, -p. For one container that is fine. For three containers that need to start in the right order, share a network and keep their data, it is a mess that nobody can remember. Compose fixes this with a single file you keep in Git. The file is the documentation, and docker compose up is the whole setup guide for a new teammate.

In this lesson we build a real stack: the notes-api from lesson 1, now backed by Postgres (to store notes) and Redis (to count hits). I ran it, broke it on purpose, and fixed it. You will see the start-up race that happens when you do not wait for the database, and the hits counter that resets while the notes survive. That small accident teaches volumes better than any explanation.

This is lesson 3 of 10. It builds on Writing a Dockerfile and Volumes, Networks and Running Containers Safely. The next lesson, Kubernetes Core Ideas, takes the same app to many machines. In lesson 5 you will see the Kubernetes version of what Compose does with names: Services and DNS.

Words used in this lesson

• Compose file is the YAML file (compose.yaml) that describes your services, networks and volumes.
• Service is one kind of container in your app, such as api or db. A service can run as one or more containers.
• Project is one running copy of a Compose app. Its name prefixes the containers, networks and volumes (for example notes_pgdata).
• Health check is a small command Docker runs again and again inside a container to ask "are you ready?".
• depends_on is the setting that says one service needs another to start first.
• Interpolation is the use of ${VAR} in a Compose file, replaced by a value from your shell or an .env file.
• Override file is a second Compose file (often compose.override.yaml) merged on top of the first, usually for development.
• Profile is a label that makes a service start only when you ask for it.
• Restart policy is a rule for what Docker does when a container exits, such as unless-stopped.
• Compose Specification is the open standard that defines the file format. It is separate from any one tool.

What problem does Compose solve?

Compare the two ways to start our stack.

Without Compose you would run, in the right order, something like:

docker network create notesdocker volume create pgdatadocker run -d --name db --network notes -e POSTGRES_USER=notes -e POSTGRES_PASSWORD=devonly \  -e POSTGRES_DB=notes -v pgdata:/var/lib/postgresql/data postgres:16-alpinedocker run -d --name cache --network notes redis:7-alpinedocker build -t notes-api .docker run -d --name api --network notes -p 127.0.0.1:3000:3000 \  -e DATABASE_URL=postgres://notes:devonly@db:5432/notes -e REDIS_URL=redis://cache:6379 notes-api

With Compose you run one line:

docker compose up -d

Compose reads the file, creates the network and volumes, builds the image if asked, starts the containers in the right order, and names everything with the project name. docker compose down takes it all down again.

The app, now with a database

I changed notes-api so it uses Postgres and Redis. The changes are small. It creates the table on start, saves notes with POST /notes, reads them with GET /notes, and increments a counter in Redis on every read.

// src/server.ts (the new version)import express from "express";import pg from "pg";import { createClient } from "redis";const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL });const cache = createClient({ url: process.env.REDIS_URL });cache.on("error", (err) => console.error("redis error", err.message));const app = express();app.use(express.json());app.get("/health", async (_req, res) => {  await pool.query("select 1");  await cache.ping();  res.json({ ok: true });});app.get("/notes", async (_req, res) => {  const hits = await cache.incr("hits");  const { rows } = await pool.query("select id, text from notes order by id");  res.json({ hits, notes: rows });});app.post("/notes", async (req, res) => {  const { rows } = await pool.query(    "insert into notes (text) values ($1) returning id, text",    [String(req.body.text ?? "")],  );  res.status(201).json(rows[0]);});await pool.query(  "create table if not exists notes (id serial primary key, text text not null)",);await cache.connect();const port = Number(process.env.PORT ?? 3000);const server = app.listen(port, "0.0.0.0", () =>  console.log(`listening on ${port}`),);process.on("SIGTERM", () => {  console.log("got SIGTERM, closing");  server.close(() => process.exit(0));});

Note the $1 in the SQL: that is a query parameter, which keeps user text out of the SQL string (the lesson from the Security series on injection). The database address and the Redis address both come from environment variables. In Compose we will set them to db and cache, which are just the service names.

The Dockerfile is the multi-stage one from lesson 1, with one extra line at the top so the base image can be changed:

# syntax=docker/dockerfile:1ARG NODE_IMAGE=node:22-alpineFROM ${NODE_IMAGE} AS buildWORKDIR /appCOPY package.json package-lock.json ./RUN npm ciCOPY tsconfig.json ./COPY src ./srcRUN npm run buildFROM ${NODE_IMAGE} AS prod-depsWORKDIR /appENV NODE_ENV=productionCOPY package.json package-lock.json ./RUN npm ci --omit=devFROM ${NODE_IMAGE}ENV NODE_ENV=productionWORKDIR /appCOPY --from=prod-deps /app/node_modules ./node_modulesCOPY --from=build /app/dist ./distCOPY package.json ./USER nodeEXPOSE 3000CMD ["node", "dist/server.js"]

(An ARG that comes before the first FROM can be used in the FROM lines. That is the only place it can be used before a stage starts.) hadolint reported the same single info message as before about USER node.

The compose file, piece by piece

Here is the whole compose.yaml. I will explain each part below.

name: notesservices:  api:    build:      context: .      args:        NODE_IMAGE: ${NODE_IMAGE:-node:22-alpine}    ports:      - "127.0.0.1:3000:3000"    environment:      DATABASE_URL: postgres://notes:devonly@db:5432/notes      REDIS_URL: redis://cache:6379    depends_on:      db:        condition: service_healthy      cache:        condition: service_healthy    restart: unless-stopped  db:    image: postgres:16-alpine    environment:      POSTGRES_USER: notes      POSTGRES_PASSWORD: devonly      POSTGRES_DB: notes    volumes:      - pgdata:/var/lib/postgresql/data    healthcheck:      test: ["CMD-SHELL", "pg_isready -U notes -d notes"]      interval: 3s      timeout: 3s      retries: 10  cache:    image: redis:7-alpine    healthcheck:      test: ["CMD", "redis-cli", "ping"]      interval: 3s      timeout: 3s      retries: 10volumes:  pgdata:

YAML is picky about spaces (never tabs), so first a quick check that the file parses. I used docker compose config, which reads the file, checks it against the Compose schema, and prints the full, normalised result:

$ docker compose configname: notesservices:  api:    restart: unless-stopped    environment:      DATABASE_URL: postgres://notes:devonly@db:5432/notes      REDIS_URL: redis://cache:6379    networks:      default: null    build:      context: /tmp/lab-dk/compose-app      dockerfile: Dockerfile      args:        NODE_IMAGE: node:22-alpine    depends_on:      cache:        condition: service_healthy        required: true      db:        condition: service_healthy        required: true    ports:      - mode: ingress        host_ip: 127.0.0.1        target: 3000        published: "3000"        protocol: tcp...networks:  default:    name: notes_defaultvolumes:  pgdata:    name: notes_pgdata

(Shortened.) Several things appear that I did not write: a default network, long forms of ports, and required: true. Compose filled in defaults. docker compose config is the first command to run when something looks wrong, because it shows what Compose actually understood.

Now the pieces.

name: notes sets the project name. Without it, Compose uses the folder name. Every resource gets this prefix: containers notes-api-1, network notes_default, volume notes_pgdata. The project name keeps two copies of the same app from clashing. (An old top-level version: field is obsolete. The specification says it is only informative and Compose warns if you use it, so leave it out.)

services: is the map of containers. The key (api, db, cache) is the service name, and also its DNS name on the network. This is the idea from lesson 2 made automatic: Compose creates one user-defined network for the project and puts all services on it, so db and cache resolve as host names. I checked from inside the api container:

$ docker compose exec api getent hosts db cache172.19.0.2        db  db172.19.0.3        cache  cache

build: or image:. api has build, which tells Compose to build from the Dockerfile in this folder. db and cache use image:, which pulls a ready-made image. The args line passes NODE_IMAGE into the build. The syntax ${NODE_IMAGE:-node:22-alpine} means "use the shell variable NODE_IMAGE, or this default if it is not set". Defaults like :- and required variables like ${VAR:?message} are part of the Compose interpolation rules. I tested both:

# unset variable with ${MISSING:?need MISSING}error while interpolating services.web.environment.A: required variable MISSING is missing a value: need MISSING# unset variable with plain ${MISSING}The "MISSING" variable is not set. Defaulting to a blank string.

The first stops you. The second only warns and uses an empty string, which can hide a bug. For important values use the :? form.

You can also put values in a .env file next to compose.yaml, or pass --env-file. I tested the default and an override:

PW: default          # no env filePW: fromdotenv       # with --env-file .env.test containing POSTGRES_PASSWORD=fromdotenv

ports: is "HOST:CONTAINER". I wrote 127.0.0.1:3000:3000 so only this machine can reach the API (lesson 2). Write ports as quoted strings. The spec defines the short form as a colon-separated string, and older YAML rules can read an unquoted 22:22-style value as a base-60 number, which is a known trap. (I did not trigger that trap myself, but quoting costs nothing.)

environment: sets variables. These point at db and cache. The password devonly is for local development only. Real secrets do not belong in a Compose file in Git (use an .env file that is git-ignored, or Compose secrets).

volumes: has two places. Under db it mounts the named volume pgdata at Postgres's data folder, as in lesson 2. At the bottom, volumes: pgdata: declares the named volume so Compose creates and manages it.

restart: unless-stopped tells Docker to restart the container if it exits, unless you stopped it by hand. The spec's other values are no (default), always, and on-failure. I tested it:

$ docker compose exec api sh -c 'kill 1'      # send SIGTERM; app exits cleanly$ docker compose ps api ; docker inspect notes-api-1 -f 'restarts={{.RestartCount}}'Up 4 secondsrestarts=1$ docker compose logs api | tail -3api-1  | listening on 3000api-1  | got SIGTERM, closingapi-1  | listening on 3000$ docker compose kill api ; docker compose ps -a apiExited (137) 4 seconds ago

Docker brought the API back after it exited by itself (restart count 1). But after docker compose kill (a stop by a human command) the container stayed down. That is the "unless stopped" part.

Starting up for real

docker compose up -d --build

Compose output (trimmed, real):

 Image notes-api Built Volume notes_pgdata Created Network notes_default Created Container notes-cache-1 Created Container notes-db-1 Created Container notes-api-1 Created Container notes-cache-1 Started Container notes-db-1 Started Container notes-cache-1 Waiting Container notes-db-1 Waiting Container notes-cache-1 Healthy Container notes-db-1 Healthy Container notes-api-1 Starting Container notes-api-1 Started

Read the order. The cache and the database started first. Compose then waited until both were Healthy, and only then started the API. That wait is the condition: service_healthy working. Then:

$ docker compose psNAME            IMAGE                COMMAND                  SERVICE   STATUS                   PORTSnotes-api-1     notes-api            "docker-entrypoint.s…"   api       Up Less than a second    127.0.0.1:3000->3000/tcpnotes-cache-1   redis:7-alpine       "docker-entrypoint.s…"   cache     Up 5 seconds (healthy)   6379/tcpnotes-db-1      postgres:16-alpine   "docker-entrypoint.s…"   db        Up 4 seconds (healthy)   5432/tcp

Notice that only the API shows a published port. Postgres and Redis are on the project network and reachable by the API by name, but not from your laptop. That is the safe default from lesson 2. (If you want to use psql from your host, add ports: ["127.0.0.1:5432:5432"] to db, or use docker compose exec db psql.)

Let's use the app:

$ curl -s -XPOST -H 'content-type: application/json' -d '{"text":"first note"}' localhost:3000/notes{"id":1,"text":"first note"}$ curl -s localhost:3000/notes{"hits":1,"notes":[{"id":1,"text":"first note"}]}$ curl -s localhost:3000/notes{"hits":2,"notes":[{"id":1,"text":"first note"}]}

The note went into Postgres. The hits counter lives in Redis and goes up by one on every read.

A full up --wait from nothing (with the images and build cached) took about 5 seconds on my machine. --wait makes the command return only when every service is running or healthy.

Health checks and depends_on: the start-up race

Here is the most common Compose problem. A container being "started" does not mean the program inside is "ready". The Docker docs say it plainly: on startup Compose waits only until a container is running, not until it is ready. Postgres takes a couple of seconds to initialise, and during that time connections are refused.

To show the problem, I wrote a second file that uses the short form of depends_on (just a list of names) and has no restart policy:

    depends_on:      - db      - cache

Then I ran it and looked a few seconds later:

$ docker compose -f compose.naive.yaml up -d Container notes-cache-1 Started Container notes-db-1 Started Container notes-api-1 Started$ docker compose -f compose.naive.yaml ps -aapi     Exited (1) 5 seconds agocache   Up 6 seconds (healthy)db      Up 6 seconds (health: starting)$ docker compose -f compose.naive.yaml logs apiapi-1  | Error: connect ECONNREFUSED 172.19.0.3:5432api-1  |     at /app/node_modules/pg-pool/index.js:45:11...

All three containers started in the right order, but the API came up while the database was still saying health: starting. The app tried to connect, got ECONNREFUSED, crashed on the unhandled error and exited with code 1. Short depends_on only controls the order in which containers are created and started.

The fix has two parts:

1. A health check on the dependency. It is a command Docker runs inside the container on a timer:

    healthcheck:      test: ["CMD-SHELL", "pg_isready -U notes -d notes"]      interval: 3s      timeout: 3s      retries: 10

pg_isready is a tool that ships with Postgres. It exits 0 when the server accepts connections. Docker runs it every 3 seconds, and marks the container healthy after it succeeds. After 10 failures in a row it is unhealthy. Redis uses redis-cli ping. The test can be ["CMD", ...] (run the program directly) or ["CMD-SHELL", "..."] (run through a shell, handy for $VARs and pipes).

2. A condition in depends_on:

    depends_on:      db:        condition: service_healthy

Now Compose will not start api until db is healthy. The three conditions from the spec are service_started (the default, and what the short form means), service_healthy, and service_completed_successfully (for a one-time job such as a database migration that must finish first). The spec also has restart: true inside a dependency, which restarts the dependent service when you restart the dependency through Compose.

I tested the "later" case on my own app. With everything running, I stopped only the database:

$ docker compose stop db$ curl -s -m 5 localhost:3000/health(no output)$ docker compose ps --format '{{.Service}} {{.Status}}'api Restarting (1) Less than a second agocache Up 9 seconds (healthy)

The /health request got no answer. The API had crashed, because my async route handler threw an error with nobody to catch it (Express 4 does not catch errors from async handlers by itself, so Node's default is to exit). The restart policy kept restarting it, and each attempt failed until the database came back. This is a bug in my app, not in Compose, and it is a real one: catch errors in handlers and return a 503, and retry connections. Compose cannot fix that for you.

Data and the lifecycle: down versus down -v

Here is the accident I promised. After adding a note, I ran:

$ docker compose down Container notes-db-1 Removed Container notes-cache-1 Removed Network notes_default Removed$ docker compose up -d$ curl -s localhost:3000/notes{"hits":1,"notes":[{"id":1,"text":"first note"}]}

down removed the containers and the network, but not the volume. The note came back from Postgres. But hits is 1 again: Redis had no volume, so its counter was lost with the container.

Which is right depends on what you want. For a cache, losing data on restart is fine. If the counter mattered, I would give Redis a volume (and turn on its persistence). The lesson: only things you gave a volume survive down.

To remove the volume too (a full reset):

docker compose down -v       # also removes named volumes declared in the file

I ran it and the output ended with Volume notes_pgdata Removed. After down -v, the next up starts with an empty database. This is the Compose version of "turn it off and on again", and it is what you use when your local data is in a bad state. It is also why you must be careful: down -v on the wrong project deletes real data.

Useful commands

The run command is handy. For example, I ran a one-off container for the api service, which has the right environment and network:

$ docker compose run --rm --no-deps api node -e 'console.log(process.version, process.env.REDIS_URL)'v22.23.3 redis://cache:6379

It printed the Node version and the REDIS_URL that Compose had set. --no-deps skips starting the other services.

When the file is wrong: real error messages

Compose checks your file against the specification. Here are three mistakes and the real messages:

# "port:" instead of "ports:"validating bad1.yaml: services.web.port false schema# depends_on refers to a service that does not existservice "web" depends on undefined service "ghost": invalid compose project# YAML with a tab character (not Compose, the YAML parser)found character '\t' that cannot start any token

The first message is not friendly: false schema is the JSON Schema way of saying "this key is not allowed here". It means a typo in a key name. This is the Compose version of the strict-mode trap from lesson 5: an unknown key is an error. Run docker compose config in CI so these are caught before they reach a teammate.

Development habits: override files, bind mounts and profiles

An override file for development

Compose reads compose.yaml and, if it exists, compose.override.yaml, and merges them. Put what is different in development into the override. Keep compose.yaml close to production. I wrote an override that runs the TypeScript source directly with Node's --watch flag and mounts ./src from my laptop:

# compose.override.yamlservices:  api:    build:      target: build    command: ["node", "--watch", "--experimental-strip-types", "src/server.ts"]    environment:      NODE_ENV: development    volumes:      - ./src:/app/src

target: build builds only up to the build stage of the Dockerfile (which has all the dev dependencies). command replaces the container's command. The bind mount shows my local src folder inside the container. The merged result (from docker compose config) has the new command, the extra environment value and the bind mount, with all the other settings from the base file kept. Then I edited src/server.ts on the host, and the logs showed the container restarting by itself:

api-1  | listening on 3000api-1  | Restarting 'src/server.ts'api-1  | got SIGTERM, closingapi-1  | listening on 3000

I changed the query's sort order, and the restart in the log appeared about a second after I saved, with no rebuild. That is the "edit, save, see it" loop. (Node 22 can strip TypeScript types by itself, which is why --experimental-strip-types works for this simple file. That flag is only for development.)

One thing bit me while testing. After running with the override, I ran docker compose up -d --wait without --build, and the API started restarting in a loop. The image notes-api was still the one built with target: build, which has no production CMD, so the container exited at once. Lesson: after you change build: settings, add --build.

For other files, -f lets you choose: docker compose -f compose.yaml -f compose.prod.yaml up. The merge rule from my test: single values (like command) are replaced, maps (like environment) are merged key by key, and lists of volumes are combined.

$ docker compose -f base.yaml -f override.yaml config    environment:      DEBUG: "1"      MODE: dev          # base said "prod", the override won

Profiles for optional services

Some services you only want sometimes: an admin tool, a mail catcher, a test runner. Give them a profiles label, and they start only when requested:

$ docker compose -f prof.yaml config --servicesweb$ docker compose -f prof.yaml --profile tools config --servicesadminerweb

Without the flag only web is part of the project. With --profile tools, adminer joins. The spec says that a service without profiles always starts, and a service with profiles starts only when one of its profiles is active.

Other habits worth having

• Put one project per folder, and use the name: field.
• Keep secrets out of the file. Use .env (in .gitignore) or Compose secrets.
• Pin image versions (postgres:16-alpine, not postgres). A major version change of a database can refuse to start on old data.
• Do not publish database ports to everyone. Leave them off or bind to 127.0.0.1.
• Run docker compose config in CI and hadolint on the Dockerfile.
• Use Compose for local development and small single-host setups. For real production you probably want an orchestrator like Kubernetes (next lesson), because Compose runs on one machine and cannot move containers when that machine dies.

Try it

1. A Compose file has services web and db. Inside web, what host name connects to the database? Do you need to create a network?
2. You run docker compose down and up again. Which data survives, and which does not, if only db has a volume?
3. Your API crashes at start-up with ECONNREFUSED for the database even though depends_on: [db] is set. Why? Write the fix.
4. What is the difference between docker compose down and docker compose down -v?
5. You edited the Dockerfile but docker compose up -d still runs the old image. What do you do?
6. You want an adminer service that only starts when you ask. Which key do you use?
7. What is wrong with ports: - 5432:5432 on a database in a shared office network?

Answers

1. The service name, db. No, Compose creates a default network for the project, and the service names resolve on it.
2. The data on the db volume survives. Anything else written inside containers (for example a Redis counter without a volume) is lost, because down removes the containers.
3. Short depends_on only orders the container start, and does not wait for readiness. Add a healthcheck to db (for example pg_isready) and use depends_on: db: condition: service_healthy. Also add connection retries to the app.
4. down removes containers and networks and keeps named volumes. down -v also removes the named volumes, so the data is deleted.
5. Run docker compose up -d --build (or docker compose build first). Without --build Compose reuses the image it already has.
6. profiles: ["tools"] under the service, and start it with --profile tools.
7. It binds to all interfaces (0.0.0.0), so other people on the network can try to connect to your database. Use 127.0.0.1:5432:5432, or do not publish it and use the shared network.

Interview questions

Key takeaways

• Compose describes a multi-container app in one YAML file. docker compose up -d starts it, down removes it.
• Service names are DNS names on the project network. No -p is needed for container-to-container traffic.
• A started container is not a ready one. Add a healthcheck and use depends_on with condition: service_healthy. Short depends_on only sets the start order.
• Even with that, apps should retry connections. In my test an unhandled error crashed the API when the database stopped.
• down keeps volumes and down -v deletes them. Only data on a volume survives. My Redis counter reset but the Postgres note stayed.
• Use compose.override.yaml for development, a bind mount plus a watch command for fast edits, and profiles for optional services.
• Run docker compose config to see what Compose really understood, and quote your port strings.
• Compose is for one machine. For many machines, use an orchestrator, which is the next lesson.

What's next

Next: Kubernetes Core Ideas: Cluster, Pods and Deployments, Lesson 4. Compose runs on one machine and does not help when that machine dies. Kubernetes keeps your containers running across many machines. You will meet the control plane, nodes, pods and Deployments, and I will show you what a real Kubernetes API server, scheduler and controller do when you delete a pod. Practise now: write a compose.yaml for an app of your own with a database and a health check, and try stopping the database to see how your app behaves.