Slimming images and layer hygiene

Find what makes an image big or leaky, and gate it in CI.

docker history of a 207 MB image shows a 20.5kB layer for rm -rf /var/lib/apt/lists/*, right above a 21.5MB layer for apt-get update. The cleanup step added bytes instead of removing them. And the package lists still ship to every host that pulls the image. Big images are slow to pull, slow to scan and carry more to patch. Leaky images carry files nobody meant to publish. Both problems come from the same few habits, and you can measure both and gate them in CI. You'll do that here with docker history, dive, Trivy and a small size budget.

Here's a short recap of the mechanics, which "Layers and the build cache" in Docker for beginners teaches in full. Each RUN, COPY and ADD adds a read-only layer. If you delete a file in a later layer, that only records a whiteout that hides it. The bytes stay in the layer that added them. Use the main lab VM; the lesson files go in ~/lab/slimming.

Find the big layers

Dockerfile.bloat installs curl the way many Dockerfiles still do, one command per instruction:

FROM debian:trixie-slim
RUN apt-get update
RUN apt-get install -y curl
RUN rm -rf /var/lib/apt/lists/*

Read the history from the bottom up. The Debian base is 110MB. apt-get update downloaded 21.5MB of package lists into its own layer, and apt-get install added 22.7MB. The rm layer is 20.5kB: the whiteout entries plus a little metadata. The lists are hidden in the final filesystem, but they're still in the image. --no-cache forces apt to run here. Without it, a rebuild would reuse the cached layers.

Dockerfile.lean does the same job in one instruction and skips recommended packages. It adds back the one recommendation that matters for curl, the CA certificates:

FROM debian:trixie-slim
RUN apt-get update \
 && apt-get install -y --no-install-recommends curl ca-certificates \
 && rm -rf /var/lib/apt/lists/*

You get one layer of 18.8MB instead of 44MB. The lists are deleted before the layer is committed, and --no-install-recommends leaves out packages curl doesn't need. DISK USAGE drops from 207MB to 165MB. CONTENT SIZE, roughly what a pull transfers, drops from 53.5MB to 36.4MB. Your numbers will differ with the current Debian package versions. The same rule holds for every package manager: apk add --no-cache, pip install --no-cache-dir, npm ci followed by npm cache clean --force in the same RUN. A cache mount goes further (RUN --mount=type=cache, covered in "BuildKit builds: stages, cache and mounts" in Docker in depth). The download cache never enters a layer, and it still speeds up the next build.

--squash is not the answer

Older guides fix the bloated file by flattening it with docker build --squash. On a default Docker 29 install the flag's still accepted, and ignored:

BuildKit prints a warning and builds the same four layers. Squashing belonged to the legacy builder with an experimental daemon. The warning names the replacement: a multi-stage build whose final stage copies only the result. That way, nothing you've deleted in an earlier stage is ever part of the image.

Score an image with dive

dive reads an image layer by layer. It reports files that were added and then replaced or deleted later: the bytes that ship without being visible. In CI mode it applies thresholds and exits non-zero on failure. It normally talks to the Docker daemon through the socket. That would give the dive container root-equivalent access to the host ("The Docker socket and daemon hardening"). Reading a docker save archive with --source docker-archive needs no socket. The image is pinned by digest and pulled through mirror.gcr.io:

The colour codes dive prints are stripped with sed for this page. The last line prints dive's exit status, 1 here. In a CI job that status is the gate, so end the step with it instead of printing it. dive found 25MB of waste. That's the package lists (written in one layer, hidden by the next), plus debconf and dpkg files that the install rewrote. Efficiency 83.5% fails the 95% floor, and the 61% user-wasted ratio fails the default limit of 10%. Now the lean image:

Efficiency is 98.4%, and it still fails the default highestUserWastedPercent of 10%. The files listed are dpkg's status database and debconf's caches. They exist in the Debian base, and any apt-get install rewrites them, so the old copies in the base layer count as waste. Every image that installs a package on a Debian base hits this. Set the threshold from what your own images really look like, here 25%. Keep the efficiency floor, since that's what catches lists and caches left in layers.

What the build context brings in

Size is the half you can see. The other half is files that reach a layer by accident, and COPY . . is the usual way in. The app directory holds a five-line Dockerfile and a one-line app.py. Give it the junk a real working copy has: a .env with a (fake) token and a .git directory:

FROM python:3.15-slim
WORKDIR /app
COPY . .
USER 10001
CMD ["python", "app.py"]

Everything in the directory is in /app, including the token file and 2MB of git objects. The warning on the first line comes from Buildx. It tries to record the current commit in the image provenance when the context has a .git directory, and this one isn't a real repository. A secret scanner run on the saved image finds the token. The Trivy image is pinned by digest like dive. In March 2026 attackers published malicious Trivy images and force-pushed the trivy-action tags (CVE-2026-33634). References by digest weren't affected. TRIVY holds the pinned reference in each step:

Trivy's secret scanner matched the GitHub token format, masked the value in its report and named the instruction that added the file. --exit-code 1 makes Trivy exit 1 on any finding, as the last line shows. In CI that fails the step. Trivy reads the archive from docker save, so it doesn't need the Docker socket either. Now add a .dockerignore (its basics are in "Writing a Dockerfile" in Docker for beginners):

.git
.env
*.pem
Dockerfile
.dockerignore

Only app.py reaches the image, and the scan exits 0. A .dockerignore limits what the context sends. Copying named files (COPY app.py ./) limits what a layer receives, and it's the stronger of the two. A scanner is still worth your time, because it checks the result, including layers the final filesystem hides:

FROM alpine:3.24
COPY app/.env /tmp/.env
RUN rm /tmp/.env

The file was deleted one step after it was copied. Trivy still reports it, attributed to the COPY layer, because that layer ships. Credentials you need during a build belong in a secret mount, never in the context. "Build-time secrets and how images leak them" measures every place they can otherwise end up. If a scan like this finds a real credential in an image that was pushed, rotate it first.

A size budget

dive measures waste, not size. A budget catches the other regression: a new dependency, a base image that grew, a debug tool someone left in. check-size.sh compares the image's disk usage with a limit you pass. It reads the .Size field of docker image inspect, in bytes:

#!/bin/sh
# check-size.sh IMAGE MAX_MB: fail when the image's unpacked size is over budget
img=$1 max=$2
bytes=$(docker image inspect --format '{{.Size}}' "$img") || exit 2
mb=$((bytes / 1000000))
if [ "$mb" -gt "$max" ]; then
  echo "FAIL $img: $mb MB, budget $max MB"
  exit 1
fi
echo "ok   $img: $mb MB, budget $max MB"

Set the budget a little above the current size. Raise it on purpose when a change justifies it, in the same review. Compressed size is the better number when pull time is what you care about. It's the CONTENT SIZE column of docker image ls, or the sum of layer sizes in the pushed manifest.

Deterministic dependencies

Gates only mean something when two builds of the same commit produce the same contents. Install from lockfiles (npm ci with package-lock.json, pip install --require-hashes -r requirements.txt, go.sum). Pin the base image by digest as described in "Tags, digests and promotion" in Docker in depth. And pin package versions where an unexpected upgrade would hurt. Debian's snapshot archive makes an apt-get install repeatable when that matters. BuildKit can also normalise file timestamps from SOURCE_DATE_EPOCH, which removes one more source of differences between rebuilds. In a pipeline, run the secret scan, the waste check and the budget against the built image, before the push. Then a failing image never reaches a registry. "Docker in CI/CD: build to promote" in Docker in depth puts them in order with tests, signing and promotion.

Clean up

The dive and Trivy images stay in the image store. Remove them with docker image rm when you don't need them any more.

Try this

Before the clean-up step, on the main lab VM in ~/lab/slimming, tighten the budget: run ./check-size.sh lab-slim:lean 160 and ./check-size.sh lab-slim:lean 170. Predict which of the two passes from the 164 MB the budget step printed for the lean image, then check. Pick the number you'd actually commit as the budget, and note why a value right at the current size would fail the next small package bump.

Takeaway

A delete in a later layer hides a file without removing it. The 20.5kB rm layer sat on top of 21.5MB of package lists, and Trivy still found the token deleted one step after its COPY. Clean up in the same RUN that made the mess, copy named files instead of COPY . ., and scan the saved archive, since that's what you ship. A common review slip is trusting --squash, which Docker 29 accepts and ignores.

Related