CoursesDocker for beginnersThe layered filesystem & build cache

The layered filesystem & build cache

Why instruction order makes builds fast.

Beginner12 min · lesson 10 of 16
In plain terms
Each instruction that changes the filesystem is a layer. Docker caches unchanged layers. Copy dependency files and install before copying app code.

A Docker image is a read-only stack of filesystem layers. Your app, its libraries, and the base OS files are packed once and do not change. A container is a running instance of that image; many containers can share one image. Docker does not write the image in one shot. Each build instruction that changes files adds a layer. Layers stack; the image is the combined result.

You list those steps in a Dockerfile: one instruction per line. Each instruction that changes files adds a layer. FROM sets the base image. COPY adds your files. RUN executes a command at build time and records the result as a layer. Layers are content-addressed, so two images that both start FROM node:22-alpine share that base on disk. pull means download from a registry; Docker Hub is the default public one.

Every image layer is read-only. When a container starts, Docker adds one writable layer on top. Logs, uploads, and temp files go there. Remove the container and that layer is deleted. That is why writes to the container filesystem do not survive docker rm.

An image is a stack of layers
Read it bottom to top. Each instruction adds a layer, the base is shared between images, and only the writable layer belongs to a single container.

See the layers yourself

Here is a small app you can build on your own machine. Four files in one folder: a package.json listing a single dependency (a package of ready-made code your app borrows), the package-lock.json that npm writes beside it (it pins the exact version of every package so two builds come out identical), a server.js holding your code, and the Dockerfile below. The name after -t, payments-api:1.0 here, is a tag: a plain label plus a version, so you have something to point at later.

Dockerfile
FROM node:22-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY . .
EXPOSE 3000
CMD ["node", "server.js"]

Three of those instructions are new. WORKDIR sets the folder that every later command runs inside, so you never have to spell out the full path. EXPOSE writes down which port the app listens on. CMD names the command Docker runs when a container starts. Build it now and read what scrolls past. Every line that opens with a step number is one layer being made, in the order you wrote them.

terminal
$ docker build -t payments-api:1.0 .
[+] Building 9.3s (11/11) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 132B 0.0s
=> [internal] load metadata for docker.io/library/node:22-alpine 0.8s
=> [internal] load .dockerignore 0.0s
=> [1/5] FROM docker.io/library/node:22-alpine@sha256:9fcc1a... 1.7s
=> [internal] load build context 0.0s
=> => transferring context: 1.44kB 0.0s
=> [2/5] WORKDIR /app 0.1s
=> [3/5] COPY package*.json ./ 0.0s
=> [4/5] RUN npm ci --omit=dev 5.6s
=> [5/5] COPY . . 0.0s
=> exporting to image 0.2s
=> => writing image sha256:4b1e2f... 0.0s
=> => naming to docker.io/library/payments-api:1.0 0.0s

Look at the lines marked [n/5]. Each one is a layer, built in order, starting from the base image. Step [4/5], the npm ci install, ate most of the clock. npm ci means a clean install of your dependencies read straight from that lock file, nothing improvised. That is precisely the step you want Docker to skip on every build after the first. You can also list the finished layers with their sizes:

terminal
$ docker history payments-api:1.0
IMAGE CREATED CREATED BY SIZE
4b1e2f9c8a3d 2 minutes ago CMD ["node" "server.js"] 0B
<missing> 2 minutes ago EXPOSE map[3000/tcp:{}] 0B
<missing> 2 minutes ago COPY . . # buildkit 12.3kB
<missing> 2 minutes ago RUN /bin/sh -c npm ci --omit=dev # buildkit 4.9MB
<missing> 2 minutes ago COPY package*.json ./ # buildkit 1.4kB
<missing> 2 minutes ago WORKDIR /app 0B
<missing> 3 weeks ago /bin/sh -c #(nop) CMD ["node"] 0B
<missing> 3 weeks ago /bin/sh -c #(nop) ADD file:... in / 8.4MB

Your WORKDIR, COPY and RUN steps sit near the top of that list, with the base image's own layers underneath them. The <missing> in the ID column is nothing to worry about. It only means those lower layers carry no tag of their own. Now read the sizes. EXPOSE and CMD weigh 0B because they record a setting and touch no files. npm ci is where the real weight went.

Change one line, rebuild, watch the cache

Docker hangs on to every layer it has ever built. On the next build it walks down your Dockerfile in order and reuses each stored layer for as long as that instruction's inputs are unchanged. The first instruction whose inputs did change gets rebuilt, and so does every layer sitting above it, because each layer is stacked on the one below. So edit a single line of server.js and build again:

terminal
$ docker build -t payments-api:1.1 .
[+] Building 0.6s (11/11) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load metadata for docker.io/library/node:22-alpine 0.4s
=> [internal] load .dockerignore 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 1.44kB 0.0s
=> CACHED [2/5] WORKDIR /app 0.0s
=> CACHED [3/5] COPY package*.json ./ 0.0s
=> CACHED [4/5] RUN npm ci --omit=dev 0.0s
=> [5/5] COPY . . 0.0s
=> exporting to image 0.1s
=> => naming to docker.io/library/payments-api:1.1 0.0s

Steps 2–4 show CACHED. Only [5/5] COPY . . ran, because that step saw the edited file. The rebuild finished in well under a second; npm ci did not run again. That is why you copy package*.json and install before copying the rest of the source: the install layer sits under the code layer, so a code edit does not invalidate it.

Now turn the order upside down and watch it fall over. This version copies everything first, then installs:

Dockerfile
FROM node:22-alpine
WORKDIR /app
COPY . .
RUN npm ci --omit=dev
EXPOSE 3000
CMD ["node", "server.js"]

Change one line of source and rebuild with this one. COPY . . now sits below the install. Touch any file at all and that lower layer changes, which drags npm ci and everything stacked above it into the rebuild:

terminal
$ docker build -t payments-api:bad .
[+] Building 5.9s (10/10) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> [internal] load metadata for docker.io/library/node:22-alpine 0.4s
=> [internal] load .dockerignore 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 1.44kB 0.0s
=> CACHED [2/4] WORKDIR /app 0.0s
=> [3/4] COPY . . 0.0s
=> [4/4] RUN npm ci --omit=dev 5.4s
=> exporting to image 0.3s
=> => naming to docker.io/library/payments-api:bad 0.0s

Step [4/4] npm ci ran again. That is 5.4 seconds burned on a one-line edit to a file with nothing to do with your dependencies. Scale it up to a real project carrying a few hundred packages and you are choosing between a one-second rebuild and a two-minute one, every single time you fix a typo. Same app, same commands, same finished image. Two instructions swapped places.

Rebuilt the image, now run it

Rebuilding an image leaves anything already running completely alone. The old container keeps serving, and it keeps its grip on whatever ports it claimed at startup. Try to start the new image on the same port and Docker refuses:

terminal
$ docker run -d -p 3000:3000 payments-api:1.0
c0ffeecafe12ab34cd56ef78901234567890abcdef1234567890abcdef123456
$ docker run -d -p 3000:3000 payments-api:1.1
docker: Error response from daemon: driver failed programming external
connectivity on endpoint zealous_khorana (7f3c...): Bind for
0.0.0.0:3000 failed: port is already allocated.

That message comes from the Docker daemon, the background program that does the actual work of building images and running containers. The docker commands you type are orders handed over to it. Read the last line: port is already allocated. A port is a numbered doorway on your machine for network traffic, and one program at a time gets a given doorway. The -p 3000:3000 flag wires port 3000 on your machine through to the container. The container running :1.0 still owns door 3000, so the new one has nowhere to land. Find it, remove it, then start the replacement:

terminal
$ docker ps --format '{{.ID}} {{.Image}} {{.Ports}}'
c0ffeecafe12 payments-api:1.0 0.0.0.0:3000->3000/tcp
$ docker rm -f c0ffeecafe12
c0ffeecafe12
$ docker run -d -p 3000:3000 payments-api:1.1
9a8b7c6d5e4f01234567890abcdef1234567890abcdef1234567890abcdef1234
A deleted file still ships inside the image
Every layer is kept, so a file you add in one instruction and delete a few lines later is still sitting there in the earlier layer, and anyone holding the image can dig it back out. Copy in a password or an access key (a string of characters that proves who you are to another service), delete it further down, and you have hidden nothing at all. Keep secrets out of a Dockerfile in the first place. A secret in any layer is a secret in the whole image.

Three habits follow from the cache: order, trust, and size.

Order. Copy the dependency files, install them, then copy the rest of your source. Change your app code and Docker hands the cached install straight back to you. Change package.json and it correctly throws that layer away and reinstalls, which is what you want, because your dependencies genuinely did change. Put COPY . . near the top and every typo you fix rebuilds the universe.

Trust. A cached layer is fast, not fresh. If a RUN step downloads whatever counts as "latest" out on the internet, the cache will happily serve you a copy from three weeks ago and never mention it. Pin versions so the answer cannot drift under you. When you truly need Docker to forget everything and start clean, docker build --no-cache is the honest hammer for it.

Size. Deleting a file in a later layer does not claw the bytes back out of the earlier one, so they ride along in every copy of the image anybody downloads. Multi-stage builds, which come up in a later course, fix that class of problem properly. For now, keep junk out of your early layers and keep .dockerignore honest, so node_modules and .git never enter the build context at all.

Try this

Build once, make a tiny edit to a source file, build again, and read which steps come back CACHED. Thirty seconds of watching that teaches the layer cache better than any explanation.

terminal
docker build -t layerdemo:1 .
# edit a source file, then:
docker build -t layerdemo:1 .
docker history layerdemo:1 | head -n 8
output
[+] Building ...
=> CACHED [2/5] WORKDIR /app
=> CACHED [3/5] COPY package*.json ./
=> CACHED [4/5] RUN npm ci
=> [5/5] COPY . .
IMAGE CREATED BY SIZE
layerdemo:1 CMD ["node" "server.js"] 0B
<missing> COPY . . 12kB
<missing> RUN npm ci 80MB

Takeaway

Each instruction is a layer, and Docker rebuilds from the first changed layer upward. Install dependencies before copying app code so a one-line source edit stays a short rebuild instead of a full npm ci.

Quick check
01You change one line in server.js and rebuild. Docker prints CACHED next to WORKDIR, COPY package*.json and RUN npm ci, then rebuilds COPY . .. Why did npm ci get skipped?
Correct — Docker reuses cached layers until it meets the first changed instruction, then rebuilds that layer and everything on top of it. The npm ci layer sits below COPY . ., so a source edit never reaches down that far.
Incorrect — No. A RUN step rebuilds the moment any instruction before it changes. This one survived only because nothing below it moved.
Incorrect — No. RUN happens at build time and is baked into a read-only image layer. The writable layer only shows up once a container is running.
Incorrect — No. The base image has never heard of your packages. The saving comes entirely from the order of your own COPY and RUN lines.
02A running container writes a new file to disk. Where does that file actually go, and what happens to it when you delete the container?
Incorrect — Image layers are read-only and settled at build time. A running container cannot add one.
Incorrect — The base is read-only and shared. A container's writes never reach it.
Correct — That top layer belongs to a single container, so its contents go when the container goes.
Incorrect — Writes at runtime never travel back into your Dockerfile or into any future build.
03A teammate's Dockerfile runs COPY secrets.txt ., uses the file in a RUN step, then runs RUN rm secrets.txt a few lines further down. They tell you the secret is safe because the finished image no longer has it. Are they right?
Incorrect — The rm only stacks one more layer on top. The layer that brought the file in still holds it.
Correct — Every layer is kept, so a secret in any layer is a secret in the whole image.
Incorrect — Same build or not, the layer that added the file is retained and can be extracted.
Incorrect — The file is recoverable from the image itself, private registry or not.

Related