How Layers and the Build Cache Really Work

A Dockerfile is not a script that produces a filesystem — it is a stack of immutable layers, and the build cache is the difference between a 30-second rebuild and a 15-minute one. If your CI takes twenty minutes to build an app image, the fix is almost never docker build --no-cache. It is ordering.

What a layer actually is

Every FROM, RUN, COPY, and ADD produces a layer: a diff of the filesystem applied on top of the previous one. The final image is the union of those diffs, which is why docker history myapp:latest lists each instruction with its size. Deleting a file in a later instruction shrinks nothing — it adds a layer that hides the file while every earlier layer still ships it. Hold that thought; it matters again when we get to secrets.

How the cache decides

The cache is keyed on two things: the exact instruction text, and — for COPY/ADD — checksums of the copied files. Change a byte in any copied file and that COPY misses; every instruction after it runs again, cached or not. So the ordering rule: least-frequently-changing things first, most-frequently-changing last. For a Node app that means dependencies before source:

FROM node:22-bookworm-slim
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY . .
CMD ["node", "server.js"]

With this order, a code-only change hits cache on the first three steps and rebuilds only the final COPY. The naive version — COPY . . first — invalidates the npm ci layer on every commit and reinstalls node_modules each build. That is the bug that ships.

The other half: the build context

COPY checksums cover everything sent to the daemon, so a stray change in your context (.git, a log file, a downloaded artifact) can bust the cache you just carefully ordered. .dockerignore is your firewall:

.git
node_modules
*.log
.env
coverage
dist

It also keeps COPY . . from shipping your .git directory and .env into the image — a leak that has ended careers. BuildKit (the default builder since Docker 23.0) prints [CACHED] per step when you build with --progress=plain; get in the habit of reading it.

Try this

Take any working Dockerfile and build it twice: once as written, once with dependency files copied before the rest of the source. Touch one source file, rebuild both with docker build --progress=plain ., and count [CACHED] steps in each. Then run docker history <image> on both and compare the layer layout.

Field notes

• Cache mounts keep package downloads warm without polluting layers: RUN --mount=type=cache,target=/root/.npm npm ci (BuildKit; no layer is written for the mount).
• The cache is local. Ephemeral CI runners need --cache-from/--cache-to against a registry (e.g. type=registry,ref=...:buildcache) or they rebuild everything, every time.
• RUN caches on the command string only. Keep apt-get update && apt-get install -y curl in one instruction — split it, and a cached update pairs a fresh install with a stale package index.