Docker Build Cache & Layering

Have you ever wondered why the first build takes 5–10 minutes, but the second build finishes in just a few seconds? That’s thanks to Build Cache. But sometimes, changing just one line of code causes Docker to rebuild everything — that’s because you don’t yet understand how Layering works. 🧱

In this article, I’ll explain in detail how Layers and Build Cache work, how Docker decides when to use cache versus rebuild, and techniques to optimize for faster builds.

What Is a Layer in Docker?

Each instruction in a Dockerfile creates a layer in the image. You can think of an image as a stack of layers piled on top of each other, with each layer adding new content on top of the previous one.

How Layers Are Created

1
2
3
4
5
6

FROM node:18-alpine        # Layer 1: Base image
WORKDIR /app               # Layer 2: Create working directory
COPY package.json ./       # Layer 3: Copy package.json file
RUN npm install            # Layer 4: Install dependencies
COPY . .                   # Layer 5: Copy source code
RUN npm run build          # Layer 6: Build the application

Key Characteristics of Layers

• Read-only: Each layer is immutable once created — it cannot be modified
• Shareable: Multiple images can share identical layers (e.g., the same node:18-alpine base image)
• Ordered: Each layer stacks on top of the previous one, forming a complete file system
• Sized: Each layer has its own size — the sum of all layers equals the image size

How Does Build Cache Work?

Build Cache is the mechanism that allows Docker to skip unnecessary steps when rebuilding an image. If a layer hasn’t changed since the last build, Docker reuses the previous result instead of running it again.

Docker’s Cache Rules

Docker checks cache in order for each instruction in the Dockerfile:

1
2
3
4
5

Instruction 1 → Cache hit? ✅ Reuse → Continue
Instruction 2 → Cache hit? ✅ Reuse → Continue
Instruction 3 → Cache hit? ❌ Cache miss → Rebuild from here onward
Instruction 4 → Must rebuild (because instruction 3 changed)
Instruction 5 → Must rebuild

The most important rule: When a layer is invalidated (cache miss), all subsequent layers must also be rebuilt — even if they haven’t changed at all.

When Is Cache Invalidated?

Cache Invalidation Example

1
2
3
4
5

FROM node:18-alpine          # ✅ Cache hit (unchanged)
WORKDIR /app                 # ✅ Cache hit (unchanged)
COPY . .                     # ❌ Cache miss (you edited a .js file)
RUN npm install              # ❌ Must rebuild (because COPY above missed)
RUN npm run build            # ❌ Must rebuild (because previous layer missed)

You only edited one small .js file, but npm install has to run again even though package.json hasn’t changed. This is the most common issue with unoptimized Dockerfiles.

Techniques to Optimize Build Cache

Order Instructions by Change Frequency

Principle: Instructions that rarely change → place first. Instructions that change often → place last.

❌ Not optimized:

1
2
3
4
5

FROM node:18-alpine
WORKDIR /app
COPY . .                     # Copies everything (changes frequently)
RUN npm install              # Must re-run every time code changes
RUN npm run build

✅ Optimized:

1
2
3
4
5
6

FROM node:18-alpine
WORKDIR /app
COPY package.json yarn.lock ./    # Rarely changes → copy first
RUN npm install                   # Only rebuilds when dependencies change
COPY . .                          # Changes often → copy last
RUN npm run build

Result: When only source code changes, npm install is cached → saving 2–5 minutes per build.

Use .dockerignore

The .dockerignore file excludes unnecessary files from the build context, reducing the risk of unnecessary cache invalidation.

1
2
3
4
5
6
7
8

# .dockerignore
node_modules
dist
.git
.env
*.log
tmp*
.DS_Store

Why is this important? Without .dockerignore, the COPY . . instruction copies node_modules (hundreds of MBs) and .git into the build context. Any change in these directories triggers a cache miss.

Use Cache Mounts

Cache mounts allow you to persist package manager caches across builds, even when a layer is rebuilt.

Node.js:

1
2
3
4
5
6
7

FROM node:18-alpine
WORKDIR /app
COPY package.json yarn.lock ./
RUN --mount=type=cache,target=/root/.npm \
    npm install
COPY . .
RUN npm run build

Python:

1
2
3
4
5
6

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt ./
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt
COPY . .

Go:

1
2
3
4
5
6
7
8

FROM golang:1.21-alpine
WORKDIR /app
COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    go mod download
COPY . .
RUN go build -o /app/server

Benefit: Even when package.json changes and the layer must rebuild, previously downloaded packages are still stored in the cache mount → only new packages are downloaded, not everything from scratch.

Multi-stage Build

Multi-stage builds separate the build process from the runtime, reducing the final image size and optimizing cache for each stage.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# Stage 1: Build (heavy layers but not included in the final image)
FROM node:18-alpine AS builder
WORKDIR /app
COPY package.json yarn.lock ./
RUN npm install
COPY . .
RUN npm run build

# Stage 2: Runtime (only contains necessary files)
FROM nginx:alpine AS production
COPY --from=builder /app/dist /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

Real-World Example: Optimizing a Node.js Dockerfile

Unoptimized Dockerfile

1
2
3
4
5
6

FROM node:18
WORKDIR /app
COPY . .
RUN npm install
RUN npm run build
CMD ["node", "dist/server.js"]

Issues:

• ❌ Large base image (node:18 ~ 900 MB)
• ❌ COPY . . before npm install → any code change rebuilds dependencies
• ❌ No .dockerignore → copies node_modules, .git
• ❌ No multi-stage → final image includes devDependencies

Optimized Dockerfile

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# Stage 1: Install dependencies
FROM node:18-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci --only=production

# Stage 2: Build
FROM node:18-alpine AS builder
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci
COPY . .
RUN npm run build

# Stage 3: Runtime
FROM node:18-alpine AS runtime
WORKDIR /app
ENV NODE_ENV=production

COPY --from=deps /app/node_modules ./node_modules
COPY --from=builder /app/dist ./dist
COPY package.json ./

EXPOSE 3000
CMD ["node", "dist/server.js"]

Improvements:

• ✅ Small base image (node:18-alpine ~ 170 MB)
• ✅ Separate COPY package.json → efficient dependency caching
• ✅ Cache mount for npm → faster re-downloads on rebuild
• ✅ Multi-stage → final image only contains production dependencies
• ✅ Uses npm ci instead of npm install → ensures reproducible builds

Results Comparison

Useful Commands for Debugging Cache

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# View detailed build output (shows cache hit/miss)
docker build --progress=plain -t myapp .

# Build without cache (force full rebuild)
docker build --no-cache -t myapp .

# Invalidate cache for a specific stage only
docker build --no-cache-filter builder -t myapp .

# View image layers
docker history myapp

# Check build cache disk usage
docker builder du

# Clear build cache
docker builder prune

Reading build output: During builds, Docker shows CACHED for layers that use cache:

1
2
3
4

 => CACHED [deps 2/3] COPY package.json package-lock.json ./
 => CACHED [deps 3/3] RUN npm ci --only=production
 => [builder 4/5] COPY . .
 => [builder 5/5] RUN npm run build

Lines with CACHED mean the layer was retrieved from cache and didn’t need to be rebuilt.

Implementation Notes

• When applying to your project, keep these important points in mind:
  • Always create a .dockerignore file before writing your Dockerfile
  • Separate COPY package.json before RUN npm install
  • Use multi-stage builds for production images
  • Cache mounts (--mount=type=cache) require BuildKit (enabled by default since Docker 23.0+)
• Best practices:
  • Order instructions: rarely changes → first, frequently changes → last
  • Use npm ci instead of npm install in CI/CD
  • Pin specific versions for base images (node:18.12-alpine instead of node:latest)
• Troubleshooting:
  • Build is slow even though nothing changed? → Check .dockerignore, unrelated files may be triggering cache misses
  • Cache mounts not working? → Make sure Docker BuildKit is enabled: DOCKER_BUILDKIT=1
  • Want to see which layer missed? → Use docker build --progress=plain

🎯 Conclusion

To summarize:

• Layer = each instruction in a Dockerfile creates a layer in the image
• Build Cache = Docker stores layer results and reuses them when rebuilding
• Cache invalidation = when a layer changes, all subsequent layers must rebuild
• Optimization = proper instruction ordering + .dockerignore + cache mounts + multi-stage builds

By understanding and applying these 4 techniques, you can reduce Docker build times from 5 minutes to under 30 seconds in most cases. 🚀

If you find this article helpful, feel free to ⭐ star the repo or 📱 share it with your friends! 😊

References

• Docker Official Documentation - Build Cache
• Docker Official Documentation - Cache Invalidation
• Docker Official Documentation - Optimize Cache
• Best practices for writing Dockerfiles