Performance Optimization

Fat images and slow builds hamper daily developer productivity, while resource-greedy containers can disrupt other services running on the same server during production. Docker performance optimization is not just about speed, but also about infrastructure cost efficiency and overall system stability. This chapter covers four optimization areas most frequently addressed by Sysadmins, DevOps Engineers, and Developers: reducing image size, speeding up build processes, maintaining runtime container performance, and managing resource allocation to prevent any single container from consuming the entire host capacity.

27.1 Image Size Optimization

Large images slow down the docker pull process during deployment, increase registry storage requirements, and expand the attack surface because more packages potentially harbor vulnerabilities. Reducing image size is one of the optimizations with the most immediate impact, both in terms of speed and security.

27.1.1 Multi-Stage Builds and Minimal Base Images

Multi-stage builds separate the compilation or dependency installation stage from the final image stage that actually runs, ensuring build tools like compilers or development dependencies are not carried over into the production image. The following example builds a Go application resulting in a final image that contains only a single binary.

FROM golang:1.23 AS builder
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 go build -o app .

FROM gcr.io/distroless/static-debian12
COPY --from=builder /src/app /app
ENTRYPOINT ["/app"]

The builder stage uses the golang:1.23 image, which is hundreds of megabytes in size because it contains the entire Go toolchain, but the final stage only copies a single binary file to a distroless base image that lacks a shell, package manager, or any additional tools. The choice of base image is as important as the multi-stage pattern itself. The alpine image, which uses musl libc and busybox, is much smaller than full Debian or Ubuntu-based images, while Google's distroless base images do not even include a shell at all, making them suitable for statically compiled binaries that do not require interactive debugging inside the container.

Compare base image sizes before deciding which one to use, as the difference can be significant for frequently redeployed applications.

docker pull python:3.12
docker pull python:3.12-slim
docker pull python:3.12-alpine
docker images python

In practice, the alpine image is indeed the smallest, but this variant uses musl libc instead of glibc, which sometimes causes compatibility issues for Python packages with native extensions like numpy or psycopg2 that expect glibc. The slim variant serves as a safer compromise for such cases because it still uses glibc while stripping non-essential documentation and tools.

27.1.2 Reducing Layer Count and .dockerignore

Every RUN, COPY, and ADD instruction in a Dockerfile produces a new layer. Combining multiple related shell commands into a single RUN instruction reduces the number of layers while preventing temporary files from being permanently saved in a separate layer.

RUN apt-get update \
    && apt-get install -y --no-install-recommends curl ca-certificates \
    && rm -rf /var/lib/apt/lists/*

Notice that the apt-get update process, package installation, and apt cache cleanup are combined into the exact same RUN instruction. If the cleanup command were separated into its own RUN instruction, the apt cache would remain permanently saved in the previous installation layer despite logically being "removed" in the subsequent layer, because every layer in a Docker image is immutable and stores only filesystem diffs. The --no-install-recommends option prevents apt-get from adding recommended packages that are not strictly required to run the application.

The .dockerignore file plays an often overlooked role in image size optimization, particularly for the COPY . . instruction, which copies the entire build context contents. Without .dockerignore, directories like node_modules, .git, or local .env files are sent to the build context and risk being copied into the image.

.git
node_modules
*.log
.env
dist
__pycache__/

In addition to shrinking the image size, .dockerignore speeds up the build process because the Docker daemon does not need to transmit irrelevant files as part of the build context to the builder.

27.1.3 Analyzing Layer Sizes with Dive

The docker history command displays the size of each layer in an image alongside the Dockerfile instruction that generated it, offering the quickest way to spot which layers are consuming the most space.

docker history payment-service:latest

For a more granular analysis down to individual files inside each layer, the Dive tool provides an interactive view that breaks down size contributions per layer and highlights potential wasted space caused by files overwritten or deleted in subsequent layers without actually shrinking the image size.

dive payment-service:latest

Dive displays an image efficiency score along with a list of files taking up the most space, helping uncover cases like build logs or cache files accidentally copied into the final image. For quick verification without installing extra tools, compare image sizes before and after optimization using docker images and ensure the difference aligns with expectations after applying multi-stage builds or changing base images.

27.2 Build Speed Improvement

Slow builds hinder developer iterations during development and lengthen CI/CD pipeline wait times before an image is ready to deploy. BuildKit, the default builder since Docker Engine 23.0, offers several caching features that are far more flexible than the legacy builder.

27.2.1 BuildKit Cache Mount for Dependencies

Cache mounts allow a RUN instruction to use a persistent cache directory that is not stored as part of the image layer, making it ideal for package manager caches like pip, npm, or apt that should be reused across builds without inflating the final image size. This feature requires a syntax directive on the first line of the Dockerfile.

# syntax=docker/dockerfile:1
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt
COPY . .
CMD ["python", "app.py"]

Without --mount=type=cache, any change to requirements.txt forces pip to re-download all dependencies from scratch because the installation layer is rendered invalid once its source file changes. A cache mount keeps the /root/.cache/pip directory stored in the BuildKit cache across builds, meaning previously downloaded packages do not need to be re-downloaded even if the RUN layer itself must be re-executed. Verify that the cache is working by running the build twice in succession and comparing the time taken during the dependency installation step.

time docker build -t myapp .
time docker build -t myapp .

27.2.2 Instruction Order and Build Cache

Both BuildKit and the legacy builder use a layer cache mechanism that considers a layer valid as long as its instruction and context remain unchanged from the previous build. Once a layer is deemed modified, all subsequent layers are rebuilt, even if their contents are actually unrelated. Order Dockerfile instructions from least frequently changed to most frequently changed so that the cache stays valid for as long as possible.

FROM node:20-alpine
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
CMD ["node", "server.js"]

Copying package.json and package-lock.json before running npm ci, and only then copying the rest of the source code, ensures that the dependency installation layer continues to use the cache as long as the lock file remains unchanged, even if application code changes with every commit. If the order were reversed, with COPY . . executed before dependency installation, any tiny edit to the source code would force the entire dependency installation process to run again from scratch because the preceding COPY layer would already be invalid.

27.2.3 Parallel Builds with Buildx Bake

Projects targeting multiple images, such as several microservices inside a single monorepo, often need to build more than one image at a time. docker buildx bake executes multiple builds defined in a single configuration file, running independent targets concurrently via BuildKit.

group "default" {
  targets = ["api", "worker"]
}

target "api" {
  context    = "./api"
  dockerfile = "Dockerfile"
  tags       = ["myorg/api:latest"]
}

target "worker" {
  context    = "./worker"
  dockerfile = "Dockerfile"
  tags       = ["myorg/worker:latest"]
}

Save the configuration above as docker-bake.hcl, then run the entire group with a single command.

docker buildx bake

Compared to running docker build sequentially for each service, buildx bake leverages BuildKit parallelism so that total build time approaches the build time of the slowest single target rather than the sum of all targets. In real-world environments, this benefit becomes even more pronounced in CI pipelines with idle CPU cores, where sequential builds fail to utilize available resources fully.

27.3 Runtime Performance

Small images and fast builds do not automatically guarantee optimal container execution in production. Runtime configurations such as storage drivers, logging drivers, and health checks help determine how responsive and stable a container remains under actual workloads.

27.3.1 Selecting Storage Drivers and Logging Drivers

overlay2 is the default and Docker-recommended storage driver for modern Linux kernels, replacing legacy drivers like aufs or devicemapper. Check the active storage driver using docker info.

docker info --format '{{.Driver}}'

The logging driver also directly impacts performance, particularly for applications generating high log volumes. The default json-file driver stores all logs as JSON files on the host with no size limit unless explicitly configured, which over time can fill up the host disk and slow down log reading via docker logs.

docker run -d \
  --log-driver json-file \
  --log-opt max-size=10m \
  --log-opt max-file=3 \
  nginx:alpine

The max-size=10m option caps a single log file size before rotation occurs, while max-file=3 limits the number of log files retained before the oldest file is automatically deleted. Without these limits, Sysadmins and DevOps Engineers in production frequently encounter full host disks caused not by application data, but by unrotated container logs accumulated over years.

27.3.2 Optimizing Health Checks

The HEALTHCHECK instruction helps orchestrators and Sysadmins/DevOps Engineers detect containers that remain technically running while their applications have stopped responding properly. Overly aggressive health check intervals can burden the application with repeated check requests, whereas overly loose intervals delay problem detection.

HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD curl -f http://localhost:8080/health || exit 1

--start-period provides a grace period upon initial container startup before failed health checks count toward marking the container unhealthy, essential for applications requiring warm-up time such as Java apps or services loading large caches on startup. Without this grace period, a container might immediately be marked unhealthy and forcefully restarted by an orchestrator while still undergoing normal initialization. Ensure the health check endpoint itself is lightweight and does not perform heavy database queries, as a slow endpoint adds overhead every time it is invoked per the configured --interval.

Verify health check status via docker inspect or the STATUS column in docker ps.

docker inspect --format='{{.State.Health.Status}}' payment-api

27.3.3 Monitoring Performance with Docker Stats

docker stats displays real-time CPU, memory, network I/O, and block I/O usage for all running containers, providing the quickest method to identify which container is driving excessive host load.

docker stats --no-stream

The --no-stream option outputs a single snapshot and exits, making it suitable for monitoring scripts compared to the default mode that continuously updates every second. For deeper investigation when a container's CPU usage suddenly spikes, combine it with docker top to inspect specific processes running inside that container.

docker top payment-api

In production, abnormal CPU spikes within a container often stem from unmanaged child processes, such as accumulating zombie processes caused by a container lacking a proper init process. The --init flag on docker run inserts a minimal init process (tini) that handles these zombie processes automatically.

27.4 Resource Allocation

Without explicit limits, a container can by default consume all available CPU and memory on the host, risking a situation where a single misbehaving application compromises the stability of other services on the same host. Docker uses cgroups (control groups) in the Linux kernel to isolate and constrain each container's resources.

27.4.1 Limiting Memory and Swap

The --memory option sets the maximum amount of memory a container is permitted to use. Once this limit is exceeded, the kernel terminates processes inside the container using the OOM killer (Out Of Memory killer).

docker run -d --memory=512m --memory-swap=512m nginx:alpine

Setting --memory-swap equal to --memory effectively disables swap usage for that container because, according to official Docker documentation, --memory-swap represents total memory plus swap permitted; matching values mean zero additional swap allocation. Disabling swap ensures an out-of-memory container is immediately stopped by the OOM killer rather than quietly degrading in performance due to continuous disk swapping, a behavior far easier to diagnose in production than a container hanging without clear errors.

Check if a container was ever OOM killed using docker inspect.

docker inspect --format='{{.State.OOMKilled}}' payment-api

27.4.2 Limiting CPU

The --cpus option caps the number of CPU cores a container can utilize, specified as a decimal representing core count rather than a percentage.

docker run -d --cpus=1.5 payment-worker:latest

A value of 1.5 means the container can use at most the equivalent of one and a half CPU cores overall, regardless of how many cores the host possesses. For scenarios where multiple containers share CPU proportionally rather than via hard caps, the --cpu-shares option configures relative priority among containers specifically during CPU contention.

docker run -d --cpu-shares=1024 payment-api:latest
docker run -d --cpu-shares=512 payment-worker:latest

Using the default value of --cpu-shares=1024 as a baseline, the payment-worker container above receives half the CPU share of payment-api when both contend for CPU. However, when the host CPU is not saturated, both containers remain free to utilize as much CPU as needed without restriction. Unlike --cpus, which imposes a hard limit, --cpu-shares governs relative priority during resource contention; combining them as needed works best: use --cpus for hard caps to prevent host monopolization, and --cpu-shares to prioritize critical services under heavy loads.

27.4.3 Updating Limits Without Restarting Containers

Modifying resource limits traditionally requires stopping and re-running a container with new flags, causing brief downtime even for simple CPU or memory adjustments. The docker update command enables resource limit updates on live containers without requiring a restart.

docker update --memory=1g --memory-swap=1g --cpus=2 payment-api

Changes made via docker update take effect immediately in the container's cgroup without interrupting running processes inside, making it invaluable when Sysadmins or DevOps Engineers need to urgently scale up service capacity during sudden traffic spikes without waiting for a maintenance window. Verify that new limits were applied using docker inspect.

docker inspect --format='{{.HostConfig.Memory}} {{.HostConfig.NanoCpus}}' payment-api

Note that docker update only adjusts resource limits, not other configurations like port mappings or volume mounts. Furthermore, these changes do not persist if the container is later deleted and recreated from an image, so permanent updates still require revising the corresponding docker run command or Compose file for subsequent deployments.