Creating a Multi-Container Application

A production-ready web application rarely suffices with a single service alone. Typically, there is a primary application service, a database to store data, and a cache to accelerate access to frequently reused data. This chapter demonstrates how to define such an application using compose.yaml, ranging from declaring each service that makes up the application, connecting them via a network, persisting data using a volume, to managing configurations via environment variables. The example used throughout this chapter is an application with three services: app as the main service, db as a PostgreSQL database, and cache as Redis, which developers and Sysadmins/DevOps Engineers commonly encounter in the field when setting up development and production environments.

17.1 Service Definitions

The services element is the core part of compose.yaml because it is where each container that forms the application is defined. Each key under services represents a single service, complete with the image or build instructions used, exposed ports, and dependencies on other services.

17.1.1 The services Structure in compose.yaml

Here is the definition of the three services used as an example throughout this chapter.

services:
  app:
    build: .
    ports:
      - "3000:3000"
    depends_on:
      - db
      - cache

  db:
    image: postgres:16-alpine

  cache:
    image: redis:7-alpine

The build: . instruction on the app service builds an image from the Dockerfile in the current directory, whereas db and cache directly use official images from Docker Hub via the image instruction. The Compose Specification still requires the image instruction in every service, unless that service already includes a build instruction instead. If both build and image instructions are specified for a single service, Compose will attempt to pull that image first according to the active pull policy, and then build it from source if the image is not available.

17.1.2 Startup Conditions with depends_on

The depends_on instruction in the example above ensures Compose runs db and cache before app starts running. However, container start order is not identical to the readiness of the service inside it. PostgreSQL requires initialization time before it is truly ready to accept connections, whereas the short form of depends_on above only waits for its container to start, not for the service inside to be fully ready.

For cases requiring guaranteed service readiness, use the long syntax of depends_on with the service_healthy condition, combined with a healthcheck on the target service.

services:
  app:
    build: .
    ports:
      - "3000:3000"
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started

  db:
    image: postgres:16-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

  cache:
    image: redis:7-alpine

The service_healthy condition makes app wait until the healthcheck for db reaches a healthy status, whereas the service_started condition on cache simply waits for its container to start like the default behavior. In practice, developers frequently encounter applications crashing upon initial startup because they attempt to connect to a database that is not yet ready when setting up a multi-container app for the first time. Combining healthcheck with service_healthy provides a much more reliable solution than merely adding a manual delay to the application startup script.

17.1.3 Restart Policies

In addition to startup order, every service ideally should also have a clear restart policy defined using the restart instruction, especially for services that must remain running in production.

services:
  app:
    build: .
    restart: unless-stopped
    ports:
      - "3000:3000"

The value unless-stopped instructs Docker to automatically restart the container if the process inside stops on its own, unless the container was manually stopped via docker compose stop. Other available options are "no" (default, never restart automatically), always, and on-failure, which only restarts if the container exits with an error exit code. Sysadmins and DevOps Engineers managing services in production generally select unless-stopped or always for core services such as databases and primary applications, allowing services to recover automatically after crashes without manual intervention.

17.2 Networks in Compose

Every service defined via Compose requires a way to communicate with one another. The networks element configures how connections between these services are established, including when a service needs to be isolated from other unrelated services.

17.2.1 Automatic Default Network

If no networks element is declared at all, Compose automatically creates a single bridge-type default network for the entire project. Every service that does not explicitly specify networks automatically connects to this default network. This is why in the compose.yaml example from the previous section, the app service could immediately reach db and cache simply using their service names as hostnames, without any additional network configuration.

docker compose up -d
docker network ls

The docker network ls command above will display a new network named according to the pattern <project-name>_default, corresponding to the project directory name or the name value specified via the -p option in Docker Compose.

17.2.2 Custom Networks and Service Isolation

The default network is suitable for simple applications, but more complex applications often require isolation; for instance, a database that should only be accessible by the application service, not by other public-facing services. This requirement can be fulfilled using custom networks declared in the top-level networks element.

services:
  app:
    build: .
    ports:
      - "3000:3000"
    networks:
      - frontend
      - backend

  db:
    image: postgres:16-alpine
    networks:
      - backend

  cache:
    image: redis:7-alpine
    networks:
      - backend

networks:
  frontend:
  backend:

The configuration above places db and cache exclusively on the backend network, while app connects to both frontend and backend because it needs to reach both. If another service is added only to frontend (such as a reverse proxy), that service will not be able to reach db or cache directly since it is not on the same backend network. This pattern of isolation is common practice for Sysadmins and DevOps Engineers to reduce the attack surface, because an attacker who successfully breaches a service on the frontend does not automatically gain a direct network path to the database on the backend.

17.2.3 Network Aliases Between Services

In addition to the service name itself, a service can be assigned additional hostnames on a specific network using the aliases option. This feature is useful when an application expects a specific hostname different from the service name in Compose, such as during migrations from legacy environments that hardcode specific database hostnames.

services:
  db:
    image: postgres:16-alpine
    networks:
      backend:
        aliases:
          - database
          - postgres-primary

networks:
  backend:

The configuration above enables other services on the backend network to reach db using three hostnames simultaneously: the service name db itself, database, or postgres-primary. DNS resolution between services can be verified directly from inside a container using the exec command.

docker compose exec app getent hosts database

If the command above fails to display an IP address, recheck whether the target service is indeed on the same network with the correct alias, as aliases only apply to the network on which they are defined.

17.3 Volumes in Compose

Containers that are restarted or removed lose all data stored within their writable layer. Therefore, data that must persist longer, such as database files, must be stored using volumes. Compose simplifies volume management through the top-level volumes element, which can be shared across multiple services simultaneously.

17.3.1 Named Volumes for Persistent Data

Persisting PostgreSQL data in the previous example using a named volume is recommended so that data is not lost every time the db container is removed and recreated.

services:
  db:
    image: postgres:16-alpine
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  db-data:

The named volume db-data is declared in the top-level volumes element, and then mounted to the db service using the short syntax <volume-name>:<container-path>. According to the Compose Specification, volumes intended for reuse across multiple services must be declared in this top-level element; host paths used by only a single service can be specified directly inside the service definition without a top-level declaration. If the db-data volume does not exist when running docker compose up, Docker automatically creates it.

17.3.2 Bind Mounts in Compose

In addition to named volumes, Compose supports bind mounts using the same syntax, with the host path as the source. Bind mounts are commonly used by developers for live reloading during development, allowing code changes on the host to reflect inside the container immediately without rebuilding the image.

services:
  app:
    build: .
    ports:
      - "3000:3000"
    volumes:
      - ./src:/app/src
      - /app/node_modules

The first line mounts the ./src directory on the host to /app/src inside the container, so source file modifications are instantly read by the process running in the container. The second line containing only a container path without a host path is called an anonymous volume. This pattern prevents the node_modules directory installed inside the image from being overwritten by the host node_modules directory, which might be empty or contain different dependency versions.

For more granular control, such as making a mount read-only, use the long mapping syntax instead of a single string.

services:
  app:
    build: .
    volumes:
      - type: bind
        source: ./config
        target: /app/config
        read_only: true

17.3.3 Sharing Volumes Between Services

The same named volume can also be mounted to multiple services at the same time. This is useful when several services need access to the same data, such as a backup service that reads PostgreSQL data directly without establishing a database connection.

services:
  db:
    image: postgres:16-alpine
    volumes:
      - db-data:/var/lib/postgresql/data

  backup:
    image: postgres:16-alpine
    volumes:
      - db-data:/var/lib/postgresql/data:ro
    entrypoint: ["sleep", "infinity"]

volumes:
  db-data:

The :ro option at the end of the path in the backup service mounts the volume in read-only mode, enabling the backup service to read data without risking writing to or corrupting data currently used by the db service. The contents of this shared volume can be verified using the docker compose exec command.

docker compose exec backup ls -la /var/lib/postgresql/data

Note that removing volumes using docker compose down -v is destructive and permanently deletes all data in the named volume, including volumes shared across multiple services like the example above. Ensure that a database backup strategy is actively running in production before executing this command in any environment storing critical data.

17.4 Environment Variables

Hardcoding configurations such as database credentials, application modes, or other service addresses directly into an image is bad practice. Storing configuration separately allows the same image to be reused across development, staging, and production simply by updating its configuration settings. Compose offers multiple methods to manage these environment variables.

17.4.1 The environment Instruction in compose.yaml

The most direct approach is defining environment variables using the environment instruction inside the service definition.

services:
  app:
    build: .
    environment:
      - NODE_ENV=production
      - DB_HOST=db
      - DB_PORT=5432
      - CACHE_HOST=cache

The environment instruction above can also be written in a key-value mapping format instead of a list; both forms are valid under the Compose Specification.

services:
  app:
    build: .
    environment:
      NODE_ENV: production
      DB_HOST: db
      DB_PORT: "5432"
      CACHE_HOST: cache

The second example intentionally wraps the DB_PORT value in quotes because YAML interprets unquoted numbers as numeric types rather than strings, and certain applications require environment variables strictly formatted as strings.

17.4.2 Separating Configurations with env_file

Writing numerous environment variables directly in compose.yaml can quickly clutter the file, especially when variables contain sensitive data like passwords. The env_file instruction moves this list of variables into a separate file.

services:
  db:
    image: postgres:16-alpine
    env_file:
      - ./db.env

# db.env
POSTGRES_USER=appuser
POSTGRES_PASSWORD=changeme
POSTGRES_DB=appdb

The env_file instruction can also accept multiple files as a list. According to the Compose Specification, Compose processes these files sequentially, with later files overriding values from earlier ones if duplicate keys exist. The environment instruction defined on a service always takes precedence over values originating from env_file, effectively acting as an override when both instructions are used together on a single service.

Because files like db.env typically contain credentials, never commit them to version control. Add db.env to the .gitignore file and provide a sample file such as db.env.example containing variable names without sensitive values, ensuring other team members know which variables must be set without leaking real credentials.

17.4.3 The .env File and Variable Substitution

Unlike env_file, which injects environment variables into a container, a .env file placed in the same directory as compose.yaml performs variable substitution. This mechanism populates variable values referenced inside compose.yaml before the file is evaluated by Compose.

# .env
POSTGRES_PASSWORD=changeme
APP_PORT=3000

services:
  app:
    build: .
    ports:
      - "${APP_PORT}:3000"

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}

When executing docker compose up, Compose automatically reads the .env file in the project directory and replaces placeholders like ${APP_PORT} and ${POSTGRES_PASSWORD} with their corresponding values before the configuration is applied. The substitution results can be verified without starting containers using the docker compose config command.

docker compose config

The command above prints the final evaluated compose.yaml, complete with all substituted variable values. This provides a practical way to ensure no placeholders were missed prior to running the application. If a variable referenced via ${...} is missing from both the .env file and the shell environment, Compose substitutes it with an empty string and displays a warning rather than stopping with an error. Consequently, such omissions can easily go unnoticed without verifying the output of docker compose config first.