Advanced Docker Compose

A single compose.yaml file is often not enough as an application grows. Developers need different configurations for development and production, several services sometimes share similar base configurations and want to reuse them without copying, and Sysadmins/DevOps Engineers need guarantees that the startup order between services is completely safe before the application is considered ready to serve traffic. This chapter continues the discussion from the fundamentals of Docker Compose with four advanced topics: structuring override files to separate configurations per environment, using extends for configuration composition between services, controlling startup and shutdown order through depends_on in greater depth, and a collection of the most commonly used daily docker compose CLI commands.

18.1 Override Files

Override files allow us to keep the same base configuration in one file, then add or replace parts of the configuration using another file according to the environment currently in use, without having to duplicate the entire compose.yaml.

18.1.1 Automatic compose.override.yaml

Compose automatically reads a file named compose.override.yaml in the same directory as compose.yaml every time we run docker compose up, without needing to specify it via any option. This pattern is suitable for storing development-specific configurations, such as source code bind mounts or extra ports for debugging, so that the main compose.yaml remains clean and ready for production use.

# compose.yaml
services:
  app:
    image: myapp:latest
    ports:
      - "3000:3000"

  db:
    image: postgres:16-alpine

# compose.override.yaml
services:
  app:
    build: .
    volumes:
      - ./src:/app/src
    environment:
      DEBUG: "true"

  db:
    ports:
      - "5432:5432"

When we run docker compose up in a directory containing both files above, the app service is automatically built from the local source via build: . instead of pulling image: myapp:latest, while also receiving additional bind mounts and environment variables from compose.override.yaml. The ports instruction on the db service is also added, so the database port that was not previously exposed to the host becomes directly accessible during development. Verify the merged result of these two files before actually running the containers using the following command.

docker compose config

18.1.2 Specifying Manual Override with -f

For environments other than development, such as staging or production, we can name our own override files and invoke them explicitly using the -f option, because Compose only reads compose.override.yaml automatically.

# compose.prod.yaml
services:
  app:
    image: myapp:latest
    restart: unless-stopped
    deploy:
      resources:
        limits:
          memory: 512M

docker compose -f compose.yaml -f compose.prod.yaml up -d

Compose merges these files according to the order they are specified in the -f option, with later-named files overriding or extending configurations from earlier ones. According to the official Docker Compose documentation, all relative paths within these merged files are calculated relative to the base file, which is the first file specified via -f, not relative to the location of the override file itself. If a project structure spreads Compose files across subdirectories of different teams, relative paths in override files might resolve incorrectly if this method is used; for such cases, the include element becomes a safer choice because each included file continues to use its own project directory as its path reference.

In practice, Sysadmins/DevOps Engineers usually prepare several override files at once, such as compose.staging.yaml and compose.prod.yaml, and then choose the appropriate -f combination via deployment scripts or CI/CD pipelines, so that a single base compose.yaml can be used consistently across all environments without configuration duplication.

18.2 Extends and Composition

The extends instruction allows a service to inherit configuration from another service defined in the same file or a separate file, which is useful when several services share identical base configurations but need minor adjustments.

18.2.1 Inheriting Configuration with extends

Imagine a worker application that has two execution modes: as a web server and as a background worker, but both share the same image, environment, and base configuration. We can separate this shared configuration into a dedicated file so it does not have to be written twice.

# common.yaml
services:
  app-base:
    build: .
    environment:
      CONFIG_FILE_PATH: /code/config
      API_KEY: xxxyyy

# compose.yaml
services:
  web:
    extends:
      file: common.yaml
      service: app-base
    command: /code/run_web
    ports:
      - "8080:8080"

  worker:
    extends:
      file: common.yaml
      service: app-base
    command: /code/run_worker

Both web and worker services inherit the build and environment instructions from app-base, and then each adds a different command instruction according to its role. Check the result of this configuration inheritance via docker compose config to ensure all fields from app-base have been merged correctly before running.

docker compose config web

18.2.2 Limitations of extends

According to the Compose Specification, extends has a number of important limitations that must be understood before using it. Dependencies declared via depends_on, links, volumes_from, or references like service:<name> in network_mode, ipc, or pid in the extended service are not automatically inherited. Compose leaves the responsibility to us to re-declare those dependencies explicitly in the service using extends, if needed. Circular references between services that extend each other are also not supported and will cause Compose to immediately terminate with an error.

Another limitation to consider is that extends does not work when the application is deployed via docker stack deploy; Compose will reject the configuration with the error message configuration contains forbidden properties. Therefore, the extends pattern is best suited for deployments using standard docker compose, not for applications planned to run on Docker Swarm.

18.3 Depends_on and Startup Order

Compose always creates and stops containers following the dependency order specified via depends_on, links, volumes_from, and network_mode: "service:..." references. This section discusses two depends_on conditions not commonly used in basic multi-container applications: waiting for a process to complete entirely and keeping services synchronized when their dependencies restart.

18.3.1 Waiting for Process Completion with service_completed_successfully

Some applications require a one-time initialization process before the main service is allowed to run, such as database schema migrations. The service_completed_successfully condition in depends_on makes the dependent service wait until the target service exits with a successful exit code, rather than merely waiting for its container to start.

services:
  migrate:
    build: .
    command: ["./manage.py", "migrate"]
    restart: "no"

  app:
    build: .
    depends_on:
      migrate:
        condition: service_completed_successfully
    ports:
      - "3000:3000"

The migrate service above is intentionally set with restart: "no" because its task is only to run once until completion, not a long-running process like typical services. Compose only starts app after migrate exits with a success status (exit code 0); if migrate fails and exits with an error code, app will never be executed. This one-shot service pattern is commonly used by Developers to ensure the database schema is always consistent with the application version being deployed, without requiring manual migrations outside of Compose.

18.3.2 Automatic Restart Following Dependencies

The restart: true option within the long syntax of depends_on causes Compose to automatically restart dependent services whenever their dependencies are restarted or updated through Compose operations, such as docker compose restart or re-running docker compose up after configuration changes.

services:
  web:
    build: .
    depends_on:
      db:
        condition: service_healthy
        restart: true

  db:
    image: postgres:16-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      retries: 5

The restart: true feature was introduced in Docker Compose version 2.17.0, so make sure the Compose version in use supports it. Note that this option only applies to restarts triggered explicitly by Compose operations, not automatic restarts by the container runtime when a container dies due to a crash. Without this option, if db is manually restarted while web has already established old database connections, web might keep holding onto disconnected sockets without knowing it needs to reconnect; this pattern helps Sysadmins/DevOps Engineers avoid that condition without having to manually restart web every time db is updated.

During shutdown, Compose reverses the startup order: services depending on other services are always stopped first, followed by their dependencies. In the two examples above, app and web are each stopped before migrate or db, ensuring core services like databases remain active as long as other services might still need them.

18.4 Docker Compose CLI Commands

In addition to up and down which are commonly used, there are several other docker compose commands frequently needed by Developers and Sysadmins/DevOps Engineers when operating multi-container applications daily.

18.4.1 Monitoring Status and Logs

The ps command lists all services along with their container statuses, while logs shows the log output from running services.

docker compose ps
docker compose logs -f app

The -f option on logs makes Compose continuously display new logs in real time (follow mode), similar to tail -f, and stops only when manually interrupted with Ctrl+C. Without specifying a service name, docker compose logs displays logs from all services simultaneously, which in practice is very helpful when tracing event sequences between services during multi-service errors.

18.4.2 Executing Commands Inside Services

The exec command runs a new command inside a running service container, whereas run creates a new container from the same service to execute a one-off task without affecting already running containers.

docker compose exec db psql -U postgres
docker compose run --rm app npm test

The --rm option on run ensures that the temporary container created is automatically removed after the command completes, preventing leftover testing or task containers from accumulating and filling up the disk.

18.4.3 Rebuilding and Restarting Services

After modifying a Dockerfile or source code used by the build instruction, we need to rebuild the image using build, or combine it directly with up using the --build option.

docker compose build app
docker compose up -d --build

The --build option on up forces Compose to rebuild images for every service that uses a build instruction before running its container, eliminating the need to run build and up as two separate commands. To simply restart a service without rebuilding its image or altering configurations, use restart.

docker compose restart app

18.4.4 Cleaning Up Resources

The down command stops and removes all containers and networks created by Compose for the project.

docker compose down

By default, down does not remove named volumes or built images, keeping data in volumes safe if up is executed again later. Adding the -v option instructs Compose to also remove all named volumes declared in compose.yaml, an action that is destructive and irreversible.

docker compose down -v

Before running down -v in any environment storing critical data like production databases, ensure a valid data backup exists, as this command immediately deletes volume contents without additional confirmation. Verify project configurations or the existence of currently used Compose files at any time using the config command discussed in the previous section, especially before running potentially destructive commands like this one.