Advanced Docker Compose

8 min read•Docker for Developers

Advanced Docker Compose

This chapter covers advanced Docker Compose features: profiles for conditional services, sophisticated dependency management, variable substitution, extensions, and patterns for different environments.

Compose Profiles

Profiles allow you to selectively start services based on context.

Defining Profiles

YAML

services:
  app:
    image: myapp:latest
    # No profile - always started
 
  postgres:
    image: postgres:16
    # No profile - always started
 
  adminer:
    image: adminer
    profiles: ["debug"]
    ports:
      - "8080:8080"
 
  prometheus:
    image: prom/prometheus
    profiles: ["monitoring"]
 
  grafana:
    image: grafana/grafana
    profiles: ["monitoring"]
 
  test:
    image: myapp:test
    profiles: ["test"]
    command: npm test

Using Profiles

BASH

# Start only default services (no profile)
docker compose up
 
# Start with specific profile
docker compose --profile debug up
 
# Multiple profiles
docker compose --profile debug --profile monitoring up
 
# Start all services regardless of profile
docker compose --profile "*" up

Profile Use Cases

Profile	Services	Purpose
(default)	app, postgres	Core application
debug	adminer	Database GUI
monitoring	prometheus, grafana	Observability
test	test runner	Running tests

Note

Services without a profile start by default. Services with profiles only start when that profile is explicitly activated.

Variable Substitution

Use environment variables in compose files:

Basic Substitution

YAML

services:
  app:
    image: myapp:${VERSION:-latest}
    ports:
      - "${PORT:-3000}:3000"
    environment:
      - DATABASE_URL=${DATABASE_URL}
      - NODE_ENV=${NODE_ENV:-development}

BASH

# Use environment variables
VERSION=1.0.0 PORT=8080 docker compose up
 
# Or from .env file (loaded automatically)
docker compose up

The .env File

BASH

# .env file (loaded automatically by Compose)
VERSION=1.0.0
PORT=3000
NODE_ENV=production
DATABASE_URL=postgres://user:pass@host/db

Variable Syntax

YAML

services:
  app:
    image: myapp:${VERSION}           # Required variable
    image: myapp:${VERSION:-latest}   # Default value if unset
    image: myapp:${VERSION-latest}    # Default only if undefined
    image: myapp:${VERSION:?Error}    # Error if unset or empty
    image: myapp:${VERSION?Error}     # Error if undefined

Variable Substitution Rules

Syntax	Variable Set	Variable Empty	Variable Unset
`${VAR}`	value	empty	empty
`${VAR:-default}`	value	default	default
`${VAR-default}`	value	empty	default
`${VAR:?error}`	value	error	error
`${VAR?error}`	value	empty	error

Multiple .env Files

BASH

# Load specific env file
docker compose --env-file .env.production up
 
# Services can have their own env files

YAML

services:
  app:
    env_file:
      - .env
      - .env.local
      - .env.${ENVIRONMENT:-development}

Extension Fields

Reuse configuration with extension fields (YAML anchors):

Basic Extensions

YAML

# Define reusable configuration with x- prefix
x-common-env: &common-env
  TZ: UTC
  LOG_LEVEL: info
 
x-healthcheck: &default-healthcheck
  interval: 30s
  timeout: 10s
  retries: 3
 
services:
  api:
    image: myapi
    environment:
      <<: *common-env
      DATABASE_URL: postgres://...
    healthcheck:
      <<: *default-healthcheck
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
 
  worker:
    image: myworker
    environment:
      <<: *common-env
      WORKER_QUEUE: jobs
    healthcheck:
      <<: *default-healthcheck
      test: ["CMD", "curl", "-f", "http://localhost:3001/health"]

Complex Extensions

YAML

x-app-common: &app-common
  build:
    context: .
    target: production
  restart: unless-stopped
  networks:
    - app-network
  logging:
    driver: json-file
    options:
      max-size: "10m"
      max-file: "3"
 
services:
  api:
    <<: *app-common
    ports:
      - "3000:3000"
    environment:
      SERVICE: api
 
  worker:
    <<: *app-common
    command: npm run worker
    environment:
      SERVICE: worker

Advanced Dependencies

Dependency Conditions

YAML

services:
  app:
    depends_on:
      postgres:
        condition: service_healthy
        restart: true # Restart if dependency restarts
      redis:
        condition: service_started
      migrations:
        condition: service_completed_successfully
 
  postgres:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5
      start_period: 10s
 
  migrations:
    image: myapp:latest
    command: npm run migrate
    depends_on:
      postgres:
        condition: service_healthy

Dependency Conditions

Condition	Description
`service_started`	Container has started (default)
`service_healthy`	Healthcheck passes
`service_completed_successfully`	Container exited with code 0

Startup Order Pattern

YAML

services:
  # 1. Database starts first
  postgres:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready"]
      interval: 5s
      retries: 10
 
  # 2. Migrations run after database is healthy
  migrations:
    image: myapp
    command: npm run db:migrate
    depends_on:
      postgres:
        condition: service_healthy
    restart: "no"
 
  # 3. Seed data after migrations complete
  seeder:
    image: myapp
    command: npm run db:seed
    depends_on:
      migrations:
        condition: service_completed_successfully
    restart: "no"
 
  # 4. App starts after seeding
  app:
    image: myapp
    depends_on:
      seeder:
        condition: service_completed_successfully

Networking Configuration

Custom Networks

YAML

services:
  frontend:
    networks:
      frontend:
        aliases:
          - web
 
  api:
    networks:
      frontend:
        aliases:
          - backend
      backend:
 
  database:
    networks:
      backend:
        ipv4_address: 172.28.0.10
 
networks:
  frontend:
    driver: bridge
 
  backend:
    driver: bridge
    ipam:
      config:
        - subnet: 172.28.0.0/16
          gateway: 172.28.0.1

External Networks

YAML

services:
  app:
    networks:
      - shared-network
 
networks:
  shared-network:
    external: true
    name: my-existing-network

Secrets and Configs

Using Secrets

YAML

services:
  app:
    image: myapp
    secrets:
      - db_password
      - api_key
 
  postgres:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
 
secrets:
  db_password:
    file: ./secrets/db_password.txt
  api_key:
    environment: "API_KEY" # From environment variable

Note

Secrets are mounted as files in /run/secrets/<secret_name>. Many official images support *_FILE environment variables to read from secrets.

Using Configs

YAML

services:
  nginx:
    image: nginx
    configs:
      - source: nginx_config
        target: /etc/nginx/nginx.conf
        mode: 0440
 
configs:
  nginx_config:
    file: ./nginx/nginx.conf

Production Configuration

Production Compose File

YAML

# docker-compose.prod.yml
services:
  app:
    image: myapp:${VERSION}
    deploy:
      replicas: 3
      resources:
        limits:
          cpus: "0.5"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 256M
      restart_policy:
        condition: on-failure
        delay: 5s
        max_attempts: 3
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 30s
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "5"
 
  postgres:
    image: postgres:16-alpine
    volumes:
      - postgres-data:/var/lib/postgresql/data
    deploy:
      resources:
        limits:
          memory: 1G
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password

Logging Configuration

YAML

services:
  app:
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "10"
        compress: "true"
 
  # Or use external logging
  app-external:
    logging:
      driver: syslog
      options:
        syslog-address: "tcp://logs.example.com:514"
        tag: "myapp"
 
  # Or disable logging
  noisy-service:
    logging:
      driver: none

Resource Constraints

YAML

services:
  app:
    deploy:
      resources:
        limits:
          cpus: "2"
          memory: 1G
          pids: 100
        reservations:
          cpus: "0.5"
          memory: 256M

Development vs Production

Development Configuration

YAML

# docker-compose.override.yml (development)
services:
  app:
    build:
      context: .
      target: development
    volumes:
      - ./src:/app/src
      - ./tests:/app/tests
    ports:
      - "3000:3000"
      - "9229:9229" # Debug port
    environment:
      - NODE_ENV=development
      - DEBUG=app:*
    command: npm run dev
 
  postgres:
    ports:
      - "5432:5432" # Expose for local tools
 
  # Dev-only services
  mailhog:
    image: mailhog/mailhog
    ports:
      - "8025:8025"

Production Configuration

YAML

# docker-compose.prod.yml
services:
  app:
    image: myregistry.com/myapp:${VERSION}
    restart: always
    deploy:
      replicas: 2
    environment:
      - NODE_ENV=production
    # No volume mounts
    # No debug ports
 
  postgres:
    # No exposed ports
    restart: always
 
  # No dev services like mailhog

Environment Configuration Pattern

BASH

# Directory structure
project/
├── docker-compose.yml          # Base configuration
├── docker-compose.override.yml # Dev defaults (auto-loaded)
├── docker-compose.prod.yml     # Production overrides
├── docker-compose.test.yml     # Testing overrides
├── .env                        # Default env vars
├── .env.production             # Production env vars
└── .env.test                   # Test env vars

BASH

# Development (uses override automatically)
docker compose up
 
# Production
docker compose -f docker-compose.yml -f docker-compose.prod.yml \
  --env-file .env.production up -d
 
# Testing
docker compose -f docker-compose.yml -f docker-compose.test.yml \
  --env-file .env.test up

Compose Watch (Live Sync)

Docker Compose Watch syncs code changes automatically:

YAML

services:
  app:
    build: .
    develop:
      watch:
        # Sync source code
        - action: sync
          path: ./src
          target: /app/src
 
        # Rebuild on package changes
        - action: rebuild
          path: ./package.json
 
        # Sync and restart on config changes
        - action: sync+restart
          path: ./config
          target: /app/config

BASH

# Start with watch mode
docker compose watch
 
# Or
docker compose up --watch

Note

Compose Watch is ideal for development—it's faster than rebuilding for code changes while still rebuilding when dependencies change.

Include and Merge

Include Other Compose Files

YAML

# docker-compose.yml
include:
  - path: ./database/docker-compose.yml
  - path: ./monitoring/docker-compose.yml
 
services:
  app:
    image: myapp
    depends_on:
      - postgres # From included file

Merge Behavior

YAML

# Base file
services:
  app:
    image: myapp
    ports:
      - "3000:3000"
 
# Override file - merges with base
services:
  app:
    # image: inherited
    # ports: inherited
    environment:
      - DEBUG=true
    volumes:
      - ./src:/app/src

Troubleshooting Compose

Debug Configuration

BASH

# View resolved configuration
docker compose config
 
# View specific service
docker compose config --services
docker compose config --volumes
docker compose config --profiles
 
# Validate without running
docker compose config --quiet || echo "Invalid configuration"

Service Logs

BASH

# All logs
docker compose logs
 
# Follow specific service
docker compose logs -f app
 
# With timestamps
docker compose logs -t
 
# Last N lines
docker compose logs --tail 100
 
# Since time
docker compose logs --since 1h

Resource Issues

BASH

# View resource usage
docker compose top
docker stats
 
# Check disk usage
docker system df
 
# Clean up
docker compose down --volumes --remove-orphans
docker system prune -a

Quick Reference

Profile Commands

Command	Purpose
`--profile <name>`	Activate a profile
`--profile "*"`	Activate all profiles

Environment Commands

Command	Purpose
`--env-file <file>`	Use specific env file
`config`	Show resolved configuration

Dependency Conditions

Condition	Waits For
`service_started`	Container starts
`service_healthy`	Healthcheck passes
`service_completed_successfully`	Exit code 0

In the next chapter, we'll explore development workflows including hot reloading, debugging, and developer productivity with Docker.