Docker containers management#

Why#

Label everything, so you can:

• filter metrics (Prometheus) and logs (Loki) by service, env, version
• track the full chain: log/metric → container → image → git commit → code

Labels#

• Use Docker labels for all the metadata.
• Build labels: use the OCI standard keys, already understood by registries, scanners, Renovate, GitHub, …
• Run labels: define your own prefix to avoid collisions with other tools’ labels (e.g. acme_).

Image labels are inherited by the containers, so every container carries both build and run labels.

Build phase#

• Independent CI for building, better a shared one (reusable pipeline/template), so every service is built the same way.
• Label the image with repo, commit, tag and build date (the commit alone is not enough when you have many repos):

docker build \
  --label org.opencontainers.image.source="https://github.com/acme/orders" \
  --label org.opencontainers.image.revision="$(git rev-parse HEAD)" \
  --label org.opencontainers.image.version="$TAG" \
  --label org.opencontainers.image.created="$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  -t registry.example.com/orders:"$TAG" .

• Write the Dockerfile for layer cache: put what changes rarely first (base image, dependencies), the source code last.

FROM node:22-slim
WORKDIR /app
COPY package.json package-lock.json ./
# cached until dependencies change
RUN npm ci
# only this layer rebuilds on code changes
COPY . .
CMD ["node", "server.js"]

• Make the CI load the layer cache: CI runners usually start empty, so push/pull the cache to the registry.

docker buildx build \
  --cache-from type=registry,ref=registry.example.com/orders:cache \
  --cache-to type=registry,ref=registry.example.com/orders:cache,mode=max \
  ...

• Immutable tags: never re-push an existing tag and never use latest in production, a tag must always point to the same image. For full safety deploy by digest (orders@sha256:...).

Run phase#

Labels#

• acme_release_date: when it was deployed.
• acme_env: production, staging, …
• acme_service: a friendly service name, similar to the container name but independent from it, so multiple replicas (orders-1, orders-2) group under the same service.
• acme_host: the node/VM where the container is deployed.

services:
  orders:
    image: registry.example.com/orders:1.4.0
    labels:
      acme_service: orders
      acme_env: production
      acme_release_date: "${RELEASE_DATE}"
      acme_host: "${HOSTNAME}"

Container settings#

• Healthcheck: let Docker know if the app is really working, not just running.
• Restart policy: restart the container automatically when it crashes or the host reboots.
• Log rotation: the default json-file driver never rotates, logs can fill the disk. Set it once for the whole host in the daemon config (see below), and override it per service only if needed.
• If needed, think about adding CPU/memory limits, so a single container can’t starve the whole host.

services:
  orders:
    restart: unless-stopped
    healthcheck:
      # the command must exist in the image (curl, wget, ...)
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 5s
      retries: 3
    # optional, only to override the daemon default
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M

Default log rotation for every container on the host, in /etc/docker/daemon.json:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

Restart the daemon (systemctl restart docker) to apply it. It only affects newly created containers, so recreate the existing ones (docker compose up -d --force-recreate).

Using the labels#

• Prometheus (via cAdvisor): container labels are exposed as container_label_acme_service, container_label_acme_env, … Dots become underscores, so the OCI ones are long (container_label_org_opencontainers_image_revision): rename them with metric_relabel_configs if you want shorter names.
• Loki (via Promtail/Alloy docker_sd_configs): map __meta_docker_container_label_acme_* and __meta_docker_container_label_org_opencontainers_image_* to log labels. Keep high-cardinality values (commit, dates) out of Loki labels, put them in the log line or structured metadata instead.

Chain: alert on acme_service=orders, acme_env=production → container → org.opencontainers.image.source + .revision → commit on git → code.