Docker Memory Optimization: Preventing OOM Kills in Production

How Docker Memory Limits Work

Docker memory limits are enforced by Linux cgroups (control groups). When a container’s memory use exceeds the ‘–memory’ limit, the kernel OOM killer selects a process in the container to kill. If the container process dies, Docker restarts the container according to its restart policy. This produces a clean failure signal: ‘docker inspect –format={{.State.OOMKilled}}’ returns ‘true’ after an OOM kill.

The ‘–memory-swap’ flag controls the combined memory+swap allocation. With ‘–memory=2g –memory-swap=4g’, the container can use up to 2g of RAM and 2g of swap. With ‘–memory=2g –memory-swap=2g’, swap is disabled — the container can only use 2g of RAM and is killed if it exceeds it. With ‘–memory-swap=-1’, unlimited swap is allowed (not recommended for production).

Memory limits in Compose are set under ‘deploy.resources.limits.memory’. In Docker Compose v3, limits require the ‘deploy:’ section. They apply when deploying with ‘docker compose up’ on recent versions of Docker Desktop and Docker Engine that support the deploy key outside of Swarm mode.

The ‘–memory-swappiness’ flag controls how aggressively the kernel swaps container memory pages. A value of 0 tells the kernel not to swap unless absolutely necessary. A value of 100 means the kernel aggressively swaps pages to free physical memory. For production containers where latency matters, set ‘–memory-swappiness=0’.

Memory Configuration and Diagnostics

# Set memory limit with swap disabled
docker run -d \
  --name myapp \
  --memory=512m \
  --memory-swap=512m \
  --memory-swappiness=0 \
  myapp:latest

# Set memory limit allowing some swap (e.g., for burst tolerance)
docker run -d \
  --name myapp \
  --memory=512m \
  --memory-swap=768m \
  myapp:latest
  # This allows 512m RAM + 256m swap

# Docker Compose memory limits
services:
  app:
    image: myapp:1.2.3
    deploy:
      resources:
        limits:
          memory: 512M
        reservations:
          memory: 256M

# Check current memory usage for all containers
docker stats --no-stream --format \
  'table {{.Name}}\t{{.MemUsage}}\t{{.MemPerc}}'

# Check if a container was OOM-killed
docker inspect myapp --format='{{.State.OOMKilled}}'

# Check OOM kill status and restart count for all containers
for cid in $(docker ps -q); do
  name=$(docker inspect --format='{{.Name}}' $cid | tr -d '/')
  oom=$(docker inspect --format='{{.State.OOMKilled}}' $cid)
  restarts=$(docker inspect --format='{{.RestartCount}}' $cid)
  memlimit=$(docker inspect --format='{{.HostConfig.Memory}}' $cid)
  [ "$memlimit" = "0" ] && limit_str="NO LIMIT" || limit_str="$((memlimit/1024/1024))M"
  echo "$name: oom=$oom restarts=$restarts limit=$limit_str"
done

# Inspect memory use inside a running container
docker exec myapp cat /proc/meminfo
docker exec myapp cat /sys/fs/cgroup/memory/memory.usage_in_bytes

The cgroup file ‘/sys/fs/cgroup/memory/memory.usage_in_bytes’ shows the container’s actual memory consumption as tracked by the kernel cgroup, which may differ slightly from what ‘docker stats’ shows due to timing. This is the authoritative source for the container’s memory consumption relative to its cgroup limit.

Memory Optimisation Strategies

Diagnosing Memory Problems

Investigating a Memory Leak in a Running Container

A memory leak in a container shows up as steadily increasing memory usage in ‘docker stats’ over time. The first step is confirming the growth is in the application, not in OS buffers or caches.

# Watch memory usage over time (sample every 5 seconds)
watch -n5 'docker stats --no-stream --format "{{.Name}} {{.MemUsage}} {{.MemPerc}}" myapp'

# Inside the container — check process memory
docker exec myapp ps aux --sort=-%mem | head -20

# For Node.js: check heap usage
docker exec myapp node -e 'const v8=require("v8"); console.log(v8.getHeapStatistics())'

# For Python: if heapy or tracemalloc is available
docker exec myapp python3 -c 'import tracemalloc; tracemalloc.start()'

# Check if memory growth is in shared memory (tmpfs mounts)
docker exec myapp df -h | grep tmpfs
docker exec myapp cat /proc/meminfo | grep -E '(MemTotal|MemFree|Buffers|Cached|Shmem)'

# Check cgroup memory stats for detailed breakdown
docker exec myapp cat /sys/fs/cgroup/memory/memory.stat

Handling OOM Kills in Production

When a container is OOM-killed, the immediate response is to increase the memory limit and then investigate the cause. Repeated OOM kills without investigation indicate a real problem that memory limit increases are only delaying.

# Confirm the container was OOM-killed
docker inspect myapp --format='
OOM Killed: {{.State.OOMKilled}}
Restart Count: {{.RestartCount}}
Exit Code: {{.State.ExitCode}}
Finished At: {{.State.FinishedAt}}'

# Check host dmesg for OOM killer messages
sudo dmesg | grep -i 'oom\|killed process' | tail -20

# Immediate fix: increase memory limit on running container
docker update --memory=1g --memory-swap=1g myapp

# If the container has already stopped, change the Compose file
# and recreate:
# Edit deploy.resources.limits.memory in docker-compose.yml
docker compose up -d --no-deps myapp

# Root cause: is this a one-time spike or a leak?
# Monitor for 24 hours after the limit increase
watch -n30 'docker stats --no-stream --format "{{.Name}} {{.MemUsage}}"'

Memory Limits as a Security Control

Practical Examples

Memory Trending for OOM Prevention

#!/bin/bash
# Log memory usage every minute for trend analysis
# Run as a background cron job: * * * * * /opt/scripts/mem-trend.sh
LOGFILE="/var/log/docker-memory-trend.log"
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')

docker stats --no-stream --format '{{.Name}} {{.MemPerc}} {{.MemUsage}}' | \
  while read name pct usage; do
    echo "$TIMESTAMP $name mem_pct=$pct mem_usage=$usage"
    # Alert if any container is above 85% of its limit
    PCT_NUM=${pct/\%/}
    if awk "BEGIN{exit ($PCT_NUM < 85)}"; then
      echo "$TIMESTAMP ALERT: $name at $pct memory usage" | \
        tee -a /var/log/docker-memory-alerts.log
    fi
  done >> "$LOGFILE"

Tracking memory usage over time lets you detect gradual leaks (increasing trend over hours/days) versus load spikes (sharp increases that then decrease). An alert at 85% of the limit gives time to investigate before the container is OOM-killed. Adjust the threshold based on your alerting lead time requirements.

Quick Reference: Key Commands and Options

# Inspect current configuration
docker inspect <container> --format='{{json .HostConfig}}' | python3 -m json.tool

# View container resource usage
docker stats <container> --no-stream

# Check container logs
docker logs <container> --tail 50 --timestamps

# Verify running processes inside container
docker exec <container> ps aux

# Check container environment variables
docker exec <container> env | sort

These diagnostic commands work across all container scenarios: ‘docker inspect’ shows the full HostConfig including all runtime parameters, ‘docker stats’ shows real-time resource consumption, and ‘docker logs’ shows output from the main process. Use these as the starting point for any container investigation before diving into more specific tooling.

Common Memory Issues

Set Limits Based on Observed Use, Disable Swap, Alert at 85%

The three key memory configuration decisions for Docker containers are: set a memory limit on every container based on observed production memory use plus headroom, disable swap with ‘–memory-swap’ equal to ‘–memory’ to produce clean failure signals instead of slow swap degradation, and monitor memory percentage so you catch containers approaching their limits before the OOM killer does. Repeated OOM kills without an application-level fix are a symptom management pattern, not a solution.

• Set memory limits on every production container. A container without a limit can consume all host memory and trigger OOM kills across the entire host, not just in the container.
• Set ‘–memory-swap’ equal to ‘–memory’ to disable swap. Swap use in Docker containers produces slow, hard-to-diagnose performance degradation. An OOM kill is a clearer failure signal than indefinite swap-degraded performance.
• Monitor memory percentage with ‘docker stats’ and alert before reaching 100%. A container at 90% of its memory limit under normal load will be OOM-killed during any traffic spike. Increase limits proactively rather than reactively.