Docker Persistent Storage for Production Services

Why Stateful Services in Docker Require More Deliberate Configuration

Stateless containers are the natural fit for Docker — replace the container, the service continues as before. Stateful containers (databases, file storage, session stores) require the data to outlive the container. This changes the entire operational model.

The failure sequence for an improperly configured stateful container is always the same: the database runs correctly for days or weeks; a deployment replaces the container; all data is gone. The cause is invariably the same: data was stored in the container’s writable layer rather than a named volume. The writable layer is discarded on ‘docker rm’. Named volumes are not.

Beyond data persistence, stateful services in Docker face three additional production challenges: UID ownership mismatches between the container process and the volume directory cause the database to fail to start; file locking behaviour of database engines (fsync, advisory locks, WAL) may interact unexpectedly with some storage backends; and backup procedures must operate independently of the container lifecycle, because a backup script that stops the container to create a consistent snapshot also takes the service offline.

Each of these challenges has a standard solution. The goal of this article is to provide those solutions in a directly applicable form for the most common stateful services: PostgreSQL, MySQL, Redis, and file upload storage.

Storage Configuration for Production Stateful Services

# PostgreSQL with named volumes and correct configuration
# Create volumes first (makes ownership management explicit)
docker volume create pg-data
docker volume create pg-wal

# Pre-set ownership to match PostgreSQL container UID (999)
docker run --rm \
  -v pg-data:/data \
  -v pg-wal:/wal \
  alpine sh -c 'chown -R 999:999 /data /wal'

# Run PostgreSQL with all volume mounts
docker run -d \
  --name postgres \
  --restart unless-stopped \
  --memory=2g --memory-swap=2g \
  --stop-timeout 120 \
  -v pg-data:/var/lib/postgresql/data \
  -v pg-wal:/var/lib/postgresql/wal_archive \
  -e POSTGRES_USER=appuser \
  -e POSTGRES_DB=appdb \
  -e POSTGRES_PASSWORD_FILE=/run/secrets/pgpass \
  --tmpfs /run/secrets:size=1m \
  --health-cmd='pg_isready -U appuser -d appdb || exit 1' \
  --health-interval=30s \
  --health-start-period=30s \
  postgres:15

# MySQL with named volume
docker volume create mysql-data
docker run -d \
  --name mysql \
  --restart unless-stopped \
  --memory=1g --memory-swap=1g \
  --stop-timeout 60 \
  -v mysql-data:/var/lib/mysql \
  -e MYSQL_ROOT_PASSWORD_FILE=/run/secrets/mysql_root \
  -e MYSQL_DATABASE=appdb \
  --tmpfs /run/secrets:size=1m \
  --health-cmd='mysqladmin ping -h localhost --silent || exit 1' \
  --health-interval=30s \
  mysql:8.0

# Redis with persistence enabled
docker volume create redis-data
docker run -d \
  --name redis \
  --restart unless-stopped \
  --memory=512m --memory-swap=512m \
  -v redis-data:/data \
  redis:7 redis-server --appendonly yes --appendfsync everysec \
  --health-cmd='redis-cli ping || exit 1' \
  --health-interval=15s \
  redis:7

The –appendonly yes flag enables Redis AOF persistence. Without it, Redis only writes RDB snapshots and may lose up to 5 minutes of data on a crash. –appendfsync everysec provides a balance between durability (maximum 1 second of data loss) and performance (one fsync per second rather than one per write).

Storage Configuration Decisions

Production Database Backup and Restore Procedures

PostgreSQL Logical Backup with pg_dump

For most production PostgreSQL deployments, pg_dump provides consistent logical backups without taking the database offline. It captures a consistent snapshot of the database state at the moment the dump begins, even while other transactions continue.

# Live backup using pg_dump (no downtime)
docker exec postgres pg_dump \
  -U appuser \
  -d appdb \
  --format=custom \
  --compress=9 \
  > /backup/appdb-$(date +%Y%m%d-%H%M%S).pgdump

# Verify the backup is not empty
ls -lh /backup/*.pgdump | tail -1

# Restore from pg_dump backup
# Create a new database first (or restore to an existing one)
docker exec postgres psql -U appuser -c 'CREATE DATABASE appdb_restore;'
docker exec -i postgres pg_restore \
  -U appuser \
  -d appdb_restore \
  --no-owner \
  --role=appuser \
  < /backup/appdb-20260101-0000.pgdump

# Physical backup with pg_basebackup (for large databases)
docker exec postgres pg_basebackup \
  -U replication \
  -D /backup/pg_basebackup-$(date +%Y%m%d) \
  -Ft --gzip --checkpoint=fast -P

Volume-Level Backup (Database Stopped)

For databases that cannot be dumped while running (or for full filesystem-consistent backups), stop the database container first, take the volume backup, then restart. This approach works for any database engine and produces a bit-for-bit copy of the data directory.

#!/bin/bash
# Volume-level backup with minimal downtime
DB_CONTAINER=postgres
VOLUME_NAME=pg-data
BACKUP_DIR=/backup

echo "Stopping $DB_CONTAINER for consistent backup..."
docker stop --time=120 $DB_CONTAINER

echo "Creating archive of volume $VOLUME_NAME..."
docker run --rm \
  -v $VOLUME_NAME:/source:ro \
  -v $BACKUP_DIR:/dest \
  alpine tar czf /dest/pg-data-$(date +%Y%m%d).tar.gz -C /source .

echo "Restarting $DB_CONTAINER..."
docker start $DB_CONTAINER

# Wait for health check to pass
for i in $(seq 1 12); do
  status=$(docker inspect --format='{{.State.Health.Status}}' $DB_CONTAINER 2>/dev/null)
  [ "$status" = "healthy" ] && echo "Database is healthy. Backup complete." && exit 0
  echo "Waiting for healthy status ($i/12)..."
  sleep 5
done
echo "WARNING: database did not reach healthy status after 60 seconds"

Data Security for Containerised Databases

Practical Examples

Complete PostgreSQL Production Setup with Backup

#!/bin/bash
# Complete PostgreSQL production deployment

# 1. Create volumes
docker volume create pg-data
docker volume create pg-backups

# 2. Set correct ownership (PostgreSQL UID is 999 in official image)
docker run --rm -v pg-data:/data alpine chown -R 999:999 /data

# 3. Create secret (tmpfs)
mkdir -p /run/pg-secrets
echo 'very-strong-password-here' > /run/pg-secrets/pgpass
chmod 600 /run/pg-secrets/pgpass

# 4. Run PostgreSQL
docker run -d \
  --name postgres \
  --restart unless-stopped \
  --memory=2g --memory-swap=2g \
  --stop-timeout 120 \
  -v pg-data:/var/lib/postgresql/data \
  -v /run/pg-secrets:/run/secrets:ro \
  --network app-backend \
  --health-cmd='pg_isready -U appuser || exit 1' \
  --health-interval=30s \
  --health-start-period=60s \
  -e POSTGRES_USER=appuser \
  -e POSTGRES_DB=appdb \
  -e POSTGRES_PASSWORD_FILE=/run/secrets/pgpass \
  postgres:15

# 5. Schedule daily backups via cron
# 0 2 * * * docker exec postgres pg_dump -U appuser -Fc appdb > /backup/pg-$(date +\%Y\%m\%d).pgdump 2>/var/log/pg-backup.log

This setup uses a host-mounted secrets directory (:ro) rather than env variables for the password, ensuring it does not appear in docker inspect output. The network is set to app-backend only — no port publishing means the database is unreachable from outside the Docker network.

Quick Reference: Key Commands and Options

# Inspect current configuration
docker inspect <container> --format='{{json .HostConfig}}' | python3 -m json.tool

# View container resource usage
docker stats <container> --no-stream

# Check container logs
docker logs <container> --tail 50 --timestamps

# Verify running processes inside container
docker exec <container> ps aux

# Check container environment variables
docker exec <container> env | sort

These diagnostic commands work across all container scenarios: ‘docker inspect’ shows the full HostConfig including all runtime parameters, ‘docker stats’ shows real-time resource consumption, and ‘docker logs’ shows output from the main process. Use these as the starting point for any container investigation before diving into more specific tooling.

Persistent Storage Failures

Three Requirements Before Any Database Goes Into Production in Docker

Every database container in production needs three things: a named volume for the data directory (not an anonymous volume, not the container writable layer), correct UID ownership on the volume before first start, and a backup procedure that runs independently of the container and is tested for restore. None of these are defaults. All three must be explicitly configured before the first write reaches the database.

• Named volumes (not anonymous volumes) must be used for all database data directories. Verify with ‘docker inspect –format={{range .Mounts}}{{.Name}}{{end}}’ — an empty name indicates an anonymous volume that will be lost on docker rm -v or prune.
• Set –stop-timeout to 120 seconds for PostgreSQL and 60-120 seconds for MySQL/InnoDB. The default 10-second timeout causes unclean shutdowns, which trigger crash recovery on the next start and add startup latency.
• Test backup restore before the first production incident. A backup that has never been restored is a hypothesis, not a backup. Schedule monthly restore tests to a separate container and verify data integrity.