Cleaning up a full Docker partition

Docker is great when it comes to isolating applications and services but like many other things, it has some serious drawbacks.

One of the issues that repeatedly bites me is the massive amount of disk space it may use. In rare occasions it can even make your system run out of disk space. On top of this Docker lacks a decent system to clean I'ts mess up, so fixing it isn't as easy as it should be. Fortunately, solving the problem isn't rocket science and the solution is rather simple (scroll down for copy+paste).

During some maintenance on svenv.nl I noticed that Docker did stole all my disk space. A docker's run script (the script I launch at runtime) crashed with the following message:

 * /etc/init.d/mysql: ERROR: The partition with /var/lib/mysql is too full!

I returned to the command line of the host and did a quick check on the disk usage, this confirmed the issue:

df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        49G   46G     0 100% /

Whenever this happens, I usually run ncdu (apt-get install ncdu) to find what’s using so much space. Because I already suspected Docker from wasting disk space (and ncdu'ing may take up some time) I checked /var/lib/docker first.

ncdu /var/lib/docker

The results were clear, in my case, the container diffs took up all the space:

--- /var/lib/docker ------------------------------------------------------------
   41.2GiB [##########] /aufs
   17.4MiB [          ] /graph
   14.3MiB [          ] /init
  400.0KiB [          ] /containers
  148.0KiB [          ] /volumes
   60.0KiB [          ] /execdriver
   48.0KiB [          ] /vfs
   40.0KiB [          ]  linkgraph.db
    8.0KiB [          ] /trust
    8.0KiB [          ] /tmp
    4.0KiB [          ]  repositories-aufs

The solution is to remove old containers and images. All the instructions to re-build them is defined in Dockerfiles so removing old container shouldn't be a problem.

docker ps -a -notrunc | grep 'Exit' | awk '{print $1}' | xargs -r docker rm
docker images -notrunc | grep none | awk '{print $3}' | xargs -r docker rmi

After running these commands disk usage returned back to normal and the application started as usual.

Keep it clean!