Better pruning of docker volumes during make up/down

[KevinMind]

Fixes: mozilla/addons#ISSUENUM

Description

Make up produces several anonymous volumes that sometimes end up in a "dandling" state. This means they are no longer associated with a container and cannot easily be cleaned up by docker compose.

This PR modifies the existing "docker_clean_volumes" command to look for all dangling volumes that are not the database volume (don't want to clean that) and efficiently removes them.

Context

You can easily end up with dozens of volumes that take up space on disk without cleaning them up.

Testing

Make UP

Create a bunch of volumes

for i in {1..50}; do docker volume create volume$i; done

Now run make up to ensure your containers and non dangling volumes are running.

make docker_clean_volumes

Expect only the volumes associated with running containers to be there.

docker volume ls --filter dangling=true

should return an empty table.

Make DOWN

With already running containers (like above make sure make up is run) create more volumes (see above)

Now run make down

Expect no dangling volumes (except addons-server_data_mysqld. that should still be there)

Checklist

• Add #ISSUENUM at the top of your PR to an existing open issue in the mozilla/addons repository.
• Successfully verified the change locally.
• The change is covered by automated tests, or otherwise indicated why doing so is unnecessary/impossible.
• Add before and after screenshots (Only for changes that impact the UI).
• Add or update relevant docs reflecting the changes made.

[eviljeff]

Needs an issue 😉 - though I'm not sure I really understand the use-case, and why we need this clean-up script in our makefile - it seems more a general docker clean-up, and something that doesn't affect anything (other than a bit of lost disk space). Maybe @diox has other thoughts.

[KevinMind]

Needs an issue 😉 - though I'm not sure I really understand the use-case, and why we need this clean-up script in our makefile - it seems more a general docker clean-up, and something that doesn't affect anything (other than a bit of lost disk space). Maybe @diox has other thoughts.

It's not a bit... it can be dozens or over a hundred gigabytes of data. Essentially docker is not smart enough to clean up some of our orphan volumes and some of those contain huge data (our whole repository or memory cache from service X)

I regularly run into out of disk space errrors so finally decided to fix it.

The clean up script has been there for months, it just didn't work. Now it works.