I often have issues when using latest software versions like for fast moving projects like Node.js. Since I am using Debian and that hosted packages are often stable versions, I need to install those packages manually. Some projects provide prebuilt packages to download from their website but others require manual installation with all dependencies, hoping that those will not conflict with the system and requiring to remember all manually installed dependencies when cleaning up the system. Latest versions of fast moving projects often change by nature, meaning I often need to manually update them.
Docker’s execution model looked interesting for isolating those in a container rather than installing them on the host system, then being able to start this container in milliseconds when needed. I saw people have been using that strategy for some time and gave it a try. The good news is that Docker being so popular those days, there qre endless up to date images for most projects.
The simplest thing people are doing is just to create an alias that boots a container with a volume to the host working directory and executes the command inside, all arguments simply being passed down:
Adding this alias in my
.bashrc solves my two issues:
- upgrading to a newer version just requires to change the image version in my alias
- cleaning up is just asking Docker to prune unused images
The first issue I encountered was that all files being created by the command were created as root. Solving that requires to have a user defined in the container with the same identifier than the host user, and running the container with this user. I thus created a script which creates a local image with the user properly set inside, and changed aliases to use this image instead:
Again upgrading to another version is just changing the image in the file and running it again.
The second issue I had was that this does not respect npm caching. npm is keeping all downloaded binaries locally and only checks metadata against the server when installing the same package over and over. For this I decided to use docker volumes, I created a data container that would keep the cache and reuse its volumes in running containers. In the generated image I had to create the folder containing npm cache with the correct user. For convenience I also created an alias not mounting cache volumes so I can force fresh installs if needed:
The volume is set to the whole home directory to also include
.npmrc that allows logging to npm.
That organisation is quite limiting as a new data container is created each time one creates a bash session, I thus had to split this in two scripts:
- one creating the image and data container each time it is invoked
.bashrcI kept a test for the data container existence (calling the other script if not) and aliases
Upgrading to a new version now requires that I change image version in the standalone script and runs it once.
I had the same working for Maven:
Even if pretty satisfied by the setup I encountered a few limitations. The obvious one is that it consumes more space on disk than just installing packages: my hard drive is way large enough to accomodate it. It also consumes a lot of bandwidth on setup to download all filesystem layers. Execution time does not seem to be affected by running inside containers.
The real limation I found was that those executables are actually isolated, they cannot rely on the presence of other executables on the system. For most cases this is exactly what I wanted: the software I write does not implicitly depends on things installed on my system. It is howerver a problem for npm commands, I used to have a few helpers relying on
git to deploy to Github pages: