BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Articles Reproducible Development with Devcontainers

Reproducible Development with Devcontainers

Bookmarks

Key Takeaways

  • Devcontainers are not deployment containers: containers for deployment have very different needs than containers for development
  • Turning onboarding into a nonevent is one of the most obvious and immediate benefits of developing in containers
  • Devcontainers support the full testing cycle as they package up everything needed to develop the application
  • Since a devcontainer is dedicated to a single project, it can have the correct version of Ruby, Python, or JavaScript installed globally removing the need for complicated virtual environments
  • Devcontainers can help normalize the best development practices across individuals and teams

Developing from a Chromebook at the car wash

So, the other day I took my car to the car wash. It's one of those fancy car washes where you hand your car over to the attendants and then go and wait while they clean it inside and out.

That gave me  some time to kill. And I had some coding work I was itching to make some progress on. But all I had was a WiFi connection and the tiny Chromebook I'd thrown in my bag.

So I brought up the project on GitHub, opened it in GitHub Codespaces, and started up right where I'd left off, with my full development environment running in the cloud.

Not just the editor... a whole virtual machine, customized for my project.

I proceeded to make progress on my project. Total time lost to getting my development environment set up on on a new (cloud) machine: maybe five minutes.

Now, CodeSpaces are cool, but this article isn’t actually about them, or even about cloud-based development more generally. This article is about the technology and practices that enables developers to spin up a whole project-customized development environment from zero, in a few seconds or minutes. Whether that’s:

  • On a new-hire’s laptop on their first day
  • On a secondary travel laptop
  • On the workstation of a designer who needs to be able to try out visual changes locally across several in-house codebases… without being familiar with the backend tech stacks.
  • On a consultant’s laptop that plays host to a dozen different, unrelated codebases at at time
  • ... or in a fresh shared instance in the cloud.

What if  instead of being a minor ordeal, initial project setup was a non-event? What if you could package your development environment right along with your code? What if you could level the playing field on your team, such that everyone benefits from those teammates who tweak their environment for maximum efficiency? What if the friction of having the project fail to build on just one developer’s laptop were a thing of the past?

What if you could contribute a few lines of code when you’re bored at the car wash? 

This is the future of developer experience, one that you can start enjoying right now. By using containers for development: sometimes known as devcontainers.

What is a devcontainer?

Let’s start with a definition When I say "container", I'm talking about the kind of container you typically run with Docker. The implication here is also that your project runs natively in Linux. Which is true of most web application development these days. But if you're targeting iOS, or the Windows Desktop, or some other non-UNIX-like platform, what follows may be less applicable to your project.

Now, this article is not an introduction to Docker. For length reasons, I'm going to assume that you have some basic familiarity with containerization.

However, it's worth very briefly talking about what makes containers so much better-suited to developing inside them than some of the older virtualization technologies like Parallels, VirtualBox, or Vagrant.

In a nutshell, it's because containers aren't virtualization at all. Yes, containers give us something that looks like a tiny computer inside a computer. But rather than trying to simulate a computer, a container works by creating an isolated set of namespaces. Including the filesystem namespace, network ports, the process table, and all the other namespaces that go into a running operating system.

Which means that unlike virtualization, containers have the potential to run project code and tools at native speeds without bringing a development machine to its knees. And because the host operating system can map files into the container namespace, we can edit source code using native tools while running the code inside the container.

Also unlike most virtualization technology, containers aren't opaque binary images to be passed around. We determine what Linux version, system packages and libraries, utilities, filesystem mappings, open ports, and supporting services will go into a devcontainer using human-readable configuration files that get versioned right alongside the project's source code.

In effect, a devcontainer is a fully functional, batteries-included development environment that is shared, versioned, reproducible, self-documenting, and always up-to-date so long as it's in use. A devcontainer is like the Ramen noodles of development environments: just add hot water and you're ready to go.

This article is also not a tutorial. Building out a full devcontainer is an ongoing iterative process; one that's very specific to your project. Instead, I'm going to give you a tour of what a devcontainer can look like, and what it can feel like to use one, to be part of a team with a reproducible development environment.

The Devcontainer Experience

Why are containers the development environment of the future? Let's look at some examples of advantages we get from a devcontainer

Instant onboarding

Recently I joined a client for a six-month engagement. Like most teams with a large, old project, they had a lengthy set of initial setup instructions and scripts scattered across their README and wiki pages. As always, parts of the directions were outdated or contradictory. The setup scripts had very specific expectations that they'd be running on a brand-new MacBook of a particular era, with a particular version of MacOS, and that this laptop would be dedicated to developing on this project---so it was fine to make global configuration changes to it.

This confusing, lengthy, and constantly outdated onboarding process is the norm on most teams I've seen. During your first full week of onboarding, if you can get part of the project's test suite running, you're doing pretty well!

As my very first project on joining this team, I created a devcontainer configuration that turned all of this documentation into executable configuration.

To do this, I made a set of configuration files for Docker, separate from docker configuration files used in creating deployment containers.  They're in a .devcontainer directory in the project repository.

.devcontainer/
├── Dockerfile
├── README.md
├── devcontainer-load-profile.sh
├── devcontainer.json
├── docker-compose.yml
├── entrypoint.sh
├── init-once.sh
├── init.sh
└── profile.sh

At the very least, it usually includes a docker-compose configuration file, which defines what container or containers to start up, and how to connect them to each other and to the host computer.

version: "3.2"
services:
app:
  user: developer
  build:
    context: .
    dockerfile: Dockerfile
  volumes:
    - type: bind
      source: ..
      target: /workspace
    - type: bind
      source: ${HOME}${USERPROFILE}/.ssh
      target: /home/developer/.ssh
  working_dir: /workspace
  command: sleep infinity
  environment:
    BUNDLE_PATH: vendor/bundle
    INTERFACE: "0.0.0.0"
  ports:
    - "3000:3000"

Typically there's also a Dockerfile to customize the app development container.

FROM ruby:2.7.2
RUN apt-get update \
&& apt-get install -y yarnpkg vim lsof \
&& ln -s /usr/bin/yarnpkg /usr/local/bin/yarn \
&& rm -rf /var/lib/apt/lists/*

COPY sixmilebridge-load-profile.sh /etc/profile.d/

ARG USERNAME=developer
ARG USER_UID=1000
ARG USER_GID=$USER_UID

RUN groupadd --gid $USER_GID $USERNAME \
  && useradd -s /bin/bash --uid $USER_UID --gid $USER_GID -m $USERNAME

If we're using VS Code, there's also a devcontainer.json. Devcontainers tend to accumulate some script files as well, that are hooked into various points in the container lifecycle.

.devcontainer/
├── Dockerfile
├── README.md
├── devcontainer-load-profile.sh
├── devcontainer.json
├── docker-compose.yml
├── entrypoint.sh
├── init-once.sh
├── init.sh
└── profile.sh

Once I had finished creating this devcontainer definition, I committed all of it to the project repository.

At first, some of the folks on the team there were like: "well, we're never going to use it…. but you do you!". 

Then a funny thing happened. While I was there, several new hires used the devcontainer to get up and developing more or less instantly. Someone else from another team used the devcontainer to make PRs on a codebase they didn't usually work on, without having to spend a week getting it set up. By the time I moved on, the devcontainer had become one of my most lasting and appreciated contributions.

Turning onboarding into a nonevent is one of the most obvious and immediate benefits of developing in containers. And it's not just for new-hires. It could mean someone from your front-end team is able to jump in and make tweaks to your backend app code. It could mean you, three years from now, being able to quickly come back and fix a bug.

Share essential utilities

Project setup checklists and scripts quickly go stale, because once we have the project configured on a machine we never think about them again. But devcontainers are regularly rebuilt, whenever anybody tweaks them. A devcontainer is executable documentation of what libraries, services, system configurations, open ports, nice-to-have utilities, etc. go into day-to-day development with your project.

For instance, does your team sometimes use ngrok to expose a local development machine to a remote user? Don't write the setup instructions in the wiki... add it to the devcontainer.

RUN wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip \
  && unzip ngrok-stable-linux-amd64.zip \
  && mv ngrok /usr/local/bin \
  && rm ngrok-stable-linux-amd64.zip

Then everyone who uses the devcontainer will have the right tools when they need them.

vscode ➜ /workspace (main ✗) $ ngrok --version
ngrok version 2.3.40

Everyone can run all the tests all the time

I've seen more and more projects where you're considered to be doing well if you can run just the unit tests locally---but only the CI system has all the right magical invocations and extra supporting services to run the system or integration tests. In the extreme case, only a select few infrastructure gnomes know how to fix the system tests when they don't work... which can leave developers twiddling their thumbs when their changes break the build.

With a devcontainer, one that everyone shares and that's also used in CI, we can upgrade our expectations to: everyone can run all the tests, all the time. They might still run faster in parallel on CI, but keeping the integration tests passing becomes everyone's business.

Devcontainers can support the full testing cycle because they are able to package up, not just a tiny computer for developing on the app itself, but also the constellation of supporting services needed to run the app. Does the app need a redis server and a particular version of PostgreSQL with specific extensions installed? A docker-compose configuration can ensure that these are spun up, available and connected when the devcontainer is started.

It can even encode wizardly tweaks from the Postgres expert on the team, to optimize the development database server to optimize it for responsiveness over reliability.

redis:
  image: redis:${REDIS_VERSION:-6.0.9}
postgres:
  image: mdillon/postgis:${POSTGRES_VERSION:-9.6}
  command: ["-c", "fsync=off", "-c", "full_page_writes=off", "-c", "synchronous_commit=off"]
  environment:
    POSTGRES_USER: ${POSTGRES_USER:-postgres}
    POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-password}

Cleanly switch between projects

Speaking of which, have you ever had to install a specific system library or version of PostgreSQL to satisfy one app, only to have that break another app you were working on? With devcontainers, you can cleanly switch between multiple projects on one machine. This is essential for consultants, but it's applicable to any organization that has more than one codebase.

And while we're talking about switching between projects... if you're used to working with languages like Python, Ruby, or JavaScript, you're used to having to deal with version managers like VirtualEnv, RVM, or NVM. These tools build and install and manage multiple versions of Python or Ruby or Node side-by-side and ensure that each project uses the just the right version of the language runtime. In the process, they add an extra level of indirection. They are a hassle at the best of times, and when you're dealing with a language ecosystem you're unfamiliar with, they can be an added obstacle to becoming productive.

Using devcontainers eliminates this entire class of utility. Since a devcontainer is dedicated to a single project, it can have the correct version of Ruby, Python, or JavaScript installed globally. If this means compiling the runtime from source, that can be rolled into the devcontainer's Dockerfile. I haven't had to touch a language version manager since I started using devcontainers for everything, and I don’t miss it!

Share shortcuts

Over time a lot of project teams evolve a "standard" set of shell aliases and git aliases that shorten common actions. Some of them are basic aliases that are applicable to any project, but there are often a few shortcuts that are very specific to how one team works with their app.

alias gs="git status"
alias be="bundle exec"

Usually, these just get spread by someone on the team evangelizing them until others slowly adopt them. It can be jarring to be pairing on some code and then realize that the shortcuts you're used to aren't there.

The presence or absence of these shortcuts can also lead to a subtle social partitioning of the team into the "cool kids" who always have the best shell aliases, and the uncool kids who lag behind.

What if anyone on the team could instantly add a useful shell alias for everyone else? That's exactly what we can do when we're all using a devcontainer. Instead of posting a shell alias in Slack, you can make a PR that adds it for everyone, and then show off how to use it in Slack. And since the devcontainer contains a common, shared UNIX userspace, you can be sure that those shortcuts will work for everyone.

Debug more effectively

Devcontainers provide some less obvious perks as well. A container is, by definition, a highly controlled environment. It's almost like having a tiny network of computers under a microscope. I don't have space to get into details in this article, but devcontainers make it easier to do what I call investigative debugging: instrumenting the file writes, network I/O, and even system calls of an application in order to understand exactly what it is up to.

Reproduce problems

One of the biggest benefits of devcontainers shows up once most people on a team are using it. Have you ever had one developer on your team suddenly start having an issue that no one else sees? Eventually, after a lot of troubleshooting, it turns out that they received a system update that was incompatible with one of the libraries the project depends on. And no one knew how to help them, because... it worked on their machine!

Consistently using devcontainers can drastically cut down on the "works on my machine" phenomenon. Nothing can make individual developer environments perfectly identical, but having a common container definition can eliminate a huge number of potential variables. And once you nail down whatever library update broke the project, you can easily fix it. Because with a container, you can get as specific and locked-down as you need to get with utilities and system library versions.

Code in the cloud

And once you've got a devcontainer definitions, you're not limited to "my machine" at all!

Cloud-based development using tools like Gitpod, Amazon Cloud 9, JetBrains Space, or GitHub Codespaces is a thing now, and it's only going to become more of a thing.

Cloud-based development environments enable remote pair-programming. They give you the ability to drop in and write some code anywhere you have a browser... even if you accidentally left your laptop bag on the train. And if you have a devcontainer definition that works locally, you can fire up an IDE in the cloud.

Devcontainers are very applicable to open-source work as well. Have you ever wanted to contribute a small change to an open-source project? But when you pulled the code down, you realized it was a long and involved setup process in order to get the unit tests running? And so you gave up and dropped a suggestion in their bug tracker instead?

What if open-source projects came with devcontainers that could make casual contributors immediately productive? It might feel a lot more inviting to casual contributors!

Devcontainers in VS Code

Let's talk about editors and IDEs.

IDEs are starting to add features to embrace container-based development. VS Code, the open-source editor from Microsoft, is definitely at the forefront of this trend. In fact a lot of my thinking about devcontainers, including the term "devcontainer" itself, has been inspired by the way VS Code tightly incorporates container support. But more editors and IDEs are adding container-awareness.

Tight integration of a devcontainer with a devcontainer-aware IDE can help make developers more effective as soon as they start working on a project. For instance, it's common these days for a project to have linting or formatting rules customized for the codebase. But traditionally, developers new to the project would have to install the linting tool and make sure their editor was correctly configured to use it. With a container-aware editor configuration, linting and code formatting is working out of the box as soon as a developer fires up the project for the first time.

That's not to say that a devcontainer locks you down to all using the exact same editor configuration. Far from it! For instance, when using VS Code, the devcontainer can include a base level of project-specific settings and plugins, but you can also layer your own settings, plugins, and color schemes, keybindings, etc. on top of that.

For that matter, there's no rule that your team has to settle on a single editor either. A project can incorporate devcontainer-aware configurations for multiple IDEs. Heck, you could include a full VIM setup right inside the devcontainer, including the editor itself!

Devcontainers aren't Deployment Containers

Now, no tool or technique is a panacea. In a minute I want to talk about some cases where you might not want to use devcontainers. But before I get there, I want to share one of the biggest tripping-points I've seen in rolling out devcontainers for a project.

Often, I’ve run into the objection: "we already have a container definition, can't we re-use it?". Or , on the flip side: "yeah, this devcontainer stuff doesn't apply to us, because we're not using containers to deploy".

I think both of these points of resistance stem come from a false premise: the idea that containers are always for deployment. And this misconception is understandable. If you're already using containers at all in your project, it's probably because that's how you're deploying your application. You may even be using containers in your continuous integration infrastructure. Isn't that what containers are for?

It's true that deployment is the use-case that has popularized containers. But devcontainers are useful whether you are deploying containers or not! And in fact, thinking of devcontainers as fancy deployment boxes is a good way to miss out on most of their power.

Here's the thing: containers for deployment have very different needs than containers for development. In fact, a lot of the pressures on deployment containers are almost diametrically opposed to the pressures on devcontainers.

We want deployment containers to be as small and stripped-down as possible. We want them to be lean, fast, and security hardened. That means minimizing nonessential libraries and tools. It may mean using a base image such as Debian Slim, or even Alpine Linux, which lacks the usual glibc libraries found in ordinary Linux distributions. But for a devcontainer, you're trying to provide a full, comfortable development environment. That means a batteries-included Linux distro like Ubuntu, complete with command-line tools, compilers, manpages, and the whole kit and kaboodle!

For deployment, you want to minimize the security cross-section. For development, you want maximum ports open for debugging etc.! For deployment, it's an error NOT to be talking to observability services like Honeycomb or New Relic. In development, you don't want to be sending messages to those services, and you may want to "fake out" or stub-out some external services. In deployment you want to optimize your Docker builds for the smallest number of layers, while in development you may want to optimize for quickly adding incremental changes that don't require a full image rebuild.

In these and other ways, the goals of deployment containers and devcontainers are opposed to each other. That's why when I start at a new client and begin building a devcontainer, I normally start from scratch. I build a brand new set of container configuration files, working from the project setup instructions rather than from any existing Dockerfiles. This gives me a portable, reproducible environment that's built for development... not for deployment.

Now, this doesn't mean that your devcontainer and deployment container configurations can't share some parts in common. I don’t have space to cover that here. But here's a hint: you'll probably find that it's easier to start with your devcontainer and strip it down to a deployment container, than it is to start with a deployment container and build it up into a comfortable development environment.

Counter-indications for devcontainers

With all this said, devcontainers aren't right for every project.

Everything we've talked about is predicated on running containers in Docker, which is a Linux-based technology. Most web and enterprise applications are deployed to Linux-based servers these days, so developing in a container means developing in something close to the delivery environment. The same goes for Android development. But if your deployment target is not a Linux or Linux-like system, you may not want to go down this road. If you are targeting, say, iOS devices or Windows Native, containerized development may not be the best investment for you.

Also, as of 2021, there's a clear hierarchy in desktop platforms for Docker-based development.

Running Docker on a Linux-based machine is probably the best experience, since Linux is the native host for containers.

Perhaps surprisingly, the next-best option these days is Windows. That's because with the advent of the Windows Subsystem for Linux version 2, or "WSL2", Windows now runs Linux natively in parallel with the Windows kernel. You can actually pick your Linux distro of choice right out of the Windows store, and start running Linux binaries straight from the Debian or Fedora repositories without any recompilation or emulation.

Docker Desktop on Windows uses WSL2 for its backend. Which means that Docker containers on Windows are effectively running in their native Linux habitat, with no virtualization performance penalty. In my usage, it's stable and runs Rails projects at native speeds.

MacOS is built on BSD, not Linux, and it doesn't have a WSL2 equivalent. Which means that some level of virtualization is involved in order to make Docker work. I don't use Macs for development anymore. But I've heard from friends that they experience some flakiness and performance issues with Docker, particularly around file I/O.

What to do about this? Fortunately, it’s a known problem that Docker and Apple are incentivized to fix. And as a matter of fact, as of when I’m drafting this article, Docker recently announced some major updates addressing MacOS performance. With any luck this caveat will soon be the most dated aspect of this article!

Get in coder, we're developing in containers now

There are certain technologies that, once they mature, change the development state of the art forever.

Back when I started programming, version control still wasn't universally embraced. Some projects still relied on periodically zipping-up copies of the code for history. Over the course of my career, version control became universal. More recently, continuous integration went from being a novel new idea, to being an industry standard. Today, both distributed version control and continuous integration are table stakes: we can barely imagine a software project without them.

In 2021, we're at the beginning of this inflection point for development in containers. In five years we'll laugh about how we used to think it was normal to spend days getting our developer laptop set up before making our first commits to a project. But you don't have to wait that long. With a little effort, you can already have all the benefits of devcontainers for yourself and your team.

You can have a portable, reproducible development environment that follows you from machine to machine, and even into the cloud. You can get new hires up-and-running in an hour, instead of days. You can make it easier to contribute to your open-source projects. You can make sure that every test that runs in CI can also be run locally. You can share your specialized development configurations and scripts with your teammates with a push to GitHub. You can do all this by committing to making devcontainers a normal part of your project's development workflow.

Conclusion

So that's why I think you should drop everything and create a devcontainer definition for your current project. And not only that, you should work inside that devcontainer and improve it until it's so comfortable it feels like home. Your collaborators will thank you, and your future self will thank you.

About the Author

Rate this Article

Adoption
Style

BT