Developing in a team: shared environments

Part of the personal growth as a software engineer involves learning to work in a team. This is a natural step in anyone’s career which extends to pretty much any other craft. On the other hand, our profession offers such a vast degree of technical enablement that we are flooded with programmatic tools to automate most tasks, down to the tiniest detail. Despite the rich variety of our ecosystem, however, there are still some areas that need improvement. For example, architecture as a code has only been emerging in the past few years and it’s not uncommon to still rely on haphazard shell scripts glued together by magic. In these areas, software development still needs to grow. Today, I want to focus on something that isn’t often perceived as code: development environments.

What is a development environment?

With this term, I collectively refer to all tools, programs, scripts and utilities that a developer uses on a daily basis in their job. Some examples include:

• IDEs like IntelliJ or VSCode
• Runtimes like node or python
• Command line utilities like jq or curl
• Frameworks like nestjs or SprintBoot
• Containerization services like docker or containerd
• Orchestration ensembles like docker-compose or k8s

One development environment to rule them all

Within a team, developers have different backgrounds, experience level, preferences and opinions. It naturally follows that each person is more comfortable with a different set of tools. This is perfectly understandable, but it may rapidly evolve into a blocker for day-to-day activities. Like with standards, the problem is that no one solution is widely accepted.

This leads to fragmentation, which in turn leads to an explosive increase in complexity. Just image a team where:

• Alice works on a Windows 10 machine, runs node 12, uses VSCode and connects to a local postgres 10 instance.
• Bob instead prefers linux so his system of choice is Ubuntu 18.04, runs node 14, uses IntelliJ IDEA and spins up a containerized docker postgres 13 image.
• Meanwhile, Carl works on MacOS Sierra, runs node 10, uses Emacs and connects to a postgres 12 container running inside a microk8s cluster.

It’s trivial to notice that, whereas each development environment serves essentially the same purpose, no two configurations are the same. Alice, Bob and Carl will write slightly different code that caters to their own specific setup. Fine details that are specific to one framework or another, one runtime or another, are extremely hard to notice and may cause subtle bugs.

As software engineers, we know very well that our local environment will never be perfectly equivalent to a production environment and we learn to live with that. This discrepancy is enough to keep us up late at night to fix all kinds of odd behaviours. Adding inconsistencies caused by different development setups to the mixture is like begging for trouble. As long as there are variables that we can control, we should take the opportunity to do so. That’s the eternal quest of complexity management.

For the reasons mentioned above, encouraging a consistent development environment across all members of a team contributes to lowering complexity, fosters code cohesion and reduces time spent on troubleshooting.

Development environment as a code

The paradigm of “X as a code” has been a popular trend in the past years. Personally, I think the more we can control automatically the better. Relying on manual setup makes us prone to more trivial mistakes and while code isn’t necessarily bug-proof, at least it’s reproducible and steadfast. That’s why translating a development environment into code is good practice to manage team projects. Here’s several different ways to achieve a higher level of automation.

Write a setup script

Many teams at Mind Foundry share this practice, which is especially useful for bringing new hires up to speed with our current development process. We maintain several setup scripts, one for each project. The scripts can be executed on supported systems and will install all relevant tools we expect developers to use in the day-to-day development cycle, with a very strict policy on accepted versions. On top of that, they will provide all necessary configurations for those tools to run consistently. At the moment, we maintain scripts that can be run on either linux or windows, with the latter requiring more manual approaches in some cases.

Some of the perks we get:

• Enforcing the same version for runtimes like node.
• Ensuring all developers run the same browser so that frontend code can be more homogeneous. We also support only one browser for several products, so that is not a limitation on our part.
• Ensuring git is configured properly on all machines, e.g. having full name and email so that commits are trackable, and having ssh keys in the same directories so it’s easier to troubleshoot authentication problems when they happen (although rare).
• Installing direnv to unclutter local directories and allow environment variables to be loaded reliably.

Write and maintain run configurations for your IDE

This is a habit I particularly like. Some IDEs like IntelliJ IDEA or VSCode allow people to share run configurations, i.e. declarative configuration files that can be interpreted by the IDE itself to allow execution of common tasks. Some examples:

• Create and scaffold a new migration script
• Run all frontend unit tests and update their snapshots
• Run all system tests in the backend
• Lint the codebase and audit installed packages for vulnerabilities

To exemplify even further, this is my setup for a project I am working on.

The advantage of sharing run configurations is that it allows all developers to run the same commands on equal ground even if they are not prone to tinkering with scripts and CLI details. Each member of the team should collaborate to improve automation by pitching in those areas they are more expert with.

.env files are nice: use them

.env files simply declare a list of variables that will be loaded in the current environment. They are especially useful at development time as they contain a self-sufficient and consistent set of values that guarantee a component is properly configured. However, versioning .env files is generally discouraged, as it opens up potential security flaws. That’s why it is often better to instead commit a script that is able to generate .env files, so that all developers running that script have the same setup, yet specific credentials like passwords and usernames may differ.

For example, one of our backends is written in TypeScript. The shell script above generates a random password that is then used to spin up a docker container with a local database. The same password is also inserted in the .env file for the main application and in the one tailored for our ORM. All .env files are slightly different since they use different variable names, but their values are cohesively orchestrated by a single conductor. Everything runs automatically by pressing a button.

Of course, this is one approach to enforcing cohesive environments. Other people may prefer to have a set of docker containers coordinated by docker-compose or a local k8s cluster. We feel that, for our local workflow, this is an overkill, but other teams with a richer set of microservices may choose to do just that.

Pin dependency versions

Pinning a version means requiring the exact literal version of a package or module, rather than a set of possible versions. Many package or dependency managers use simple notations to designate ranges of valid versions:

• In pip, package>=2.0.0 means all versions from 2.0.0 onward
• In maven, [1.5.2, 1.8.3] means from version 1.5.2 to version 1.8.3
• In npm, ~1.0.2 means version 1.0.2 or any more recent patch version (i.e. the third number). 1.0.4 matches the expression, but 1.1.0 does not, because the minor build number is different

However, I cannot stress this enough: prefer pinning dependency versions whenever possible. Even if not intended, minor changes or patches may introduce differences that change the behaviour of code. It is often challenging to find bugs caused by such changes because their effect slips under the radar very easily due to automatic dependency resolution.