Remote Development in Elixir with Gitpod 🍊

After hearing about the end of localhost development I fully agreed with the general opinion of the hosts. Developing in the cloud? Ditching my pretty, well customised dev setup? No way.

A few weeks later Carter Bryden made me listen to Remote Development in Elixir twice (I didn’t listen to a podcast episode more than a single time before). But he really made a point and made remote development sound interesting and fun. So I gave it a try.

What is remote development?

Remote development is basically using ssh to develop your application on a different machine somewhere in the cloud. But not like old-school VNC / Remote Desktop / Citrix where you have a full-blown desktop which is often sluggish and unresponsive. You can still run your IDE/Code Editor locally on your machine, but the code you are editing is stored in an isolated environment (usually a Docker container). This also means, that the code is built on the remote machine, which can significantly speed up your compilation times.

The environment you are working in is ephemeral. This is a key piece, because this means that the full development environment can be automated and is therefore reproducible. This happens via a Docker image and a configuration file. In these files you can specify all the steps required to get your app up and running (e.g. yarn install and yarn dev), extensions, run shell scripts, etc. Whenever you start a new environment, everything will run again and you get a fresh state.

The most popular products are GitHub Codespaces and Gitpod. I picked Gitpod, as they have a generous free plan, which is especially good when you just want to try out remote development.

Using Gitpod for Elixir

As Elixir still is a niche language, it does not have the excellent support in Gitpod as Node, Java or Go. They have an Elixir Docker image, but that relies on the packages from apt which are quite outdated. Getting up and running with Elixir took me a few hours which I want to save you. Here is my setup.

TL;DR - just show me the code

You can open the workspace directly in Gitpod here or just have a look at the repo:

  1. Open in Gitpod
  1. Source on GitHub

Workspace requirements

  1. Using the latest Erlang and Elixir version.
  1. Using the latest PostgreSQL version.
  1. Include common VSCode extensions for Elixir Development
    1. ElixirLS for VSCode (see below)
    1. hex.pm IntelliSense
    1. TailwindCSS IntelliSense
    1. Phoenix Framework for VSCode

🤔 Elixir LS and the use macro

There is an issue in the elixir ls repo which will cause autocompletions to be missing when using the use macro. This is caused when the Erlang/Elixir versions do not match the version ElixirLS has been compiled with.

To make sure, autocompletions always work properly, we will build ElixirLS and the ElixirLS extension vor VSCode manually within the Dockerfile.

Parts of the Gitpod workspace

  1. gitpod.Dockerfile: A Dockerfile where we setup the development environment.
  1. gidpod.yml: A configuration file where we define the tasks we need to run to get the app up and running. Thins like fetching dependencies, starting up the server, exposing ports, defining extensions.
  1. install_extensions.sh: A helper script to install our custom ElixirLS extension upon workspace start.

Let’s have a look at each of these files in more detail.

gitpod.Dockerfile

FROM gitpod/workspace-full

# 1. Install dev tools
RUN brew install fzf \
    && $(brew --prefix)/opt/fzf/install --completion --key-bindings --no-fish

# 2. Install PostgreSQL 14
# If you are fine with an older version, you can skip the PostgreSQL block
# and just use FROM gitpod/workspace-postgres as base image.
ENV PGWORKSPACE="/workspace/.pgsql"
ENV PGDATA="$PGWORKSPACE/data"

RUN sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list' \
    && wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - \
    && sudo apt -y update

RUN sudo install-packages postgresql-14 postgresql-contrib-14

# Setup PostgreSQL server for user gitpod
ENV PATH="/usr/lib/postgresql/14/bin:$PATH"

SHELL ["/usr/bin/bash", "-c"]
RUN PGDATA="${PGDATA//\/workspace/$HOME}" \
 && mkdir -p ~/.pg_ctl/bin ~/.pg_ctl/sockets $PGDATA \
 && initdb -D $PGDATA \
 && printf '#!/bin/bash\npg_ctl -D $PGDATA -l ~/.pg_ctl/log -o "-k ~/.pg_ctl/sockets" start\n' > ~/.pg_ctl/bin/pg_start \
 && printf '#!/bin/bash\npg_ctl -D $PGDATA -l ~/.pg_ctl/log -o "-k ~/.pg_ctl/sockets" stop\n' > ~/.pg_ctl/bin/pg_stop \
 && chmod +x ~/.pg_ctl/bin/* \
 && printf '%s\n' '# Auto-start PostgreSQL server' \
                  "test ! -e \$PGWORKSPACE && test -e ${PGDATA%/data} && mv ${PGDATA%/data} /workspace" \
                  '[[ $(pg_ctl status | grep PID) ]] || pg_start > /dev/null' > ~/.bashrc.d/200-postgresql-launch
ENV PATH="$HOME/.pg_ctl/bin:$PATH"
ENV DATABASE_URL="postgresql://gitpod@localhost"
ENV PGHOSTADDR="127.0.0.1"
ENV PGDATABASE="postgres"

# 3. Install Erlang, Elixir, Node via asdg

# Erlang dependencies
RUN sudo install-packages build-essential autoconf m4 libncurses5-dev libwxgtk3.0-gtk3-dev libwxgtk-webview3.0-gtk3-dev \
    libgl1-mesa-dev libglu1-mesa-dev libpng-dev libssh-dev unixodbc-dev xsltproc fop libxml2-utils libncurses-dev openjdk-11-jdk

# Phoenix Dependencies
RUN sudo install-packages inotify-tools

RUN brew install asdf \
    && asdf plugin add erlang \
    && asdf plugin add elixir \
    && asdf plugin add nodejs \
    && asdf install erlang 25.1 \
    && asdf global erlang 25.1 \
    && asdf install elixir 1.14.0-otp-25 \
    && asdf global elixir 1.14.0-otp-25 \
    && asdf install nodejs 16.17.1 \
    && asdf global nodejs 16.17.1 \
    && bash -c ". $(brew --prefix asdf)/libexec/asdf.sh \
        && mix local.hex --force \
        && mix local.rebar --force" \
    && echo -e "\n. $(brew --prefix asdf)/libexec/asdf.sh" >> ~/.bashrc


# 4. Build vscode-elixir-ls extension
#
# We build this manually because ElixirLS won't show autocompletions
# when using the `use` macro if ElixirLS has been compiled with a different
# Erlang / Elixir combination. See https://github.com/elixir-lsp/elixir-ls/issues/193
#
# Aditionally, OpenVSX only contains a version published under the deprecated namespace.
# This causes issues when developing locally because it would always install the wrong extension.
RUN bash -c ". $(brew --prefix asdf)/libexec/asdf.sh \
    && git clone --recursive --branch v0.11.0 https://github.com/elixir-lsp/vscode-elixir-ls.git /tmp/vscode-elixir-ls \
    && cd /tmp/vscode-elixir-ls \
    && npm install \
    && cd elixir-ls \
    && mix deps.get \
    && cd .. \
    && npx vsce package \
    && mkdir -p $HOME/extensions \
    && cp /tmp/vscode-elixir-ls/elixir-ls-0.11.0.vsix $HOME/extensions \
    && cd $HOME \
    && rm -rf /tmp/vscode-elixir-ls"

1. Install dev tools (optional)

I do install fzf here including it’s shortcuts. Mainly to have ctrl+r to search for past commands in the history. You can also skip this step or customise it to your needs if specific dev tools should be present in the image.

2. Install PostgreSQL 14

The default PostgreSQL version of the gitpod/workspace-postgres does include PostgreSQL 12. When starting new projects, I’d like to start on the latest version, so I install the latest version. The code is taken directly from the original gitpod image. If you are fine using PostgreSQL 12 then you can remove the whole section and replace the first line with FROM gitpod/workspace-postgres.

3. Install Erlang, Elixir, Node via asdf

To install specific versions of Erlang, Elixir and Node I rely on the awesome asdf version manager. All the commands are basically installing dependencies, setting up asdf, and install specific versions. This should feel very familiar if you developed using Elixir before.

4. Build ElixirLS VSCode extension

As discussed before, to get proper autocompletions, the ElixirLS extension for VSCode is compiled inside the image. The extension is placed inside $HOME/extensions and installed upon workspace start.

gitpod.yml

This file should be way easier to read / understand, even if you never saw it before.

image:
  file: .gitpod.Dockerfile

tasks:
  - name: "Phoenix Dev Server"
    init: |
      mix setup
      mix compile
    command: iex -S mix phx.server

ports:
  - port: 4000
    onOpen: open-preview

vscode:
  extensions:
    - bradlc.vscode-tailwindcss
    - benvp.vscode-hex-pm-intellisense
    - victorbjorklund.phoenix

1. Image

A reference to the location where the Dockerfile from before is placed.

2. Tasks

The tasks required to get the Phoenix development server up and running. The init block contains everything which is required to prepare the application. It has to be a terminating task - do not start the server in the init block. This involves running mix setup and mix compile.

The dev server is started within the command block. This can be a long-running process and won’t be automatically terminated.

3. Ports

Tell gitpod on which port the dev server runs on, so that gitpod can properly forward the port and open a private url to access the app.

4. Extensions

A list of all extensions which should be enabled for all people using the workspace. Note that when running VSCode in the browser, all extensions will be fetched via OpenVSX - an open source version of the Extension Marketplace. You can still install extensions from the official Microsoft Marketplace when running VSCode locally.

install_extensions.sh

The only way to install an extension which is placed inside the Docker image I could come up with, is to create an folderOpen task which runs the following code.

#!/bin/bash
#
# Install ElixirLS extension if we run inside gitpod and run desktop vscode.
# This is a workaround until https://github.com/gitpod-io/gitpod/issues/12791
# is fixed.

ELIXIR_LS_VERSION=0.11.0

if test $USER = "gitpod"
then
  code --install-extension $HOME/extensions/elixir-ls-$ELIXIR_LS_VERSION.vsix
else
  echo "Nothing to do"
fi

exit 0

The script checks if we are running within the container by checking the current user gitpod. If so, it installs the extension from the previously compiled vsix file.

To make sure this script runs when the workspace is opened, place the following two files inside the .vscode folder of the workspace.

settings.json

{
  "task.allowAutomaticTasks": "on"
}

tasks.json

{
  "version": "2.0.0",
  "tasks": [
    {
      "label": "Gitpod: Install ElixirLS extension",
      "type": "shell",
      "command": "./install_extensions.sh",
      "group": "none",
      "presentation": {
        "reveal": "silent",
        "panel": "new",
        "close": true
      },
      "runOptions": {
        "runOn": "folderOpen"
      }
    }
  ]
}

That’s it.

There are quite a few moving parts to get a proper Elixir workspace up and running, but I am very satisfied with this setup and use it for all my Elixir projects since a month or so.

As I am travelling right now and don’t have access to the more powerful desktop machine, the additional bonus in computing power (especially when running multiple workspaces) and increased battery life of my notebook is absolutely worth it.

I hope you enjoyed this.

Cheers

-Ben