A Docker Compose Setup for Clojure/Script Development

August 28, 2021 | 15 min. read

In the past few weeks I've been learning the Clojure programming language. I really like it so far, and it feels like a (well designed) breath of fresh air from my day-to-day work in PHP and JavaScript. In addition to doing Advent of Code problems to familiarize myself with the standard library and thinking about problems the Clojure way, I've been messing around with the Luminus web framework It's actually not strictly a "framework", and instead is a project template including several heterogenous Clojure libraries for things like routing, HTML templating, database connections, etc. I'd be interested in a more comprehensive, batteries-included framework like Rails or Laravel but you get most of what you need from the included libraries and rolling your own stuff beyond that isn't too hard..

My preferred web development workflow is on my local machine using Docker Compose, but unfortunately Luminus doesn't come with an out-of-the-box docker-compose.yml template. I've come up with one that works pretty well, and I'm happy to share it here.

Installing Leiningen & Creating a Luminus Project

A prerequisite to all of this, obviously, is installing Clojure. Clojure is hosted on the JVM, so you'll need a JDK version 8 or higher.

The most frictionless way to get a new Clojure project up and running is with Leiningen. Installation instructions can be found here. To make sure you have it installed, you can run lein --version. It'll probably chug for a bit while spinning up the JVM, then spit out a version number. Here's what I got:

$ lein --version
OpenJDK 64-Bit Server VM warning: Options -Xverify:none and -noverify were deprecated in JDK 13 and will likely be removed in a future release.
Leiningen 2.9.6 on Java 16.0.2 OpenJDK 64-Bit Server VM

Leiningen has a pretty handy templating feature that allows for different project structures based on what template is used. For example, lein new default my-library will create a new project titled my-library with the default library structure. Similarly, lein new app my-app will create a project titled my-app using the app directory structure.

Luminus leverages this feature to allow you to mix-and-match different components via specifying different profiles when creating your project. For example,

$ lein new luminus my-luminus-site

... will create a barebones Luminus app with little more than an HTTP server and routing library. Instead, if you wanted libraries for interacting with a PostgreSQL database you could run:

$ lein new luminus my-luminus-site +postgres

... and you'd get a setup including everything you need. For this example, we're going to use PostgreSQL for our database and ClojureScript on the frontend via a tool called shadow-cljs. shadow-cljs lets us use NPM to manage JS dependencies when we need JS interop, and I find it easier to use and set up than the default ClojureScript setupGranted, I probably haven't given the ecosystem a fair shake. A lot of Clojure build/dependency tooling is very much "here's a few well-designed but minimalist tools, assemble them to your liking" or, as one great blog post put it, an amalgamation of orthogonal parts. I'm very much the opposite (especially when it comes to frontend tooling!) in that I just want a single installable command that will install dependencies reliably and build productionized code painlessly. that uses Leiningen and CLJSJS to manage JS dependencies. To make our project, run:

$ lein new luminus my-luminus-site +postgres +shadow-cljs

Now, if you cd into my-luminus-site, you should see a blank project template.

Setting up Docker Compose

Although we already installed Leiningen locally to create our project, I much prefer to keep each component of the project running in Docker containers for easy and reproducible setup/teardown (and keeping my dev environment clean).

There's three distinct components to our project: the Clojure backend, the ClojureScript frontend, and the Postgres database. Let's create a docker-compose.yml file in our my-luminus-site directory and create those services:

version: "3.9"

services:
  backend:
    image: clojure:lein
    ports:
      - "7000:7000"
      - "3000:3000"
    volumes:
      - ./:/tmp
      - ~/.m2:/root/.m2
      - ~/.lein:/root/.lein
    command: lein run

  frontend:
    image: theasp/clojurescript-nodejs:latest
    ports:
      - "7002:7002"
      - "9630:9630"
    volumes:
      - ./:/tmp
      - ./node_modules/:/tmp/node_modules
      - ~/.m2:/root/.m2
      - ~/.lein:/root/.lein
    command: npx shadow-cljs watch app

  postgres:
    image: postgres:latest
    environment:
      - POSTGRES_USER=db-user
      - POSTGRES_PASSWORD=db-password
    ports:
      - 5342:5342

Let's go through each of these services individually.

...

  backend:
    image: clojure:lein
    ports:
      - "7000:7000"
      - "3000:3000"
    volumes:
      - ./:/tmp
      - ~/.m2:/root/.m2
      - ~/.lein:/root/.lein
    command: lein run
...

For the backend, we're using Clojure's lein image which comes with Leiningen and the JDK preinstalled. Luminus by default exposes port 3000 for HTTP and port 7000 for nREPL, so those need to be exposed. We're mounting the local my-luminus-app directory to the tmp/ directory in the Docker container since that's the default working directory of the Docker image. The ~/.m2 and ~/.lein directories are where our Java JAR dependencies are installed on our host machine. If we didn't mount those, we'd have to reinstall our dependencies every time we restarted the container which would be pretty tedious. The command lein run just launches our app's HTTP server and nREPL.

...

  frontend:
    image: theasp/clojurescript-nodejs:latest
    ports:
      - "7002:7002"
      - "9630:9630"
    volumes:
      - ./:/tmp
      - ./node_modules/:/tmp/node_modules
      - ~/.m2:/root/.m2
      - ~/.lein:/root/.lein
    command: npx shadow-cljs watch app

...

For the frontend, we're using an image that includes all of ClojureScript's dependencies as well as NodeJS (which is needed if we're using NPM with shadow-cljs). Like the backend, we're going to expose nREPL on port 7002 and the live code reload server on port 9630. The mounted volumes are the same as the backend as well - we're just mounting in the current directory and all of our dependencies. Our command launches shadow-cljs' nREPL and live code reloading server.

...

  postgres:
    image: postgres:latest
    environment:
      - POSTGRES_USER=db-user
      - POSTGRES_PASSWORD=db-password
    ports:
      - 5342:5342
...

This one's pretty simple. It's just a Postgres database with a user named db-user and the password db-password. Note: since we aren't mounting the data in a volume, our data won't persist between restarts of the container. If you'd like that feature, just add a volume binding mapping some host directory to /var/lib/postgresql.

Tweaking Configs

Like I mentioned in the intro, Luminus by default doesn't come with a good Docker setup out of the box. We're going to need to tweak a few settings before we can fully use this.

First, we need to change the host binding of shadow-cljs. Since shadow-cljs' nREPL by default only expects messages to come from localhost and we're running it in a Docker container, shadow-cljs will reject any connections we try to make. In shadow-cljs.edn, just add the :host key to the :nrepl map like so:

{ :nrepl {:port 7002
          :host "0.0.0.0"}
...

Also, since we added the +postgres flag, our code expects a database URL in our config somewhere. Our Clojure REPL is similarly picky about where requests come from, so we'll have to bind a new IP address just like for shadow-cljs. In dev-config.edn, add the :database-url and :nrepl-bind keys like so:

{:dev true
 :port 3000
 ;; when :nrepl-port is set the application starts the nREPL server on load
 :nrepl-port 7000
 :nrepl-bind "0.0.0.0"

 ; set your dev database connection URL here
 :database-url "postgresql://db-user:db-password@postgres/"
}

Also, I'm not 100% sure if I just got unlucky and ran into a bug or it was user error, but I noticed a missing dependency by default. If you don't already see it, you'll need to add the cheshire dependency to the project.clj file like so:

...
    :dependencies [[ch.qos.logback/logback-classic "1.2.5"]
                   [cheshire "5.10.1"]
                   [clojure.java-time "0.3.2"]
...

The version numbers might be different and there might be additional dependencies after the time of writing this, but the important thing is to make sure that [cheshire "5.10.1"] line is present.

Running Our Project

Now, if everything is set up properly, we can run the entire project with a single command (be sure you have Docker and Docker Compose installed!):

$ docker-compose up

If you don't care about seeing the logs or don't want to leave the terminal open, you can append the -d option to the above command and everything will run in the background. The containers will probably take a while to spin up, but after a minute or so you should see something like this:

And, if you go to your browser and visit localhost:3000, you should see the following:

Clojure editors/IDEs are outside the scope of this post, but I use Emacs and the fantastic CIDER package to interact with the REPL. You should be able to connect to the backend and frontend with cider-connect-clj and cide-connect-cljs respectively. The backend REPL is on localhost:7000, and the frontend REPL is on localhost:7002.

Packaging as a Single Docker Image

This is a good setup for development, but isn't well suited for deployment. We probably want to use a database not running locally, and things like live code reloading aren't intended for production. Ideally, we'd have a single Docker image that we could run via docker run on a web server. Here's a Dockerfile that builds exactly that:

FROM node:16 as install_npm

WORKDIR /usr/app
COPY ./package.json /usr/app/package.json

RUN npm install

FROM clojure:lein AS build_clj

COPY ./project.clj /app/project.clj
COPY ./shadow-cljs.edn /app/shadow-cljs.edn
COPY ./resources /app/resources
COPY ./src/ /app/src
COPY ./env/prod /app/env/prod

WORKDIR /app

RUN lein uberjar

FROM clojure:latest

COPY --from=build_clj /app/target/uberjar/my-luminus-site.jar .
COPY --from=install_npm /node_modules

CMD java -jar my-luminus-site.jar

If you haven't worked with Docker in a while or haven't gotten past the first few tutorials, this might look different than previous Dockerfiles you've seen. That's because it's a multistage Dockerfile, which (as the name suggests) builds the image artifacts step by step. This lets you use different images for each specific build step and only copy into the image exactly what you need instead of installing every build dependency on one gigantic image. Here's the first step in the above Dockerfile:

FROM node:16 as install_npm

WORKDIR /usr/app
COPY ./package.json /usr/app/package.json

RUN npm install
...
```docker
  
As you can guess, we use the `node` image to install all of our
JS dependencies, as specified in the local `package.json` that we
copy into the working directory of the container. We also alias this step to
`install_npm`, since we'll refer to it later.

```docker
...

FROM clojure:lein AS build_clj

COPY ./project.clj /app/project.clj
COPY ./shadow-cljs.edn /app/shadow-cljs.edn
COPY ./resources /app/resources
COPY ./src/ /app/src
COPY ./env/prod /app/env/prod

WORKDIR /app

RUN lein uberjar

...

This next step uses the clojure:lein image, which has Clojure, its dependencies, and Leiningen preinstalled. We copy over all our config files and source code into the image, and then run lein uberjar to build a single JAR file. This step is also aliased as build_clj for reference later.

...

FROM clojure:latest

COPY --from=build_clj /app/target/uberjar/my-luminus-site.jar .
COPY --from=install_npm /usr/app/node_modules .

CMD java -jar my-luminus-site.jar

This last step is pretty minimal. In this step we don't need Leiningen (since we already built the project), so we can get away with just using the clojure:latest Docker image. It copies over our JAR and node_modules from the previous steps, and runs our JAR file to serve the web app.

Before we build the image, we'll need to add a :database-url key to the my-luminus-site/env/prod/resources/config.edn file. It's just like the dev-config.edn file, but use the production database's connection URL instead I'm not sure why the template doesn't default to also have a blank prod-config.edn, like the dev and test configs. It's not a huge deal as long as you don't accidentally commit the env/resources/prod directory, or if you really want to you can just add a prod-config.end and fiddle with the project.clj as necessary. Regardless, it still feels like a missing stair..

Now, just run the following:

$ docker build -t my-luminus-site .

And then run the Docker image with:

$ docker run --rm -it my-luminus-site:latest

And we're off to the races!

Addendum

While I've worked with Docker and Docker Compose quite a bit, I'm still a novice when it comes to Clojure (especially ClojureScript) and the JVM in general. If I've made any errors here or you have helpful tips/suggestions, please send me an email!