Testing Go applications with PostgreSQL and docker

I spawn and terminate docker containers automatically with Jenkins.

This allowed me to have a lighter application and to keep my testing environment closer to the live environment.

In this post I describe how I did this.

The problem

A problem i recently faced was how to build continuous integration for a web application I am writing.

I decided to setup a Jenkins instance and let it poll my repository every some minutes for new commits. It’s probably not the best solution, but it is a good balance between features, customizability and manageabilty.

Since I do not run any relational database on my home server, I decided to test my web application using SQLite3.

But yesterday I had some troubles when deploying my application to Heroku: for some reason, having SQLite3 as dependency breaks the build.

I also noticed that having SQLite3 as dependency slows down the compilation A LOT.

Since I only needed SQLite3 for testing purposes, but my real usage scenario involves PostgreSQL, I decided to delete SQLite from my application and find a way to run PostgreSQL without actually running it.

Don’t get me wrong: I have nothign agains relational databases. It’s just that I didn’t need one on my home server so far and it’s not likely I’m going to need it any time soon. Some of the applications that I run use SQLite and I am fine with that.

Enter docker

My solution for such problem is to spawn a docker container running postgres, do my testing, and the kill such container.

Turns out that doing so is extremely simple, given how well done the PostgreSQL docker container is.

Long story short, when you spwan a postgres docker container, it is possible to set some environment variable in order to set:

  • the superuser password (POSTGRES_PASSWORD)
  • the username of an user with superuser privileges (POSTGRES_USER)
  • a name for a database which will be create automatically on startup (POSTGRES_DB)

And some more stuff.

So now my CI strategy is the following:

  • spawn a PostgreSQL docker container
  • check the code out from the git repository
  • set some other environment variables
  • run tests
  • kill and remove the docker container
  • mail me the status of the build

How-to

Since this is not a complete tutorial, I will be describing just the steps necessary to spawn and kill docker containers on-demand.

First things first

In order to use the docker command, you either need to be root  or the current user must be in the docker group.

Since my Jenkins under its own user, I had to add such user to the docker group, stop the current Jenkins instance and start it back.

Download the PostgreSQL images

Now it’s time to download the docker images.

My preference is to run the official image. After all, I’ll be downloading the full image just once.

docker pull postgres

Configure jenkins

In order to avoid problems in the future, let’s consider what we want to do and what could go wrong in the future:

  • we want to be able to spawn jenkins container
    • this might fail IF a container with the same name already exists
      • we must avoid name collisions (this is very likely to happen in the build immediately after a failed build: a build failure might prevent all future builds from succeeding)
  • we want to be able to remove the container after testing is done
    • this should be easy, provided we don’t have name collisions
  • we want a brand new, clean container/database for every build.

The simplest way to avoid the collision problem on the container name is to customize such name with one of the environment variables provided by Jenkins.

I decided to use $BUILD_NUMBER. This, together with a unique prefix only used by the current project in Jenkins, ensures that there will be no name collisions.

Now there are three necessary steps:

  1. Prepend the container creation to the testing process
  2. Set the database url accordingly
  3. kill and remove the container

In step1, we have to run the docker command:

docker run -p 5433:5432 –name jenkins-test-db -e POSTGRES_PASSWORD=jenkins -e POSTGRES_USER=jenkins -e POSTGRES_DB=mytestdb -d postgres

Picture:

pgdocker1

Step 2: Configure the application accordingly.

Since my application uses environment variables for most of its configuration, configuring the database url is simple for me. You might have to customize this a little.

pgdocker2

Step 3: stop and remove the container.

Easy peasy

pgdocker3

Notes

  • SSL is not configured for this container, so make sure to append “?sslmode=disable” to your database URL.
  • A failure in the build will probably leave a stale docker container. Just keep this in mind: build failure == a docker container somewhere that needs to be killed.
  • It just came to me that the TCP port is a source of collision too. I’ll update this post.

 

Advertisements

Rispondi

Inserisci i tuoi dati qui sotto o clicca su un'icona per effettuare l'accesso:

Logo WordPress.com

Stai commentando usando il tuo account WordPress.com. Chiudi sessione / Modifica )

Foto Twitter

Stai commentando usando il tuo account Twitter. Chiudi sessione / Modifica )

Foto di Facebook

Stai commentando usando il tuo account Facebook. Chiudi sessione / Modifica )

Google+ photo

Stai commentando usando il tuo account Google+. Chiudi sessione / Modifica )

Connessione a %s...