Yes, Virgina - you can use NGINX with Docker in a twelve-factor app

Here at Button we really love twelve-factor apps. We love using Docker to deploy them and we love using NGINX to serve them.

With all this love for those tools you’d think they would be an effortlessly harmonious match made in DevOps heaven.

Erm. Not exactly. You see, sometimes love is ugly. Sometimes love is harsh. Sometimes the things you love don’t love each other. In fact, sometimes, no matter how awesome you know it would be if they could just put their differences aside and work together, you’ve got to accept that some things you love just aren’t going to play. nice. together.

Ahhhh NGINX, the true rockstar in our stack - uncompromising, brilliant, and unapologetically opinionated. In a show of defiance (or perhaps a totally necessary performance optimization), NGINX won’t read environment variables in its configuration file. This conflicts directly with factor III: “Store config in the environment”. But before we throw up our hands and send out a tweet citing creative differences, let’s see if we can’t play mediator in this feud and reach some common ground.

What to do? NGINX wants to be configured statically and Docker wants to use environment variables for runtime configuration. The answer is obvious (and in no way cringeworthy): dynamically configure the static configuration at runtime!

In other words, modify your container’s command to insert environment variables into the configuration before it starts NGINX.

Don’t hate. Interpolate.

This is essentially an exercise in templating, for which there is no shortage of tools at our disposal. However we’re going to need something a bit more flexible and powerful than the usual suspects sed and awk. For this, we’ll choose the awesome Jinja templating library for Python.

First, add the following to your Dockerfile to install Jinja:

RUN apt-get install -y python-pip

RUN pip install Jinja2

Now create a Python script named config_nginx.py. Its job is to load your NGINX conf as a template and use Jinja to drop in the environment vars.

Jinja doesn’t have the capability to do this out-of-the-box, but it’s pretty easy to implement with a custom filter. This will allow us to call a Python function with 2 arguments.

From the docs:

For example in the filter {{ 42|myfilter(23) }} the function would be called with myfilter(42, 23).

This fits our use case quite nicely. We’ll register filter function named env_var that will take a default value and the name of the environment variable we want to resolve.

jinja_env.filters['env_var'] = lambda default, varname: os.getenv(varname, default)

You’d use this in your NGINX conf template like so:

server {
  # use the value of the environment var LISTEN_PORT if it is defined, '8080' otherwise
  listen {{ '8080' | env_var('LISTEN_PORT') }} default_server;
}

Great! All set! With this templating filter, NGINX can be now completely configured at runtime!

…unless you need to resolve hostnames dynamically.

Resolving conflict

Yep. Our web server just threw a TV out the window and demanded a giant bowl of M&Ms with all of the brown ones removed. NGINX can’t be bothered with having to resolve any hostnames after it has started. For example, if you need to proxy to an upstream host whose name is somehow derived from the request, the operation will fail. That is, unless you specify one or more resolvers.

There is, however, an additional layer of indirection here. We’re running NGINX inside a Docker container. You may not have any idea which resolvers are available able to the container (especially so if you are deploying to a distributed pooling environment like Apache Mesos or Amazon ECS). So again we must rely on runtime templating to get our config straight.

read the container’s resolvers into a list with your Python script:

resolvers = []
with(open('/etc/resolv.conf')) as rc:
    ns_re = re.compile(r'^nameserver (.+)$')
    for line in rc:
        if ns_re.match(line):
            tokens = line.split(' ')
            resolvers.append(tokens[1].strip())

provide them to Jinja:

template.render({'resolvers': resolvers})

and write them into the template:

resolver
  {%- for r in resolvers -%}
    {{ ' ' + r}}
  {%- endfor -%};
  

All together now

The whole thing might look something like this:

config_nginx.py:

import os
import re
from jinja2 import Environment, FileSystemLoader

if __name__ == '__main__':

    resolvers = []
    with(open('/etc/resolv.conf')) as rc:
        ns_re = re.compile(r'^nameserver (.+)$')
        for line in rc:
          if ns_re.match(line):
                tokens = line.split(' ')
                resolvers.append(tokens[1].strip())

    templates_dir = os.path.join(os.path.dirname(__file__), 'templates')
    jinja_env = Environment(loader=FileSystemLoader(templates_dir))
    jinja_env.filters['env_var'] = lambda default, varname: os.getenv(varname, default)
    template = jinja_env.get_template('nginx.conf.j2')
    result = template.render({'resolvers': resolvers})

    print(result)

nginx.conf.j2:

server {

    listen {{ '8080' | env_var('LISTEN_PORT') }} default_server;

    resolver
      {%- for r in resolvers -%}
        {{ ' ' + r}}
      {%- endfor -%};
}
daemon off;

remember to add the “daemon off;” directive to your configuration. Otherwise, nginx will just spin up its child processes and exit, causing your container to shut down immediately

which yields:

server {

    listen 8080 default_server;

    resolver 123.45.6.78 9.87.65.432;
}
daemon off;

The last thing to do is change your Dockerfile’s CMD to run the templating script and use its output to configure NGINX:

CMD /bin/bash -c 'python /usr/src/app/config_nginx.py :> /etc/nginx/nginx.conf' && /bin/bash -c 'nginx -c /etc/nginx/nginx.conf'

And with that, we’ve successfully mended fences and softened hearts. The band is back together and the show will go on. Deploy with confidence.



And Rock On.