Environment-based Django settings

2017-09-04 by Senko Rašić

Django's default settings.py architecture is overly simple. It doesn't account for the fact that a single project is likely to be deployed to different developer workstations, test servers, and production servers. This, together with the unfortunate fact that Django combines project specification (which apps, middleware, or other components are used) with the configuration (eg. which database to use and how to connect to it), has lead to nearly everyone organizing the settings to have a bit more structure.

Over time and dozens of projects, at Good Code we've settled on a setup based on the Twelve-Factor App approach of storing config in the environment. This is similar to the approach suggested by the Two Scoops of Django book, a bit more streamlined.

Here's how it works:

Project name is "project"

This is not directly tied with settings, but makes the whole thing nicer. A Django project is basically a set of applications along with their configuration, and no behaviour lies (or should lie) there. As such, we just name it project.

Package instead of module

The default settings module is a single file settings.py in the project directory. In our approach, it's a package (a collection of modules in a directory) instead:

project/
    settings/
        __init__.py
        base.py
        env.py
        test.py

The __init__.py file there to signal that the directory is a Python package, and just imports everything from base.py.

The actual specification of what the project consists of is in base.py, which is basically the old settings.py adapted for our approach - instead of hardcoding the values there, we hardcode the defaults (if any), and pull the actual values from the environment by using the helpers defined in env.py.

For automated tests, the configuration is usually significantly different that it is helpful to define a modified configuration in test.py (imports everything from base, overrides what's needed).

The project's manage.py, wsgi.py, asgi.py or similar entry points don't need to be changed, as the settings module is still project.settings (importing everything into __init__.py means we don't have to change the settings module to project.settings.base).

The env helper

The env.py helper is a simple Python module, not depending on Django, that exposes a few functions for reading the environment variable and returning the values with appropriate types (casting to integer, boolean, splitting lists, and the like).

In practice, the environment variables are never added manually to the environment. Instead, they're usually stored somewhere in text file with NAME=VALUE lines which can then be imported (sourced) into the shell. To avoid making the user import these manually before starting Django, the env helper can read this file directly. It looks into the project's root directory for a file names .env and, if it exists, reads all the variables and stores them into its environment.

As a way of documenting which environment variables can be used to configure the project, we usually include an env.sample or similarly-named file that contains the (empty) variables and comments explaining their effect.

We vendor in the env.py helper to each project. This means we don't have to worry about accidentially breaking old projects if/when we tweak the helpers.

Using environment files in the base settings module

To use the information from environment instead of hardcoding the settings in the base.py module, we first import everything from env.py helper and then use its helper methods. The helper is careful about defining the default symbols to be exported (via __all__) so we don't pollute the settings namespace with extraneous names.

Here's a few examples:

from .env import * # noqa
...
DEBUG = ENV_BOOL('DEBUG', False)
ALLOWED_HOSTS = ENV_LIST('ALLOWED_HOSTS', ',', ['*'] if DEBUG else [])
SECRET_KEY = ENV_STR('SECRET_KEY', 'secret' if DEBUG else '')
...

First we import the env helper. (The noqa comment here is to tell the linter to suppress star-import warnings. Depending on your preference about these, you may also want to silence the linter or rewrite th eimport statement to explicitly list all the helpers it needs.)

Next, we pull the DEBUG flag from the environment, and specify the default as False. The ENV_BOOL helper is flexible enough to case-insensitively parse true, yes and 1 as True, and false, no, and 0 as False. Since the helper is vendored in, it's easy to adapt it to project-specific needs (adding on / off for example) without needing to worry about the effect of that tweak on other projects.

For the ALLOWED_HOSTS setting, we treat the ALLOWED_HOSTS as a comma-separated list (the second argument is the separator). If the list is not provided, the default is used. In this case, we want to allow any host if we're debugging (this used to be the default in Django but was changed recently due to being a possible security problem), and not allow anyone if we're in production.

We do something similar for SECRET_KEY, and so on.

Using DATABASE_URL

Databases configuration is a bit more complex since it involves setting up a few settings - type, server host and port, name, user and password to connect with. Although we could have a separate env variable for each, a much nicer, standard, way is to use a database URL instead.

For Django, there's a great package called dj-database-url that parses DATABASE_URL environment value directly (no need to use env.py) and returns the entry for DATABASES setting appropriately:

DATABASES = {'default': dj_database_url.config()}

It can also take a default value:

DATABASES = {
    'default': dj_database_url.config(default='sqlite:///db.sqlite')
}

Bringing it all together

In addition to the settings setup, we usually include at least a barebones README.md in the project root, with the instructions on how to set up and install the project, and how to run the automated tests. The details about the available configuration variables are specified in the env.sample in the same directory.

I've put a newly-created, empty Django starter project with this setup up on GitHub. Feel free to use it as a starting point or to take sample code or inspiration when starting your own Django projects!

Author
Senko Rašić
We’re small, experienced and passionate team of web developers, doing custom app development and web consulting.