How to Write Deployment-friendly Applications

The DevOps movement gave us many ways to put Python applications into production. But how can you practically structure and configure your applications to make them indifferent to the environment they run in? How do secrets fit into the picture? And where do you put that log file?

So far, I’ve held it at PyCon US 2018, and EuroPython 2018.

General

Configuration

Environment variables are the most portable and universally available way to pass key-value pairs between processes and environments.

  • Setting:
    • direnv is one of many tools that will set env variables as soon as you enter a directory. If you rely on env variables for configuration, this is very handy.
    • envconsul HashiCorp’s official Consul-to-env and Vault-to-env (don’t!).
  • Consuming:
    • environ_config allows you to got from a lowest possible denominator (environment variables) to well-known structured data (nested Python classes).
  • Templating:
    • envsubst – part of gettext – allows for env variable → file templating. Once the barrier to the runtime environment is passed, you can have files if you really need to.
    • confd supports templating from many backends including Consul, etcd, ZooKeeper, Redis, and – of course – environment variables.
    • consul-template is HashiCorp’s official Consul templating tool.

Secrets

Load Balancing

Load balancing is the best way to treat your applications as simple blocks and makes zero-downtime deployments, scalability, and fault tolerance Someone Else’s Problem™.

Introspection

Being able to look directly into applications instead of relying on side-effects is a very powerful concept. And it’s quite easy to achieve even with just the Python standard library by exposing an HTTP endpoint.

Just Another Boundary

Language/runtime boundaries are just another boundaries. It’s worthwhile to study and apply general software engineering principles.

Credits