Dead Simple Connection Pooling with Twisted

There is this common notion, that asynchronous IO is hard and that writing a custom connection pool is even harder. The nice thing however is, that in reality asynchronous IO is just “weird” in the beginning – and that a connection pool using async IO is so simple it hurts.

Recently, I wrote an authentication backend for our nginx IMAP/POP3 load balancer and while it performed fine without any pool magic, I found the bulks of TIME_WAIT connections annoying (we even had to raise the system limit for open connections, twice), so I added a pool.

It took 6 extra lines and less than 15 minutes.

In my case it was about LDAP access using ldaptor, but it doesn’t matter, so I’ll keep this article agnostic to the type of connection.

I put the LDAP related stuff in a separate class that has an instance variable called connections which is a deque – i.e. a FIFO. Why a FIFO? Since we have enough traffic at any time, we don’t have to worry about timeouts if the connections are added left and popped right – they are simply circled through and therefore kept fresh all the time.

I also added an instance variable called maxIdle that defines the maximal size of idle connections in our pool. You don’t want to have 1,000 connections in your pool just because of a single spike. This does not affect the total number of connections though!

So let’s start with a method to get a new connection:

def getConnection(self):
    try:
        defer.succeed(self.connections.pop())
    except IndexError:
        # create and return a connection

That would be the first three lines of our pooling. It tries to pop a connection from our pool and if there’s none left, we open a new one. If you wanted to limit the number of total active connections, you’d add a counter and a check here.

When you’re done, you want to free your connection again:

def returnConnection(self, connection):
    if len(self.connections) < self.maxIdle:
        self.connections.appendleft(connection)
    else:
        # kill the connection

And that marks the other three pool related lines: we check whether we want to pool the connection too and kill it if not. As there’s absolutely no concurrency or context switches that could lead to inconsistent state – this is all it takes.

Hynek Schlawack