Hynek’s Blog

No, it’s not (just) run-parallel – let’s cut the local tox runtime by 75%!

Ever since I got involved with open-source Python projects, tox has been vital for testing packages across Python versions (and other factors). However, lately, I’ve been increasingly using Nox for my projects instead. Since I’ve been asked why repeatedly, I’ll sum up my thoughts.

Or: Why does DYLD_LIBRARY_PATH keep disappearing!?

A love letter to tools that changed everything for me.

One of my (slowly evaporating) reasons why I like putting packaging metadata into an executable setup.py is the ability to have optional dependencies that are combinations of others. As of pip 21.2, this is possible without running code.

A common issue when writing tests for real-world software is how to deal with third-party dependencies. Let’s examine an old, but counter-intuitive principle.

An attempt at catharsis. This is a deeply personal blog post about the most influential project I’ve ever created: attrs, the progenitor of modern Python class utilities. I’m retelling its history from my perspective, how it begot dataclasses, and how I’m leading it into the future.

Codecov’s unreliability breaking CI on my open source projects has been a constant source of frustration for me for years. I have found a way to enforce coverage over a whole GitHub Actions build matrix that doesn’t rely on third-party services.

Trying out something new: today I’m launching my own Today I Learned section. In this essay I will sum up what my plans and hopes are.

The conflict between subclassing and composition is as old as object-oriented programming. The latest crop of languages like Go or Rust prove that you don’t need subclassing to successfully write code. But what’s a pragmatic approach to subclassing in Python, specifically?

This post is less about teaching and more about schadenfreude amusement for you, and catharsis for me. It’s the story of how one unfortunate HTML tag kicked me off almost all search engines and my months-long way back. And why it didn’t matter in the end.

The widely used Python package cryptography changed their build system to use Rust for low-level code, which caused an emotional GitHub thread. Enthusiasts of 32-bit hardware from the 1990s aside, a vocal faction stipulated adherence to Semantic Versioning from the maintainers, claiming it would’ve prevented all grief. I will show you not only why this is wrong but also how relying on Semantic Versioning hurts you – the user.

Lie and lay are infamously confusing to non-native speakers. It’s so bad that it sparked a cottage industry of click-baity articles full of sketchy ads. Since English is my third language, I stumbled a lot myself until I wrote this cheatsheet.

One of the main appeals of using Python’s asyncio is being able to fire off many coroutines and run them concurrently. How many ways do you know for waiting for their results?

Some projects have the policy that all tests must have an explanatory comment – including all of mine. At first, I found that baffling. If that’s you right now, this article is for you.

GitHub’s own CI called GitHub Actions has been out of closed beta for a while and offers generous free quotas and a seamless integration with the rest of the site. Let’s have a look at how to use it for an open source Python package.

I’m missing a key part from the public Python discourse and I would like to help to change that.

Since this topic keeps coming up, I’d like to briefly share my thoughts on Python package metadata because it’s – as always – more complex than it seems.

People always ask for book recommendations, so here’s my list of books that changed my life in one way or another. None of them is related to tech or programming, but all affect how I think and work.

Since the acquisition of Travis CI, the future of their free offering is unclear. Azure Pipelines has a generous free tier, but the examples I found are discouragingly complex and take advantage of features like templating that most projects don’t need. To close that gap, this article shows you how to move a Python project with simple CI needs from Travis CI to Azure Pipelines.

There are many good reasons not to go to every talk possible when attending conferences. However, increasingly, it became hip to boast about avoiding going to talks – encouraging others to follow suit. That rubs me the wrong way as a speaker, and I’ll try to explain why.

We have more ways to manage dependencies in Python applications than ever. But how do they fare in production? Unfortunately this topic turned out to be quite polarizing and was at the center of a lot of heated debates. This is my attempt at an opinionated review through a DevOps lens.

Most Python programmers don’t spend a lot of time thinking about how equality and hashing works. It usually just works. However there’s quite a bit of gotchas and edge cases that can lead to subtle and frustrating bugs once one starts to customize their behavior – especially if the rules on how they interact aren’t understood.

Proper cleanup when terminating your application isn’t less important when it’s running inside of a Docker container. Although it only comes down to making sure signals reach your application and handling them, there’s a bunch of things that can go wrong.

I’ve seen quite a bit of the world thanks to being invited to speak at conferences. Since some people are under the impression that serial conference speakers possess some special talents, I’d like to demystify my process by walking you through my latest talk from start to finish.

The Python standard library is full of underappreciated gems. One of them allows for simple and elegant function dispatching based on argument types. This makes it perfect for serialization of arbitrary objects – for example to JSON in web APIs and structured logs.

If your Python decorator unintentionally changes the signatures of my callables or doesn’t work with class methods, it’s broken and should be fixed. Sadly most decorators are broken because the web is full of bad advice.

Since the inception of wheels that install Python packages without executing arbitrary code, we need a static way to encode conditional dependencies for our packages. Thanks to PEP 508 we do have a blessed way but sadly the prevalence of old setuptools and pip versions make it a minefield to use.

My completely anecdotal view on the state of Python 3 in 2016. Based on my own recent experience, observations, and exchanges with other members of the Python community.

Don’t use Python’s hasattr() unless you’re writing Python 3-only code and understand how it works.

Why “Use bcrypt.” is not the best recommendation (anymore).

How to ensure that your tests run code that you think they are running, and how to measure your coverage over multiple tox runs (in parallel!).

Setting up Python to the point to be able install packages from PyPI can be annoying and time-intensive. Even worse are OS-provided installations that start throwing cryptic error messages. Especially desktops are prone to that but it’s possible to break the whole toolchain of a server by installing some shiny package you heard about on reddit.

Apple ships a patched version of OpenSSL with macOS. If no precautions are taken, their changes rob you of the power to choose your trusted certificate authorities (CAs) and break the semantics of a callback that can be used for custom checks and verifications in client software.

A completely incomplete guide to packaging a Python module and sharing it with the world on PyPI.

Google killed its Reader and my beloved Reeder for Mac and iPad officially won’t get updated in time. I think to have found an adequate setup to replace both.

As some of my older rage-filled articles indicated, we’re still running some services on SAP’s SQL Anywhere. Since it cost me many hours and sanity wrangling, I think it may be helpful to others to summarize the current situation for Python developers.

In web development, we have an unfortunate double meaning for the word models. As evident as the separation of those two seems to seasoned developers, it shows again and again that it’s not as apparent to beginners.

Even the best of us hate logging in Python sometimes. And while a lot of its problems are actually just bad docs and terrible defaults in the past, there is some pain that can be avoided.

There are many wordy articles on configuring your web server’s TLS ciphers. This is not one of them. Instead, I will share a configuration that scores a straight “A” on Qualys’s SSL Server Test in 2020.

This one falls under: “I knew there has to be an easy way!”

When I read about PyLadies for the first time, my thoughts were a common knee-jerk: “separation is bad, dividing the community, …”. Like many of my privileged peers, I was pro-diversity but I thought this is the wrong way. My views changed over time and I filed it under “lessons learned”. Unfortunately, my old thinking patterns don’t cease to pop up in discussions, so I decided to share my perspective.

Like many OSS fans, I always wanted to be an active part of the movement. My last big project was for the Amiga in the past millennium though. Nowadays I’m happy that after years of small-scale dabbling on various projects I’ve found my haven. I’d like to share my way to my recent gain of push privileges on the Python project and hope to inspire some of you to do the same.

Speed, reproducibility, easy rollbacks, and predictability is what we strive for when deploying our diverse Python applications. And that’s what we achieved by leveraging virtual environments and Linux system packages.

Deploying web applications is hard. No shiny continuous deployment talk and no DevOps coolness can change that. Or to use DevOps Borat’s words: “Is all fun and game until you are need of put it in production.“ There are some mistakes I see people doing again and again so I’d like to address them here.

In our newest installation of “why you should not use Sybase SQL Anywhere” I’d like to report the latest problem I had to solve: for some reason, I couldn’t connect using sqlanydb from Celery tasks.

So you came to the same conclusion as I: Google is actually evil indeed. That makes it kind of uncomfortable to have all your emails over there, doesn’t it? I for one decided that it’s time to leave and will show you how to do the same using an UNIX based OS.

I never did a retrospective but 2011 deserves one.

There is this common notion, that asynchronous IO is hard and that writing a custom connection pool is even harder. The nice thing however is, that in reality asynchronous IO is just “weird” in the beginning – and that a connection pool using async IO is so simple it hurts.

After switching to the Mac, I had one big itch that spoiled all the bliss: MacVim sometimes simply refused to cooperate with the system clipboard. As you can imagine, an editor that can’t exchange text with other software is a rather painful thing.

No, this isn’t going to be a smug post about nutrition. These tomatoes I’m going to talk about aren’t for eating but for kicking my ass to be more productive. I’m talking about the Pomodoro Technique of course.

So you want quick offline access to your mails using mutt to fully exploit your SSD and yet still have everything nicely working in Google’s web interface? Additionally, you would like to have access to your Google Contacts just like in your phone and everywhere else? I’ll show you how!

There’s one thing hackers are opinionated about as about the right editor: The right color scheme.

Mostly a note to myself as I forget it regularly.

Using the official sqlanydb driver for Python together with Twisted’s adbapi produces not-so-occasional crashes as of today (sqlanydb 1.0.2, Twisted 11.0.0). Apparently, the official SQL Anywhere drivers aren’t thread-safe. It cost me several days to figure out because I was searching the fault in my code so I hope to spare you some pain.

I’d thought something like this is a FAQ but the database docs on postgres don’t write a bit about forcing Django to connect using SSL to the database server.

Be careful to not overeat.

It took me a while to figure it out, so I decided to share.

After a few months of use my Kindle started to freeze randomly. At first I blamed the cold as it happened to be winter in that moment. Fortunately, it turned out to be something different.

The perception of LDAP (Lightweight Directory Access Protocol) is ambivalent. On the one hand, it is widely supported as a common authentication backend. On the other hand, there’s very little and poor documentation that is mostly targeted towards a special case (e.g. replacing NIS by LDAP).

Sounds harder than it is – especially when reading the official docs. But if you want to synchronize two DBs, just tell the “master” to write a log and slave to read it.

While developing a network sniffer I had to find a way to write pcap logs. However the docs I found were rather fragmented. I try to do a short roundup here. In fact, the format is pretty plain and it’s a pity that there seems not to be a quick’n’easy doc for it.