We have more ways to manage dependencies in Python applications than ever. But how do they fare in production? Unfortunately this topic turned out to be quite polarizing and was at the center of a lot of heated debates. This is my attempt at an opinionated review through a DevOps lens.
Most Python programmers don’t spend a lot of time thinking about how equality and hashing works. It usually just works. However there’s quite a bit of gotchas and edge cases that can lead to subtle and frustrating bugs once one starts to customize their behavior – especially if the rules on how they interact aren’t understood.
Proper cleanup when terminating your application isn’t less important when it’s running inside of a Docker container. Although it only comes down to making sure signals reach your application and handling them, there’s a bunch of things that can go wrong.
I’ve seen quite a bit of the world thanks to being invited to speak at conferences. Since some people are under the impression that serial conference speakers possess some special talents, I’d like to demystify my process by walking you through my latest talk from start to finish.
The Python standard library is full of underappreciated gems. One of them allows for simple and elegant function dispatching based on argument types. This makes it perfect for serialization of arbitrary objects – for example to JSON in web APIs and structured logs.
If your Python decorator unintentionally changes the signatures of my callables or doesn’t work with class methods, it’s broken and should be fixed. Sadly most decorators are broken because the web is full of bad advice.
Since the inception of wheels that install Python packages without executing arbitrary code, we need a static way to encode conditional dependencies for our packages. Thanks to PEP 508 we do have a blessed way but sadly the prevalence of old
pip versions make it a minefield to use.
My completely anecdotal view on the state of Python 3 in 2016. Based on my own recent experience, observations, and exchanges with other members of the Python community.
Don’t use Python’s
hasattr() unless you’re writing Python 3-only code and understand how it works.
bcrypt.” is not the best recommendation (anymore).
How to ensure that your tests run code that you think they are running, and how to measure your coverage over multiple
tox runs (in parallel!).
Setting up Python to the point to be able install packages from PyPI can be annoying and time-intensive. Even worse are OS-provided installations that start throwing cryptic error messages. Especially desktops are prone to that but it’s possible to break the whole toolchain of a server by installing some shiny package you heard about on reddit.
Apple ships a patched version of OpenSSL with macOS. If no precautions are taken, their changes rob you of the power to choose your trusted CAs, and break the semantics of a callback that can be used for custom checks and verifications in client software.
A completely incomplete guide to packaging a Python module and sharing it with the world on PyPI.
Google killed its Reader and my beloved Reeder for Mac and iPad officially won’t get updated in time. I think to have found an adequate setup to replace both.
As some of my older rage-filled articles indicated, we’re still running some services on Sybase’s SQL Anywhere. Since it cost me many hours and sanity wrangling, I think it may be helpful to others to summarize the current situation for Python developers.
In web development, we have a unfortunate double meaning for the word “models” and as obvious as the separation of those two seems to seasoned developers, it shows again and again that it’s not as obvious to beginners.
Even the best of us hate logging in Python sometimes. And while a lot of its problems are actually just bad docs and terrible defaults in the past, there is some pain that can be avoided.
There are many wordy articles on configuring your web server’s TLS ciphers. This is not one of them. Instead I will share a configuration which is both compatible enough for today’s needs and scores a straight “A” on Qualys’s SSL Server Test.
This one falls under: “I knew there has to be an easy way!”
When I read about PyLadies for the first time, my thoughts were a common knee-jerk: “separation is bad, dividing the community, …”. Like many of my privileged peers, I was pro-diversity but I thought this is the wrong way. My views changed over time and I filed it under “lessons learned”. Unfortunately, my old thinking patterns don’t cease to pop up in discussions, so I decided to share my perspective.
Like many OSS fans, I always wanted to be an active part of the movement. My last big project was for the Amiga in the past millennium though. Nowadays I’m happy that after years of small-scale dabbling on various projects I’ve found my haven. I’d like to share my way to my recent gain of push privileges on the Python project and hope to inspire some of you to do the same.
Speed, reproducibility, easy rollbacks, and predictability is what we strive for when deploying our diverse Python applications. And that’s what we achieved by leveraging virtual environments and Linux system packages.
Deploying web applications is hard. No shiny continuous deployment talk and no DevOps coolness can change that. Or to use DevOps Borat’s words: “Is all fun and game until you are need of put it in production.“ There are some mistakes I see people doing again and again so I’d like to address them here.
In our newest installation of “why you should not use Sybase SQL Anywhere” I’d like to report the latest problem I had to solve: for some reason, I couldn’t connect using sqlanydb from Celery tasks.
So you came to the same conclusion as I: Google is actually evil indeed. That makes it kind of uncomfortable to have all your e-mails over there, doesn’t it? I for one decided that it’s time to leave and will show you how to do the same using an UNIX based OS.
I never did a retrospective but 2011 deserves one.
There is this common notion, that asynchronous IO is hard and that writing a custom connection pool is even harder. The nice thing however is, that in reality asynchronous IO is just “weird” in the beginning – and that a connection pool using async IO is so simple it hurts.
After switching to the Mac, I had one big itch that spoiled all the bliss: MacVim sometimes simply refused to cooperate with the system clipboard. As you can imagine, an editor that can’t exchange text with other software is a rather painful thing.
Nope, this isn’t going to be a smug post on nutrition. These tomatoes I’m going to talk about aren’t for eating but for kicking my ass to be more productive. I’m talking about the Pomodoro Technique of course.
So you want quick offline access to your mails using mutt to fully exploit your SSD and yet still have everything nicely working in Google’s web interface? Additionally, you would like to have access to your Google Contacts just like in your phone and everywhere else? I’ll show you how!
There’s one thing hackers are opinionated about as about the right editor: The right color scheme.
Mostly a note to myself as I forget it regularly.
Using the official sqlanydb driver for Python together with Twisted’s adbapi produces not-so-occasional crashes as of today (sqlanydb 1.0.2, Twisted 11.0.0). Apparently, the official SQL Anywhere drivers aren’t thread-safe. It cost me several days to figure out because I was searching the fault in my code so I hope to spare you some pain.
I’d thought something like this is a FAQ but the database docs on postgres don’t write a bit about forcing Django to connect using SSL to the database server.
There are few people that influenced me more than Tim Ferris. He was the one who gave me the last nudge to the low carb diet and he was also the one from whom I learned about the “Information Diet”.
It took me a while to figure it out, so I decided to share.
After a few months of use my Kindle started to freeze randomly. At first I blamed the cold as it happened to be winter in that moment. Fortunately, it turned out to be something different.
The perception of LDAP (Lightweight Directory Access Protocol) is ambivalent. On the one hand, it is widely supported as a common authentication backend. On the other hand, there’s very little and poor documentation that is mostly targeted towards a special case (e.g. replacing NIS by LDAP).
Sounds harder than it is – especially when reading the official docs. But if you want to synchronize two DBs, just tell the “master” to write a log and slave to read it.
While developing a network sniffer I had to find a way to write
pcap logs. However the docs I found were rather fragmented. I try to do a short roundup here. In fact, the format is pretty plain and it’s a pity that there seems not to be a quick’n’easy doc for it.