Contrary to popular belief, it’s possible to ship portable executables of Python applications without sending your users to Python packaging hell.
I’m hard at work wrapping up doc2dash 3.0 – a dear project of mine which is now over 10 years old. It’s also a Python application. And unless packaged into a more user-friendly format like Homebrew or a Linux distribution package, enduser Python applications have a bad reputation for good reasons: if you’re not somewhat competent in Python’s packaging landscape (or at least know how to use pipx) it can get unpleasant.
Given that doc2dash manages to be both my oldest and my least successful project at the same time, I always suspected the Python packaging angle to be part of the problem. Over the years, I’ve tried all application bundlers available, but none of them worked: they all failed with varying obscure errors1.
Long story short, doc2dash 3.0 is going to have portable binaries for Linux, macOS, and Windows.
pyoxidizer.bzlis the build script.
python_config.run_commanddefines the CLI entry point that is called when the binary is started.
exe.add_python_resources()installs my (per-platform pinned) requirements and the application itself.
noxfile.py2: I have one target (
pin_for_pyoxidizer) to pin the dependencies for reproducible builds and one (
oxidize) to build a release binary. Passing
standalone_staticfor Windows builds ensures that I get a single
.exefile (the option doesn’t work on macOS and on Linux you get musl which can have performance problems). Finally, the
download_and_package_binariestarget downloads the artifacts for the latest Git tag, makes Linux and macOS binaries executable, and builds per-architecture Zip files together with the a concatenation of all licenses of all packages that are within the binary.
oxidizetarget on Linux, macOS, and Windows, checks the binaries work, and uploads them as artifacts. If you want, you can try out a binary from a recent test build (and let me know if there’s problems!).
This is just a TIL and not a proper blog post, because there’s a lot I’d still like to improve in my PyOxidizer usage – like figuring out Apple Silicon builds. That said, people using it longer and more intensely than me seem happy!
Either way, as someone who’s maintained a Python application for 10 years whose majority of bug reports were packaging-related, it’s exhilarating that I even got this far. And I want the Python community to know about this gem.
Given it’s solving a monumental problem (Python packaging isn’t problematic because people are evil or obtuse – it’s problematic because of the diversity and complexity of the ecosystem), PyOxidizer’s can’t be perfect: as of writing it has 288 issues open. For instance, since it operates on somewhat virtual filesystems, packages that rely on the
__file__ variable instead of the
importlib packaging machinery (case in point: Sphinx) won’t work (there might be workarounds – I haven’t checked).
But Gregory’s toolchain is one of the most exciting things happening to Python right now and I hope he’ll be able to keep working on it (meaning: FAANG, please throw money at him!).
Adjacent projects that you don’t have to tell me about: Briefcase (focuses on app bundles – I want a single binary), PyInstaller (couldn’t get to work with doc2dash and tried many times over the years – I have been told it’s good now too, though), py2exe / py2app (Windows / macOS-only respectively). ↩︎