bdist_rpm is dead, long life to Py2RPM
by Tarek Ziadé
One thing I’ve learned along the way while working on Python packaging matters is that it’s quite impossible to maintain in the stdlib scope tools like bdist_rpm, which builds RPMs using Distutils’ enabled projects.
I’ve asked around, and it turns out 9 out of 10 of the packagers are not using it for their daily RPM packaging work, unless they are also the developers of the projects to package. They use their own tools.
There are several reasons:
- bdist_rpm makes a lot of assumptions when creating the spec files.
- there are no way to customize some sections in the spec file that will be generated.
- the spec file is most of the time edited by packagers before it’s actually used, so the automatic full translation from Python metadata to RPM metadata is a false good idea. And while packagers have templates, they eventually end up working manually in the spec.
- What’s good for RedHat 5 is not for the latest Fedora
- Python sdtlib cycle does not match the distro cycles. So even if we update it, it would be soon deprecated again
So, when it was suggested a few months ago to add a bdist_deb command in Distutils, I told people that anything related to creating deb files should be a project on its own. And stdeb seems to have this role now.
For Windows, it’s a bit different: tools like bdist_msi are easier to maintain and we don’t have many flavors of packaging in the win32 world, just one. The release cycle of Python’s stdlib fully works here.
Following this principle bdist_rpm has been removed from Distutils.
So for the RPM world I want the same standalone project, that provides RPM goodies for Pythoneers, on the top of Distutils2 a.k.a. the new packaging module in Python 3.3, to replace bdist_rpm.
As a matter of fact I’ve started this project a while ago for my work at Mozilla and called it pypi2rpm then. It’s called like this because its main feature is to create for CentOS a RPM package out of a project released at PyPI.
Try it:
$ pip install pypi2rpm $ pypi2rpm.py SQLAlchemy ...long output... /tmp/ok/python26-sqlalchemy-0.6.6-1.noarch.rpm written
This script uses Distutils2 to browse PyPI and sort versions, then uses a custom version of bdist_rpm to invoke rpmbuild. This script can also create a RPM out of an existing project, by using an existing .spec file. It bypasses setup.py.
But this is just a quick script I’ve created for my needs. In the long term, I want to rename it to Py2RPM and make it the replacer of bdist_rpm. The features I want it to provide are:
- A standalone script that can be used to create RPMs, by using a configurable command-line call
- A standalone script that can be used to generate a spec file out of a setup.cfg file
- A way to read all options from setup.cfg, and to avoid to duplicate metadata fields like name, version etc
- A way to be aware of the differences between the various RPM-based distros.
- A feature to call via RPC the various CentOS, Fedora, etc repositories, to check if the name is already taken, if some files conflicts, etc
- A distutils2 command, so the script can be called from the top-level pysetup script if wanted.
On a side note (but this is another blog post), setup.cfg is being spec-ed out and the spec will be versioned. This will allow other tools from the RPM world, {put a packaging system name here}, to consume it and provide similar features. And this file will soon be available at PyPI so we can work with it without downloading the tarball of the release.
If you’re interested into Py2RPM, please get in touch with me. I am not a RPM expert at all, and I’d be happy to have more people involved.
Hi Tarek,
I am very much interested. I’m a python programmer and a Fedora contributor since 2003, and since then I’ve become pretty confident with the tools. I’d be very happy to help you with this (plus: I live in Paris too).
Great !
Maybe we could chat on IRC sometimes to see how we can work on this. I am in #distutils on Freenode on week days. Talk to you soon.
I’ve been redirecting people to your new projects when I’ve rejected feature requests for bdist_rpm on bugs.python.org, I hope they will join forces!
BTW, s/py2rpm/pypi2rpm/ (I fixed that on the Python wiki, but obviously can’t here 🙂
As I mention in my entry, I am renaming to Py2RPM. So you might want to revert your wiki renaming 😉
Oh, I overlooked that! It makes sense, since this translates Python projects to RPM, without always using PyPI. I fixed the wiki.
Hi Tarek,
Very interesting, this allows tools to involve occordingly targeted systems instead of Python.
I’ve a question: Does py2rmp requiere centos (or fedora …) itself ?
Not really, it just requires rpmbuild. But since my initial target is CentOS, I made some assumptions. Part of the work would consist of making sure the tool works for all flavors.
Hello Tarek,
First of all I want to thank you for working on making python packaging a better place.
I like this possibility of taking advantages of python package to generate linux package.
However has I understand you approach you are not offering the possibility to create a virtualenv as part of this process.
Imagine you have created your super webapp and you would like to deploy it on your app server farm. It would be great if we could generate a linux package (.rpm,.deb)that will get deployed in an isolated virtualenv and in addition be able to specify the package dependencies (nginx, memcached, PIL, libxml …) that need to be install system wise.
This strategy could also be used to installe/distribute “desktop web” application.
Regards,
–yml
virtualenv is a deployment tool (or used by a deployment tool like fabric); I think that bdist_rpm should run inside a virtualenv, not the reverse.
Please have a look to this:
http://cclimited.webfactional.com/make_your_module/spec.creator.html
It contains a skeleton spec file used in building python packages for various rpm distribution.
[…] bdist_rpm command was removed and started a new life as a new project, py2rpm, so that it can evolve quicker than Python to follow the modifications of the policies and […]
Also to build maemo specific deb source package i’ve create my own distutils command extension sdist_maemo : http://khertan.net/python_sdist_maemo