The 0.11 release of plone.recipe.zope2zeoserver is out. If you are under Windows you will probably enjoy this upgrade. The zeoctl starting script was not working under win32 because it is based on zdaemon, which is Linux-specific.

I have added two scripts in the recipe, to be able to launch Zeo:

• at the command line, with bin\zeo.bat
• as a service, with bin\zeoservice.exe install/start/stop/remove

I love Pylint.

Correctly configured, it is really useful to raise your code quality. But it can be really painful to install if you are not under a Debian-like or OpenSuse-like system, because of its dependencies that are not namespaced packages (logilab-common and logilab-astng).

In other words, don’t try to easy_install the packages that are on PyPI, it will not work.

That’s why I have created an egg, called logilab.pylintinstaller, that will let you install it as easy as:

$  easy_install logilab.pylintinstaller

It bundles all dependencies and grabs pylint 0.14, then installs everything.

Thanks to Sylvain Thénault from Logilab for his help on this.

This is my first important step in order to fullfill the Strategic Summit task #7817 (http://dev.plone.org/plone/ticket/7817).

I have released the iw.dist package which is a replacement for register and upload commands. Theses changes will be pushed in Python 2.6 hopefully. Until then, iw.dist provides two new commands, called mregister and mupload, which are acting the same way.

Please, use them instead of register and upload ! So I can get some feedback

It is quite simple to set up, see this page: http://pypi.python.org/pypi/iw.dist

It will not bather you at all since it does what the regular commands do, but are the first step to a tool that will let you upload the eggs to plone.org.

I have been suggested as a “champion” at the Plone Strategic Summit on this task:

# Improve release procedures for add-ons on plone.org: document a release process, and create release tools for packaging and uploading products from the command line.

This is great, because I have been working a lot in this area in the past month and I think I have a precise idea on what should be done in the Plone community to improve add-ons products visibility and releasing process. I am going to expose here the steps I think we should take, and how to do them, so people can give some feedbacks. Most of them were already explained on this blog on several entries.

PyPI vs Plone.org

The Cheeseshop (PyPI) is now playing an important role in Plone development. Everytime a Plone 3 application is built somewhere in the world, the Cheeseshop is serving hundreds of tarballs and eggs. Since Zope and Plone has been eggified, and since zc.buildout has been used as the standard way to build a Plone application, Plone developers are releasing all their eggs at PyPI.

This releasing process is really convenient, as a package can be uploaded, and shout out in just one command:

$ python setup.py register sdist bdist_egg upload

And an alias can make it even simpler:

$ python setup.py release

The problem is that many Plone.org add-on products pages that used to be up-to-date are not upgraded anymore. So Basically, Plone.org Software Center is dying because of the actual releasing process of eggs..

The SPOF problem

Another issue with the PyPI-centralized development process is that it becomes a Single Point of Failure. In other words, if PyPI is down, all the buildouts out there are stocked, unless you have a up-to date egg cache on your side.

PyPI though, together with distutils, was thaught as a distributed system: you can theorically call register and upload commands to any server that implements the PyPI Apis. But there are no other PyPI-like server yet in the community. The PyPI code is open source for sure, and anyone could take it and run his own PyPI…

PloneSoftwareCenter features

Another risk we have with a PyPI-centric approach is loosing the features that PSC provides at Plone.org. Those are great, and should be used by all add-ons out there. Milestones, bug tracker, etc.. Everything is provided at plone.org for someone to promote and work with his product.

The solution we should take

To avoid the problems mentioned, we need to:

• make PloneSoftwareCenter, therefore Plone.org, PyPI-compatible
• make distutils command-line tools able to interact with several PyPI-compatible servers, besides the official one
• provide a simple guideline for the Plone community to work with these tools

The steps

PSC

Sidnei has created 2 years ago a branch for PSC with an experimental PyPI support. I have taken this work and continued it on a branch that is almost finished. My goal is to finish it at the Paris Sprint, so PSC will be fully PyPI-compliant. I will soon blog on this to describe the work.

.pypirc and distutils

In order to be able to interact with several PyPI-like server, the .pypirc file need to evolve. I made a patch and a proposal (see: http://wiki.python.org/moin/EnhancedPyPI) I will try to push in the next Python Bug Day in two weeks, so it is integrated in Python 2.6. If it is accepted, I will release a library that implements the same patch, but for Python 2.4 and 2.5, through specific setuptools commands.

guideline

From there, I guess a guideline can be written, explaining :

• how to create a Plone 3 package (through skeletons)
• how to release it to both PyPI and plone.org

Another point of interest will be to explain how to deal with several egg servers in a buildout.

Proposed calendar

• Mid-March : provide a package to handle the new .pypirc format
• End of March : submit the guideline, based on the PSC current branch, and a public instance of the new PSC so people can try it.
• End of April : finalize PSC PyPI support at the Paris Sprint, together with Alex Clark, so it is available to Plone.org when it goes Plone 3
• After that: submit a guideline on how Plone companies can use PSC to create a private PSC, and work together with PyPI, plone.org and their own PSC, in a buildout environment

I have also proposed an OSCON topic on this, in Portland, in July. So if my talk is accepted, this can be a good place to promote Plone.org’s Plone Software Center.

Hey, fellow Plone/Zope developer, this is a quick reminder if you would like to join us at the Paris sprint in April. There will be a lot of work and fun !

If you want to come, please:

• join the openplans project here : http://www.openplans.org/projects/plone-3-paris-sprint/project-home
• send a mail on the project mailing list if you would like to suggest a topic
• send me a mail if you are sure to come, so I can add you in the attendee list. If you need a hotel, let me know. Let me also know if you are ok to spare a room.

Hope to see you there !

Last week, Shane Hattaway made available RelStorage on svn.zope.org. This product replaces PGStorage and allows you to switch from FileStorage (the Data.fs file) to a Postgresql or Oracle storage for ZODB pickles. ZEO is not needed anymore in that configuration, and each Zope instance calls the same Postgresql server.

According to Shane, RelStorage handles high concurrency better than the standard combination of ZEO and FileStorage. So I have started to make a few tests with it to see how we can use it on some projects.

I have made a few changes to plone.recipe.zope2instance to be able to define a buildout that uses RelStorage, and created a buildout on the collective that runs a Plone 3 over RelStorage.

The buildout sets a relstorage configuration and patches the ZODB 3.7 code. Instead of a file-storage option in the buildout.cfg file, a rel-storage option is added:

rel-storage =
    type postgresql
    dbname zodb
    user postgres
    host localhost
    password postgres

If you want to try it with your Postgres or Oracle server, you can get it from the collective here: http://svn.plone.org/svn/collective/collective.relstorage/trunk/

Notice: the script that patches the ZODB code does a system call instead of using the difflib module, so you need to have the patch program available at the prompt. So I guess it will fail under windows.

Notice 2: The buildout installs psycopg2 egg, but not the oracle one.