I’m really excited about this: we are organizing a Plone sprint in Paris in April at Ingeniweb headquarters !

No topics are planned yet because it depends a lot on who will be able to come. This will be done throughout January.

Please join the project and its mailing list if you are interested:

http://www.openplans.org/projects/plone-3-paris-sprint/project-home

And do not hesitate to make some topic proposal on it

EDIT: Justus provided in the comments a great link where Jim Fulton argues that __ should be marked deprecated, folllowed by Neal Norwiz and Tim Peters answers. This helps a lot understanding what __ should be used for: http://mail.python.org/pipermail/python-dev/2005-December/058555.html 

I am writing on Python OOP best practices and I was wondering what are the best ways to name attributes in classes. My main concern is about the distinction between private and protected attributes.

• private attributes are attributes that cannot be seen or used outside of the class, even buy subclasses. The Python parser calls the name mangling algorithm when it finds them to prevent name collision;
• protected attributes are attributes that can be used and seen in subclasses and should not be used outside. Nothing is done on them and they can be used like public attributes.

When should we use them ?

If you read PEP8, it’s clearly said that name mangling (using a ‘__’ prefix) is the best way to protect an attribute from beeing accessed or overriden. So it should be used for all class internals that is not intended to be overriden.

But in the biggest open source code bases like Zope or Plone, ‘__’ usage is very uncommon. The simple ‘_’ prefix is often used instead, to mark attributes that are private to the class or to the module. So there are no real distinction between private and protected attributes. It seems that the ‘private’ concept is not even used, and people often cut their class code in two parts: public and protected.

In other languages (like Delphi) that define protected and private levels though, protected attributes are not used a lot, and people tend to cut their code in private and public parts and make the protected layer as slim as possible.

Practical rules

Based on these remarks, here’s a tentative of ‘__’ and ‘_’ prefixes best usages in Python, for the use cases I know :

• use __ with property. since properties cannot use overriden methods and are tied to the class, the methods used with it should always be private;
• use __ for methods that works with private attributes. If your methods works for private attributes, make them private too;
• use _ on methods when they are clearly intended to be overriden;
• use __ for all module functions and variables that are private. A protected level is not needed since a module cannot be overriden.

Following these rules would probably make 90% of class attributes private instead of protected, and change all base code conventions. So I am wondering: am I a bit unpythonic if I try to follow this standard in attribute naming ? My guess is that most base code are not clean enough in that matter. For instance, many of them use both new-style and old-style classes under Python 2.5, which lead to a MRO algorithm that differs depending on the classes !

I would love to hear how you people deal with these conventions.

I made a mini-sprint on PSC for Christmas, since everyone around me was sitting watching christmas movies on TV and trying to digest.

Here’s a wrapup for comment, and for upcoming work.

Current branch (pypi)

I’ve merged Sidnei’s work into a new branch, with the current trunk since his work was done 2 years ago. I have made a few changes from his initial implementation:

• the PyPI API is now coded in a browser view instead of a persistent object, since it has no properties to keep at all;
• when a release is uploaded, a new release object is created for the given version if it doesn’t exists instead of raising an error and asking the user to manually create it inside the PSC;
• the doctest was simplified and uses sample tarballs and eggs.

I need to finish up a few things and to add some features such as:

• automatic project creation. When a package is uploaded and no project corresponds to it, a new project is created using the egg name and provided metadata. This will make the PSC acts like the CheeseShop. (an option will be added in PSC to activate/deactivate this feature to prevent automatic creation of projects if not wanted).
• trove web service the TROVE.txt file created by Sidnei needs to be replaced by a call to the categories; (see next section)
• multiple uploads. Make sure everything works fine when several files are uploaded for one release;
• more tests I need to write more tests from various clients and platform to make sure it works good. (by recording setuptools/distutils calls and creating tests with this).

This work should be done this week if everyone is OK with what I have proposed.

About the Trove classification

The Cheeseshop provides a Trove classification (see http://www.python.org/dev/peps/pep-0301) which evolves. For instance the “Django framework” category was added last week IIRC.

Obviously, Plone eggs should follow this classification but when they are uploaded in a PloneSoftwareCenter they might find specific categories defined locally (these categories might be specific to the project). I think we should let people freely define their classifiers in setup.py and let each server take the ones they have in their list.

The problem with Cheeseshop implementation is that it fails silently when a item in the ‘classifiers’ list doesn’t exists on the server side. The package metadata seem to be lost after that. (this looks like a bug to me, I didn’t digg the PyPI code yet). I need to ask over distutils-sig about this and see if we can come up with a Cheeseshop that will pick the categories it knows out of the classifier list, and let the other alone. This would allow PSC to deal with extra categories.

Then the PSC will have to implement the trove web services and serve its categories, so the “list-classifiers” option of the register command works.

Until then I guess we can leave the classification settings manual.

About the .pypirc file

This file that is used with the register command is working just for one server and will not allow having several set of login/password. This is not a problem when the login of your plone with your PSC is the same than the one in PyPI. Otherwise it won’t work.

Furthermore, this command is using a hardcoded ‘pypi’ realm if you look at distutils/commands/register.py:

auth.add_password('pypi', host, username, password)

The real solution here is to make distutils evolve so the .pypirc file allows having several login/password for each server, using the host url and the realm (the realm can be queried automatically too). Until then we have to make the PyPI api return a ‘pypi’ realm on 401 error (and this was done by Sidnei’s work).

To avoid maintaining several .pypirc files and forcing the realm on 401 errors though, we should create a new register command, that can work with a enhanced version of the file and allow adding passwords for several hosts. IMHO, the disutils package code is PyPI-centric but was primarly intented to work over any release server, so it has to evolve on that point.

About sdist and bdist_egg

We have discussed in my latest entry about having a new command to upload the package in the PSC:

 $ python setup.py plone_upload

The idea is to be able to upload the release to plone.org and to the Cheeseshop in one step. People reacted over this because in my example I used bdist_egg instead of dist for the packaging. I think it’s a false debate because it’s up to the developer to decide how he releases his work, using a tarball that is compiled by the target system, and/or an egg.

So we can just define an enhanced upload command that replaces the original one, to automate the upload on several servers, and let the developer manually call sdist and/or bdist_egg.

Servers could be picked up by the user at the prompt.

Schedule & tasks

This are my plans in PSF this week and next week:

• finish my current work on the branch, so the basic implementation works;
• provide an enhanced version of the register and upload command, for multiple servers uploads. This will be done in a new package, since it’s more like a distutils enhancement;
• implement the trove webservice using PSC categories.

Edit: Just after I posted this, I found Sidnei’s work, so it is basically what I was thinking of. Good job 

How to promote Plone packages today ?

Since eggs became a standard in Zope, distribution of Plone packages can be done directly to the Cheeseshop. This is quite nice since anyone can invoke an upload command like this:

$ python setup.py bdist_egg upload

This makes the package uploaded and available to anyone. This also made anyone able to promote his work without having to set up a web site or to study how the community works on that point.

I am pretty new to the Plone community, and I am trying to find the best way to promote Plone packages we do here.

My guess is that the Cheeseshop is perfect for Python package but not enough for Plone packages: in order to promote them, the best place is the plone.org software center. It brings a tracker, a front page and a release folder. In other words, anyone who wants to publish a Plone product that is seen by the whole community has to take care of uploading its packages into plone.org, or setting a link there, and to the cheeseshop.

If you have a better way to promote your Plone products, let me know !

I think this process could be enhanced a little bit through automation.

Making plone.org pypi-compatible

The Pypi index has two main features PloneSoftwareCenter does not afaik:

• it implements a file_upload method, that is called by setuptools when the upload command is invoked
• it provides package index pages that allow easy_install to look for a package

These two features are very simple, and could be added in Plone Software Center by:

• adding a file_upload view on the products page
• providing an index-compatible view (PSC has a DOAP support though)

In other words, that would allow calling the upload command on plone.org like this:

$ python setup.py bdist_egg upload --repository=http://plone.org/products/

Another command could be added in setuptools to distribute a plone package automatically:

$ python setup.py plone_promote

plone_promote would invoke bdist_egg then make an upload on cheeseshop and/or plone.org. In other words, that would allow package developers to promote their work without pain.

Having such feature would also allow people to create their own Pypi-compatible private software center when they deal with private package they want to make available for instance to private project buildouts.

If people think it’s a good idea, I am willing to code it in PSC (I made a proposal on the bug tracker).

The next issue of The Python Papers (2, 4) is now available, containing my blog entry “eight tips to start with Python“, and many interesting articles, like an introduction to Test-Driven Code Generation. This article explains, after an introduction to test-driven development, how to generate skeletons that contains unit tests ready to run (like ZopeSkel in Plone) . Generated test fixture is a real advantage in complex frameworks to avoid developers to drop TDD.

Read it, it’s free !

A lot of buildout recipes have been created lately at Ingeniweb, you can list them here: http://pypi.python.org/pypi?%3Aaction=search&term=iw.recipe&submit=search.

Let me present them quickly:

• iw.recipe.cmd : yet another command line recipe. This is useful when you need to perform some pre or post installation steps that don’t worth a recipe.
• iw.recipe.fetcher : this recipe is a wget-like command, that knows how to downloads a file given by an url.
• iw.recipe.fss : this recipe creates folders and configuration file when you use File System Storage in your Plone application.
• iw.recipe.squid : automates the configuration of Squid, and is Cache-Fu friendly
• iw.recipe.pound : compiles and installs Pound in your buildout, and creates its configuration file.
• iw.recipe.subversion : this one is a bit like infrae.subversion but it doesn’t make controls of modified files (which can be very long when you update a buildout), and it generates in download-cache when it is given, a tarball of the subversion branch. This allow us to perform offline installation with subversion branches without having to change the cfg files.
• iw.recipe.template : this recipe knows how to render a cheetah template like Paster does for instance. It makes it easy to add a few custom files in our buildouts

If you use them please let us know !

Other recipes are beeing created, like iw.recipe.pen, which does like iw.recipe.pound but for Pen. I will blog sometimes about Pen, which can be used instead of Pound when you need an universal load balancer that can work on any platform.