Plone Paris Sprint wrapup, part #1: How do you use eggs and zc.buildout in your projects ?
by Tarek Ziadé
This is the first post of the wrapups I want to do about the sprint that we had in Paris last week-end. We had a Bird of Feather about how people use eggs and zc.buildout in their projects, how they release and deploy them.
There were some people from Headnet (Anton, Mustapha) and Infrae (Kit, Sylvain) and Ingeniweb (Me), and we compared a bit how we are working with eggs, zc.buildout etc.
That is what I love in our community: companies can share their knowledge and grow up all together, technically speaking.
We all have internal recipes, command-line scripts, and are all relatively happy with zc.buildout. This discussion was very instructive.
From there, I thaught it would be a good idea to launch a new project in the community, on the top of zc.buildout (and maybe zc.sourcerelease), that would provide a common set of tools and deploying best practices, for people that works with the buildout, no matter which framework they use (Silva, Plone, etc.)
The first step for this project is to find the common needs and see if we can join forces to build common tools. To start it up, I decided to wrap up and release our internal set of tools into a single package called iw.releaser and publish it. This is the work I have done during the last months with the help of Gael, to help our team to work with zc.buildout in Plone Projects. It is Subversion dependent at this time.
I am expecting some feedback from Anton and Sylvain to see if we can make it a collective tool.
This package provides:
- a skeleton to build a project structure (buildout, packages; docs, etc.) so all projects have a standardized structure
- a ‘release’ distutils command that releases a package, upload it to the Cheeseshop or other servers, and send a shout out mail
- a set of command line tools, that can be used to deploy a buildout. These commands are doing many things besides launching a buildout building (which is a bit different from zc.sourcerelease)
This package is used at Ingeniweb for a few months now, and I tried to summarize how it is used in the docs. I bet a lot of bugs will be found if you try it, so consider this package as a non-mature package yet.
Join us all ! So you will be able to release and deploy your buildout-based apps with a few command-line calls, 😀
Have you looked at zc.sourcerelease? I believe it’s what Zope Corporation uses for this purpose…
@Martin: yes like I mentioned it, but it just builds the buildout and archive it, as we do more things (like building upgrades distributions). I was thinking about using it for some of the parts where we do similar work.
Yo,
I started using buildout sometime ago and i’m pretty happy with it.
But i had experienced some drawbacks as i used buildout to deploy an entire project from start to end including system libraries.
– online mode can be very problematic: network problems, pypi, no network in production mode.
– buildouts growed fastly and were very hard to maintain.
– some depenencies / eggs can be not very well packaged and are problematic to deploy.
– impacts on future deployments/maintenance were not that easy to predict.
That’s why i initiated a project beyond buildout which lets me deal with dependencies and have only a bunch atomics buildout which drop my dependencies on a common layout.
Then a dependency can reference another dependency according to that layout.
Well, if you want to learn more, you can have a look on http://trac.minitage.org
The project was not public before and what is on the public repo is not working at the moment (not finnished :p)
At least, you can see the roadmap there : http://trac.minitage.org/trac/roadmap
Prior versions to the 0.4 are working, but are not released on the public side.
The documentation is also in refactoring according to 0.4’s changes, however, the main concepts are there.
I add a comment to kiorky post.
Minitage is another way to deploy projects. It uses buildouts but can do much more.
The most important interesting point for me in using minitage is that with this install method you can reproduce locally on your computer the exact configuration of your production server without breaking your system.
In another way, you can, a year later, install a new computer with more recent systems librairies (postgres 11.25, glibc 8.0, …) and reproduce on this computer the exact environnement built with the librairies and software from a year ago (postgres 8.3, glibc 2.6)
And of course, everything works nice.
Another very pleasant point with minitage is that when you have defined your project configuration, even for huge projects using GIS, SGBDR, Zope, PHP and all others things mixed you just have to entry less than 10 commands lines to deploy this on the computer of the new developper that you recruit.
No more time loosed to install all required softwares, configure and deploy others things.
Actually we use it on all ours Plone projetcs and are very very happy with it.
So thanks Kiorky. And I encourage you to give minitage a try.
@Kiorky and Gael: very interesting project indeed. I think that you guys are working on system-level tools that resolves some of the problems people in python-dev and distutils-sig are talking about at this time, which i sto try to provide a clever system-wide packaging system for Python.
Does it provides tools to deploy in it every pieces needed to install/upgrade one given application ? (like a tarball or a self-installer that contains all pieces)
minitage rocks !
it permit us to deploy the same environment as customers. We can developp and test the same environment wtih multiples products and servers as postgres etc.
You just have to write a specific minilay for it.
./minimerge and it rebuil just you have modified automaticly.
With minitage (which is deployed on productions’ servers) you can build any simple or complex projects.
You wan’t a django project, a geo-django, Zope with or without postgres, postgis, mapserver etc with virtual-env, python2.4, 2.5, both etc.?
Just get an existing minilay, complete it and play (work is the good word) 🙂
Idea is simple and powerfull.
minitage is a meta-packager virtual-env compliant and completly O.S. independant. It has been tested on many Linux distributions and works too on Freebsd and MacOsX (10.5 required for Xcode3 Tools).
And it runs on offline mode too !
Kiorky is doing a great job.
0.4 will be (quasi purely pythonified), and of course more documented.
We have dreamt on, Kiorky write it !
Current doc here :
http://www.minitage.org/doc/index.html
@++