Plone is a great tool, probably one of the best CMS out there, that let you rapidly create an advanced web application. But there are some real pitfalls you might fall into if you are not familiar to the underlying technology. This post provides a list of tips that can help you out. I am not pretending to give you a set of solutions to make your website fast and scalable, but by following those tips, you will think the way experienced Zope developers think when they code an application.
-
Ask you customer what will be the load.
It’s very important to know it when you are starting a Plone project. That might affect a lot how you are going to work: if the application needs to deal with hundreds of hits per minute, and store gigas of datas, you won’t organize it the same way than if the load is ridiculous, like a few hits a day. Project the data load in the future also: how big will be the site in one month ? one year ? If the data load grows, you might need to set up a cyclic purge to make sure the stuff doesn’t get huge.
-
Ask yourself what data will be stored in the ZODB
The ZODB is a great database, and provides a lot of features for CMS: each object (e.g. Document) is automatically saved on each change. This persistent layer, à la Hibernate, is nice. But this has a cost: if hundreds of objects are created by hundreds of users per minute, it won’t work. It will technically work but will become very slow because the ZODB has to deal with concurrency changes on the objects. You might argue that this won’t be a problem if you take care of who changes the data and how, but you won’t be able to deal with how base objects work in the ZODB. For example, BTrees objects, that are supposed to be fast and to be able to deal with many items, are not really scalable on writes: on my Intel MacBook, on a BTreeFolder containing 1000 items, running 4 concurrent threads that are creating objects in a pace of 300 ms, will generate conflict error on 10% of the requests. You should really ask yourself if you need ZODB features on some data. Maybe they will fit better in a SQL database if they don’t need versionning or sophisticated workflows.
The ZCatalog in its classical shape, is also a particular case. It can weight 40% of the ZODB total size, so if you index a lot of things, and a lot of features on your websites are based on queries, it’s maybe a good idea to used a specialized database, like Xapian or Lucene. It’s faster, and it won’t make your ZODB grow. Hey, why indexes are stored in the ZODB anyway ?
-
Don’t hide your code behind caches
SQUID, memcached, CacheFu. All those tools are wonderful and mandatory when you put your site in production. But you should not hide your code behind them : that’s the best way to code a crappy, badly architectured application. You should take care on how your application scales and on your code effectiveness before you think about caching. Be careful about the complexity of each of your function, and how they scale.
-
Be careful of the pound/ZEO mirage
Making a cluster of ZEO nodes will not make your application faster, it will just raise the number of concurrent threads your application can handle simultaneously. Each node needs to be synchronized behind the scene everytime a data is changed. So the more nodes you have the more network traffic you will get. This can damage the overall performance of the application as well.
There’s a lot of work going on in this area. I am looking forward for what people will do on this sprint for example: http://plone.org/events/sprints/copenhagen-performance-sprint
You should change the title of the post. Most of this stuff doesn’t really have anything to do with the ZODB.
Most of point 1: Don’t blame the ZODB for Plone’s lack of speed. Most of your first point seems to deal with expected hits, nothing the ZODB should have a problem with. Only where it comes to the size of data *written* to the site is the ZODB under consideration. But in my experience, even an 8GB ZODB is quite speedy. What I agree with is that the customer should think about the lifespan and amount of the data.
Point 2 has some valid stuff for sites with high load. Keep in mind though that most of these problem won’t be so much of a problem, if the application above ZODB and the ZCatalog aren’t such a hog when it comes to speed. In mentioning the Catalog and high load, a namedrop towards QueueCatalog would be useful, from what I hear it’s a bitch to set up, but it can help a lot.
Point 3: You are very right here! Ehmm…. what has it to do with the ZODB though? And how is someone who decided to run Plone going to change anything here? (Yes, maybe by going to that sprint you mention below.) But for “normal people”, as soon as they run Plone and mention that it’s slow all they will ever hear is “cache! cache! cache!”…
Point4: ….and of course “ZEO! ZEO! ZEO!”. I agree here that people shouldn’t try to go to a massive ZEO setup if they don’t *know* it’s the right thing in their situation. But again: what does it have to do with the ZODB?
Just my 0.2 cents
Comment by betabug — October 4, 2007 @ 7:42 am |
@betabug: Hi, thanks for the feedback
point 1: how long it takes for your 8 gigas ZODB to pack ? Does
the folder that used to work perfectly, that now have thousands of elements is still responding fast ? Conflict errors will appear also on some situations.
point 2: QueueCatalog seems to be an interesting stuff indeed. I’m also all for a solution that doesn’t store indexes in the ZODB. I have tested such a model in the past, and it works well.
point 3: well, you have to be careful about scaling your application *with the ZODB model in mind*: you won’t scale it the same way you would do with other application models because of the *transparent persistency*
point 4: replication of course, ZEO means data replication in the ZODB.
It’s because of the ZODB model that we have such network transferts.
I’m not trying to blame the ZODB, but to explain that it’s something to be aware of when writing Plone apps. Yesterday I dealt with a website that had a lot of conflicts errors and I finally discovered that a view was changing a persistent object parameter in the code called to display the page. That was putting the whole stuff on the knees on high loads. OK it’s bad to make some ZODB writes in such code, but it is not obvious and very specific to this technology imho.
Comment by Tarek Ziadé — October 4, 2007 @ 8:36 am |
Yes, packing the 8gig ZODB took a while. The response time didn’t have a problem though, size of the ZODB doesn’t really slow it down in operation.
One BTreeFolder2 based object that I have now has 8000+ objects in it. I had stress tested it with 16000 (target size is about 12000) and there are no speed problems, even on my dev workstation (dual 1Ghz G4). The data model for this object is really on the border of moving it to SQL – but since we have no other need for an SQLDB in our app, I’m avoiding the added complexity of yet another server component.
As you know, conflict errors appear with modern Zope’s only on the writes. In our app here we have one hotspot where we get conflict errors, because we do a “write-on-read” there. Not nice, but with the load on that one it’s not a problem – as you said, people should look at the data model and data usage patterns and then decide what’s necessary. For me it just means that I get mentions of resolved conflicts in the logs, for other situations it might be a real problem. Of course I don’t use Plone, so my life is more tranquil and easy in general, while some people seem to like the adrenaline :-p
Comment by betabug — October 4, 2007 @ 8:56 am |
@betabug: I agree with the fact that having several data repositories (SQ, ZODB) adds complexity. On the other hand, it simplifies the application administration: writing a Python script to work with the SQL on the background (to do some admin tasks, reports, purge, etc) is easier an safer than stressing the database that is used to calculate and display web pages.
About the write-on-read: yes, I should have been more precise about this, the problems occurs on writings, and readings might get affected then, even though they are not done by the same user. But this is on very high loads for sure.
Comment by Tarek Ziadé — October 5, 2007 @ 10:12 am |