I like Google Alert, the idea of receiving a mail every day that summarizes all articles related to a given topic is really helpfull when you need to focus on a specific subject for a while.
But this is not enough. I want to receive a mail that points me to any mailing list or planet feed or blogs out there as well, that talks about the topic.
You can’t do it with Google Alerts as far as I know.
Let’s take an example:
I want to receive a daily mail that points me to any mail thread or blog entry, that is related to the word “buildout” or to the word “pycon”.
Basically, to do it manually, I need to read Planet Python, Planet Zope, then take a look at the Python, Zope and Plone mailing lists. It takes at least 10 minutes, and more if you want to read all entries to make sure you won’t miss anything.
Since online systems like Nabble provides RSS feed for mailing lists (don’t find yours ? just add it there !), it is easy to read them as they where regular feeds.
From there, a script that reads all the selected feeds and sends a mail pointing to the entries that match the selected words is simple to write as well, and fill the need.
But don’t code it : Atomisator will let you do this with a few lines of configuration.
Here’s a step-by-step tutorial.
Step 1 – install easy_install
- download ez_setup : http://peak.telecommunity.com/dist/ez_setup.py
- run it with your Python interpreter
Step 2 – install Atomisator and SQLite
- run the command : easy_install Atomisator
- make sure SQLite is installed on your system. If not, install it: http://www.sqlite.org/download.html
Step 3 – create an “atomisator.cfg” file
The content of the file has to be:
[atomisator] store-entries = false sources = rss http://www.nabble.com/Python---python-list-f2962.xml rss http://n2.nabble.com/Plone-f293351.xml rss http://www.nabble.com/Zope---General-f6715.xml rss http://planet.python.org/rss10.xml rss http://www.zope.org/Planet/planet_rss10.xml
filters = buzzwords words.txt
outputs = email email.cfg
This file will look into Planet Python, Planet Zope and various mailing lists (Python, Plone, Zope). Of course you can add or remove feeds in the sources option.
Step 4 – Create the words.txt file
This file contains regular expressions, one per line, that will be used to match the entries. The file has to be saved besides atomisator.cfg.
For our example:
buildout pycon
You can put any expression you want in this file, as long as you have one matching expression per line.
Step 5 – add an email.cfg configuration file.
This is where you define the target emails that will receive the alerts (tos option). You can also specify the from email, or the smtp server location. The file has to be saved besides atomisator.cfg.
In our case it can be:
[email] tos = tarek@ziade.org from = tarek@ziade.org smtp_server = smtp.neuf.fr
Step 6 – Run it !
The command to be called is atomisator (installed by easy_install) followed by the configuration file:
$ atomisator atomisator.cfg
Reading data.
Launching worker for rss - ('http://www.nabble.com/Python---python-list-f2962.xml',)
Launching worker for rss - ('http://n2.nabble.com/Plone-f293351.xml',)
Launching worker for rss - ('http://www.nabble.com/Zope---General-f6715.xml',)
Launching worker for rss - ('http://planet.python.org/rss10.xml',)
Launching worker for rss - ('http://www.zope.org/Planet/planet_rss10.xml',)
Retrieving from rss - ('http://www.nabble.com/Python---python-list-f2962.xml',)
Retrieving from rss - ('http://www.nabble.com/Zope---General-f6715.xml',)
Retrieving from rss - ('http://n2.nabble.com/Plone-f293351.xml',)
Retrieving from rss - ('http://planet.python.org/rss10.xml',)
Retrieving from rss - ('http://www.zope.org/Planet/planet_rss10.xml',)
.................................................................................................................................................
Writing outputs.
Data ready.
Check your mails. This call can be put in a daily cron.
Tested under Mac OS X and Linux.



great…
Comment by Harshad Joshi — November 8, 2008 @ 2:36 pm |
That’s hot! Makes me want to try atomisator
Comment by Alex Clark — November 8, 2008 @ 2:41 pm |
Thas great, I have a try.
Comment by KIPYA — November 8, 2008 @ 5:44 pm |
Hmm, I get following errot when I run it, with your configuration
$ atomisator atomisator.cfg
Traceback (most recent call last):
File “/usr/bin/atomisator”, line 8, in
load_entry_point(’atomisator.main==0.5.2′, ‘console_scripts’, ‘atomisator’)()
File “/usr/lib/python2.5/site-packages/atomisator.main-0.5.2-py2.5.egg/atomisator/main/commands.py”, line 121, in atomisator
processor = DataProcessor(options.config)
File “/usr/lib/python2.5/site-packages/atomisator.main-0.5.2-py2.5.egg/atomisator/main/core.py”, line 79, in __init__
create_session(self.parser.database)
File “/usr/lib/python2.5/site-packages/atomisator.db-0.3.1-py2.5.egg/atomisator/db/session.py”, line 16, in create_session
session = Session()
File “/var/lib/python-support/python2.5/sqlalchemy/orm/session.py”, line 74, in __init__
super(Sess, self).__init__(**local_kwargs)
TypeError: __init__() got an unexpected keyword argument ‘autocommit’
kipya@office:~/.config/atomisator$
Comment by KIPYA — November 8, 2008 @ 6:04 pm |
@Harshad , Alex : Thanks !
@KIPYA: did you have a previous version of SQLAlchemy installed ?
It looks like a compatibility issue. Let me know your SQLAlchemy version and I’ll help you fix that
Comment by Tarek Ziadé — November 8, 2008 @ 9:38 pm |
Mr Ziade.
there are varipus packages available @ pypi.
Which one package to select? Is it workable with MySQL?
Comment by Harshad Joshi — November 9, 2008 @ 8:27 am |
Harshad: you should install “Atomisator”, it will pull the right ones automatically on installation.
It will work with MySQL since it uses SQLALchemy. But in that very particular case, since we don’t need to store the entries, it uses a SQLite memory database that is not persistent.
To use MySQL nevertheless, remove “store-entries = false”
and add a line “database = mysql://login:password@server/database”
see SQLAlchemy documentation.
Comment by Tarek Ziadé — November 9, 2008 @ 9:03 am |
@Tarek Ziadé, after you mentioned it I’ve installed SQLAlchemy 0.5.0rc3.
Now it works. Thx
Comment by KIPYA — November 9, 2008 @ 12:06 pm |
You wrote: “Tested under Mac OS X” but it will only run under OS X 10.4. When used under 10.5 there is a Problem with multiprocessing.
Here the error message:
“The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().”
More info on this Topic can be found there: http://developer.apple.com/technotes/tn2005/tn2083.html#SECDAEMONVSFRAMEWORKS
After working on a 10.4 System there is still some trouble. Generation of the xml file works fine. But when I try to use the email output it breaks with an UnicodeEncodeError. Because this happens with german source feeds I think this is a Problem with umlauts.
Comment by Jesse — November 9, 2008 @ 4:24 pm |
@Jesse: I have tested it under Mac OS X 10.5. Could you provide me the full traceback if you still have it ? Same thing for the unicode encode error.
The second one should be fixed quite fast.
Comment by Tarek Ziadé — November 9, 2008 @ 4:49 pm |
@Jesse, I could reproduce the bug for the UnicodeEncodeError. I have fixed it : http://atomisator.ziade.org/changeset/302%3A6b6c6317e1eb
And you can get it, by upgrading atomisator.outputs, like this:
$ easy_install -U atomisator.outputs
To have the version 0.1.2
Comment by Tarek Ziadé — November 9, 2008 @ 5:59 pm |
@tarek
for the osx 10.5 Problem there is no traceback. It simply repeats the Errormessage “The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().” After a while a popup tells me that python was ended. I had to kill the process to get back my prompt.
Thanks for the upgrade. It works for me.
Comment by Jesse — November 10, 2008 @ 7:21 pm |
@Jesse, weird… What kind of machine is it ? It looks like a flaw in the multiprocessing module..
Comment by Tarek Ziadé — November 10, 2008 @ 10:34 pm |