Ben Bangert was reacting about zc.buildout doctests, saying that they are hard to read from the PyPI page, and the examples hard to use and follow.
I agree with Ben as these doctests are very hard to read when you are not familiar with zc.buildout testing modules, which provides a set of API the doctests relies on.
But from a developer point of view, adding a feature to such a package is best done through doctests, using zc.buildout.testing goodies. And a developer that is familiar with this package, will find this doctest very useful.
zc.buildout in any case, is trying to structurize its PyPI front page, and push a maximum amount of doc for users, so … kudos !
I think the problem is more about specifying the target readership.
I would like to point another example that comes in zc.buildout: dependency-links.
The main doctest that appears at PyPI as a light, human-readable section: http://pypi.python.org/pypi/zc.buildout#dependency-links
And the very same section is continued in a specific doctest that does not appear on the main page: http://svn.zope.org/zc.buildout/trunk/src/zc/buildout/dependencylinks.txt?rev=81182&view=markup
So what happened here is that the developer specified two kind of readers:
- people that will reach the package through its PyPI page
- people that will go deeper in how the package works, through recipes or tutorials
From there I think there’s a simple guideline that could be applied to enhance the package documentation when adding a feature:
- resume the feature in the main page (long_description) with examples that does not rely on specific testing API. In other words it should be made of plain english and plain python when needed;
- leave doctest that relies on internal testing API as complementary documentation.
- define for each doctest its nature (recipe, tutorial, etc)
How could we help people doing such structuration ?
The distutils metadata could be a good place to do it, by adding an extra_doctests list for example, that would contain a list of doctests. From there, PyPI could display the long_description text as usual, and add a “more information section”.
Let’s take an example:
def _(f)
return open(f).read()
setup('my.package', ...
long_decription=_('README.txt'),
extra_description=
{'recipe': [_('create_this.txt'),
_('do_that.txt')],
'tutorial': [_('how_to_use.txt')
_('how_to_2.txt')]}
From there, PyPI could provide a Table of content, with a structurized documentation, and additional pages for the package, grouped by types (recipe, tutorial) etc. -> Maybe a Sphinx-powered PyPI ? 
By the way, I have another post related, which tries to summarized good pratices in technical writing : http://tarekziade.wordpress.com/2007/02/23/technical-writing-the-seven-laws
I’d claim that the best way to add a feature is through documentation. Not necessarily doctests, though ideally you should be able to test the documentation. If you are documenting a command line interaction, show the command line interaction, then figure out a way to test it. Do NOT mangle your documentation so that its no longer documenting the thing you’re supposed to be showing.
I love the concept of doctests, and they work great when you’re documenting the usage of Python modules and functions that can be shown on a Python interpreter prompt. If they can’t properly handle a command line interaction, then thats a place where the doctests should be improved, but it should not be a reason to ruin the documentation purely for the sake of testing.
Whether another package implements the functions shown being doc tested is irrelevant, as its still showing a Python interpreter, where the reader is meant to be doing stuff on the command line. Perhaps the testing API being used here should be spitting out more readable command line interactions rather than showing usage of the testing API.
On a side-note, I do applaud the effort that went into the zc.buildout docs. They’re very extensive, and clearly work since they’re doctested. I unfortunately am unable to easily repeat the steps as the commands I’m supposed to run are shrouded in variable substitution which makes it painstaking to try and duplicate.
Comment by Ben Bangert — April 5, 2008 @ 5:35 pm |
@Ben: Agree. I’d say that it is easy to obsfuscate doctests in a package that needs a lot of test fixtures. And that at some point, it might be wise to drop doctest usage in favor on regular unit tests. But when you are building your package through doctests, the “doc” part of “doc test” is often forgotten as it is easier to keep the flow of TDD in a single place.
Maybe specific kind of doctests, used as ‘recipe’ and ‘tutorials’ should be the only ‘official’ and visible documentation of a package.
Tough topic..
Comment by Tarek Ziadé — April 7, 2008 @ 12:24 pm |
I couldn’t agree more with Ben. When I was first starting to use buildout the doctest style documentation was not very helpful. All I wanted to see was the actual information I would need to put into my configs! I didn’t want to have to sit and decipher what the documentation was trying to ‘prove’ to me.
Don’t think i’m complaining though. I think it’s great that they have as many docs as they do for buildout. And it looks like the page has grown even more since I was last there.
Comment by claytron — April 8, 2008 @ 10:16 pm |