Python standard lib : give me more withs !
by Tarek Ziadé
I used to write in files like this :
open('somefile', 'w').write(content)
It’s ugly for sure, and a more proper way is :
f = open('somefile', 'w')
try:
f.write(content)
finally:
f.close()
But since Python 2.6, the with statement is superior for this code pattern:
with open('somefile', 'w') as f:
f.write(content)
This is so natural in fact that I am always thinking about with when I work with classes that have a start/stop or open/close behavior.
So, what about adding this behavior into imaplib.IMAP4, ftplib.FTP and smtplib.SMTP ?
So we can write things like this :
>>> from ftplib import FTP
>>> with FTP('ftp.somewhere.com') as ftp:
... ftp.login('someone', 'pass')
... (some code)
...
I am working on a series of patches for this, and wondering if some other classes in the standard library could benefit from this as well..
Fetchez le Python 
heh, kinda reminds me of long gone times when i used to code in pascal. must have been like ’88 or so… 🙂
The zipfile and gzip modules definitely would benefit from this too.
IMO urllib2.urlopen/urllib.request.urlopen would also benefit from this 🙂
I’m considering adding contextmanager support to multiprocessing.pool* and possibly elsewhere in the package as well. We’ll see if I can do it prior to pycon
Indeed, this looks really handy
As Masklinn says above, zipfile would benefit from contextmanagers, especially on both zipfile.ZipFile (which opens an archive) and on ZipExtFile which is returned from zipfile.ZipFile(‘archive.zip’).open(‘file.txt’):
with zipfile.ZipFile(‘archive.zip’) as archive:
with archive.open(‘file.txt’) as f:
for line in f:
print line
Even nicer would be a new constructor in zipfile that opens an archive and a file at the same time. Maybe ZippedFile() would be a good name for this 🙂 :
with zipFile.ZippedFile(‘archive.zip’, ‘file.txt’) as f:
for line in f:
print line
[Hopefully the indentation comes out OK here!]
@Witsch : hehe, my first coding tool was Delphi 1 back in the late 90’s. Pascal is a nice language indeed.
@Masklinn and Stephen : Great ones ! I am going to add those in the issue,
don’t hesitate to continue the discussion over there for those. While I can write the patch, I really need some feedback so we make it perfect before I propose the features in python-dev
@Hors: Right that is another great one, I am going to add it too into the ticket as well.
@Jesse: That would be neat indeed ! I love your package btw, it’s dead simple to use, and helped a lot on Atomisator to make the web crawlers efficients.
Sorry for sounding stupid, but what exactly is wrong with the first one-line version? You say that it is “ugly”, but it fits on one line and is concise.
I am somewhat new to Python and would like to know why this is wrong, as it is something I would do.
@Kris:
It works correctly in cPython because garbage collection in cPython is reference-counting (with cycle detection). In other implementations of Python, like Jython or IronPython, which use different GC techniques, the open file won’t be freed (and closed) until some arbitrary time in the future.
Arguably, even in cPython the correctness of your code shouldn’t be based on vagaries of the implementation (for example, there is at least one effort to change the GC method used in cPython). And the semantics of the __del__ method (which calls .close for files) are more fiddly than you might expect.
(A contrived example…) If the open call succeeded, and then the .write() call failed for some reason, the exception that was raised would keep a reference to the file via the traceback and stack frames. So the file could stay open much longer than you expect.
In general, when dealing with external resources it’s best to be explicit about when they’re released, and the with statement makes that easier to do.
@Stephen
I don’t think the `ZippedFile` has any real use, since a ZipExtFile doesn’t need to be cleaned up/closed (it’s ‘closed’ along with the ZipFile it belongs to).
So something along the lines of
with zipfile.ZipFile(‘archive.zip”) as z:
…. for line in z.open(‘file.txt’):
…. …. # do stuff
is just as good imo.
@Kris
It has the issue of not closing the file explicitely, so the file is only closed when garbage-collected, which happens… depends of the garbage collection scheme, but as a rule you can’t know. And it’s best to control file closing, in order not to get resource conflicts.
By using `with` you know when the file is closed (right at the end of `with`’s scope) and you know it’s correctly cleaned up even if there was an error while opening it.
The one-liner is probably ok for light scripting, but for actual software it’s not a good idea.
@Tarek
Have you already brought it up to python-ideas? That’s usually the place for putting proposals in the wild, and refining them, before opening alteration tickets or creating PEPs.
@Tarek
ah, checking the archives I see you did, sorry about that then.
hells yes