Re: Thoughts on the future direction of OfflineIMAP

[John Goerzen <jgoerzen@xxxxxxxxxxxx>]

On Tue, Jul 29, 2003 at 10:02:10AM +0100, Rob Holland wrote:
> I found a bug which I assume is in offlineimap (although it may be due to 
> imaplib.py) which has prevented me using offlineimap. I posted to this 
> list, and no-one replied (everyone get ya violins out... ;).

One thing to mention here is that bug reports should be sent as indicated in
the manual (not to the mailing list).

Having said that, I went back and found your message.  The problem appears
to be that some change occured in the local mailbox which OfflineIMAP is
trying to propogate to the server.  OfflineIMAP is really designed for
bidirectional syncing, and I have no server with read-only folders against
which to test, so it would be correct to say that this area of the code
doesn't get exercised much.

imaplib also is perverted in its handling of readonly folders, which causes
some grief.

> If the code were in Perl I would dive in and have a go at debugging it. As 
> its in Python I can't (even read it properly).

I suspect that overall, you can probably make reasonable sense of it without
even knowing Python.  I don't use too many odd constructs.  (imaplib does
use some.)  If you haven't used threads before, that'll probably be the most
difficult thing for you to wrap your head around (but it would be in Perl,
too.)  But those will be going away if the Twisted experiment is successful.

Perl unfortunately just doesn't scale well to a project of this magnitude. 
I don't dispute its tremendous utility for text parsing and general utility
coding, but for a multithreaded app that needs some pretty rigorous error
handling, it just makes life more difficult than Python.

The Python Tutorial at http://www.python.org/doc/current/tut/tut.html will
get you up and running very rapidly.  The Global Module Index at
http://www.python.org/doc/current/modindex.html will help you figure out
what is being done with those "import" statements at the top of each file,
as well as how they're called later.  For instance, if you see:

import re

...

re.search('^a(.+)b$', foo)

you can look in the "re" module to find out what that search call does.

(Incidentally, Python uses Perl regular expressions.)

Beyond that, the trick will be learning the structure of OfflineIMAP.  It's
not too bad either.  Basically, there are three main types of objects that
get passed around: Repository, Folder, and UI.  (All code lives under
offlineimap/ in the distribution).  Repository objects define the properties
of a repository -- what folders are in use, etc.  Folder objects are for --
yup -- folders, and provide methods for adding and deleting messages and
flags.  UI objects handle the user interface side of things, and are
interchangable.  Three repository and folder objects are available: IMAP,
Maildir, and LocalStatus.  The LocalStatus one implements a cache of the
last known status of the local side, which is required for a proper sync. 
All of these objects are derived from an appropriate base class, which
provides general utilities for that object type.  The folder/repository Base
classes provide some synchronization code.  The rest lies in
offlineimap/account.py.  offlineimap/init.py is where everything starts. 
The rest are various utility functions involved with, mostly, parsing what
imaplib gives us.

Having said all that -- I'm working on rewriting things for Twisted.  Some
things will stay the same -- the three object types for instance -- while
others will change.  Twisted should handle read-only folders a lot better.

> So, its not possible (well, feasible I suppose, as I _could_ go and learn 
> python if I really wanted to try and fix this) for me to send bugfix 
> patches in.

It's not too difficult, and Python is a worthy language to learn.  I myself
am a Perl-to-Python convert.

> Soo...time to try isync I suppose. Which is a shame, as everyone suggested 
> I try offlineimap as its quicker and more flexible.
> 
> p.s. the above isn't a "threat" as melodramtic people like to call it, I'm 
> just presenting my view :)

Views are welcome here.  Thanks for sharing yours and permitting me to
evangelize Python a bit :-)

-- John