Thoughts on the future direction of OfflineIMAP

[John Goerzen <jgoerzen@xxxxxxxxxxxx>]

Hello,

Recently, I have been doing a lot of thinking about where to take
OfflineIMAP now that 4.0 is out.  This message is sorta laying down thoughts
"on paper" for my own clarity, and hoping that some of you may be able to
further point me along one direction or another :-)

We have the following major things to address:

1. A problem with any thread in any account shuts down the entire system.

2. imaplib.py (part of the standard Python distro) is highly unwieldy and
   needs to be rewritten.

3. Python-Tk (Tkinter) is showing signs of trouble with multithreaded
   apps and is not the best for non-Unix platforms.  wxPython may be better
   here.

4. Python's threading system is exhibiting signs of bugginess, sometimes
   crashing in the thread code.

As I see it, there are two ways forward: we can address these issues in
Python, or OfflineIMAP can be re-implemented in Java.

Looking at the Java option first:

Java threads are a lot more powerful than Python threads and would easily
solve #1 and #4.  The JavaMail API would likely solve #2.  That leaves us
with #3.  Java has AWT, which can look nice everywhere, but takes a lot of
work and just feels like a pig.  (Yes, I know there's swing, but it doesn't
have the cross-platform support that AWT does yet.  Plus it's even slower.)

It is possible to use Python code in a Java program via Jython, which could
ease the porting effort.

The other plus we get is somewhat easier installation on some platforms
(MacOS X -- just double-click on the jar).

The downsides are these:

 * Java tends to be a lot more "heavyweight" than Python.  It's slower,
   and quality VMs are not as common as Python ones, especially if you
   are looking for Open Source ones.  (Surprising but true!)

 * Windows users will have a more difficult time if/when they are supported.
   Python has the nice py2exe system to wrap up a full interpreter and
   the program in a single .EXE file.  For Java, they'll have to install
   a Java VM and then OfflineIMAP.

 * All the work reimplementing things, especially GUIs.

 * Java does not include curses as part of the default system, so people
   wanting the curses interface will have to install that package
   separately.

---------------
On the staying with Python camp:

Problem #1 (shutting down the entire system) cannot be totally solved in
Python.  It can be worked around by carefully designing an architecture
where each thread that fires off child threads always join()s them and
checks to see if they died because of an exception -- and if so, raise the
same exception for the parent.  Ugly, hackish, but I think it can be made to
work.  There will have to be a "polling" thread that keeps on eye on the
main account thread.

Problem #2 is not too hard to fix.  I already have the basics of an IMAP
library ready.  But there will be a LOT of code in OfflineIMAP to change
(and delete!) because imaplib.py is so pervasive.  So this is probably an
actual tie or disadvantage as opposed to Java.

Problem #3 can be solved with wxPython, but that will still involve
rewriting the GUI interfaces.  My guess is that this will be easier with
wxPython than with AWT.  The downside is that wxPython is not a standard
part of the Python distro, so we're in a similar situation here as the Java
curses users are.

And there's really nothing we can do about problem #4.

----------------------------------------------------
Other languages....

I have considered other programming languages too.  The only other real
contender is Perl, and Perl is just totally unsuited to an application of
this size and complexity IMHO, and its lack of exception handling is enough
to make me rule it out.  Same goes for C.  That leaves C++, which is a bear
to work right on multiple platforms, and not really a language that lends
itself to rapid development.

I may run this by comp.lang.python as well, but I expect I'd have a huge
flamefest if I did, so I'd rather avoid that.

-- John