Re: offlineimap imap<->imap reliability?

[John Goerzen <jgoerzen@xxxxxxxxxxxx>]

On Tue, Oct 21, 2003 at 04:04:40PM +0200, James Leifer wrote:
> I had a bunch of questions about offlineimap's reliability:
> 
> * Have people used it successfully for IMAP<->IMAP synchronization
>   between instances of Courier?  Dovecot?  If so, which versions?

I used it between instances of Courier.  I ran the local version on a
preauth tunnel.

I'll be perfectly honest with you and say that IMAP<->IMAP syncing
historically is not as reliable as IMAP<->Maildir syncing.  When I say "not
as relible", I do not mean "it might lose mail"; I mean "it crashes
occasionally." I think that these bugs are almost completely exterminated by
now, however.

> * How robustly does offlineimap behave when it dies or is killed in
>   the middle of a sync?  Does it leave its state files in a consistent
>   state?

OfflineIMAP is designed to be robust in the event of a failure or problem. 
I have never seen the state files corrupted.  Version 5.0 will have even
more safeguards in the face of this.

Even if a state file is corrupted (and likely this would just result in it
being truncated), the worst that will happen is your mail will be
duplicated.

Every caution is taken to prevent OfflineIMAP from ever deleting a message
that it should not.  I have not seen any version of OfflineIMAP, development
or release, ever delete a message of mine that it shouldn't have since back
before the first version was released.  That is not to say that I am making
a legal guarantee here; OfflineIMAP comes with no warranty.

This caution in OfflineIMAP goes beyond simple error checking (which it does
do).  The sync algorithms themselves are designed to minimize any possible
erroneous message loss, and contain various sanity checks.

This is why, for instance, that OfflineIMAP does not support POP3.  POP3
does not provide enough information to reliably uniquely identify a given
message across multiple sessions.  IMAP does (via the UID and UIDVALIDITY
properties).  It's also a big part of the use of Maildirs, which provide a
reliable way to store the information in the message filename without
modifying the message itself.

> * ditto, if one (or both) of the IMAP servers dies or is killed?
> * ditto, if one (or both) of the connections dies or is killed?

When the operating system notices the connection, it will return an error to
OfflineIMAP.  OfflineIMAP will then shutdown.

> * What happens if the state of either IMAP server changes during the
>   course of the sync, e.g. messages are added or deleted, flags are
>   changed?  Do flags or messages get lost?

No.  Generally, these changes will be either brought over or missed in a
given sync.  (Keep in mind that OfflineIMAP typically runs syncs every few
minutes, so they'll be brought over during the next sync.)  It generally
depends on the behavior of the IMAP server.

There is one small potential race condition that I hope to fix for
OfflineIMAP 5.0:  If a new message arrives on the remote end, and the remote
server tells OfflineIMAP about it when it asks for a list of messages in the
folder, but it gets deleted from the remote before OfflineIMAP can download
it, OfflineIMAP will exit with an error.  This will not cause any message
loss, and is exceedingly rare (it only occurs if you are reading mail on the
remote while OfflineIMAP is running, and even then, it's very rare.)

> * What happens if ``conflicts'' arise, i.e. each replica has different
>   flags modified for a particular message since the last sync?

Flags are handled on an as-changed basis; so if the local system added the
seen flag and the remote added replied, the message should end up with both. 
(I'll have to verify this is the case, but I think it is.)

Otherwise, local changes override remote ones.

Finally, deletions override everything, so if a message is deleted on either
end, it gets deleted on both ends, regardless of flag changes.

> * Does offlineimap sync user-defined flags (so-called IMAP keywords)?
>   How are conflicts resolved?

No.

> Well, that's a lot of questions for one message...  Any comments would be
> gratefully received.

Hope this helps.  You've got some honest answers on plusses and minuses of
OfflineIMAP.  The bottom line is that I am confident enough to use it for my
own mail, about which I am very conservative and paranoid :-)

-- John


> 
> Cheers,
> --
> James J. Leifer
> INRIA Rocquencourt
> 
>