Re: offlineimap's handling of moving message

[Asheesh Laroia <asheesh@xxxxxxxxxxx>]

On Wed, 26 Dec 2007, John Goerzen wrote:

> On Wednesday 26 December 2007 4:20:18 pm Joel Mawhorter wrote:
>> Hi all,
>>
>> I'm trying to use offlineimap with Gmail and I'm running into a problem.
>> If I move a message from folder A to folder B using my mail client (mutt),
>> then next time I run offlineimap the message is deleted from folder A on
>> the server and then uploaded to folder B on the server. For large messages
>> this is too inefficient for my uses since the whole message gets sent to
>> the remote server even though it is already there. Is there a way around
>> this or is this an inherent limitation of having a client other than your
>> mail client manage synchronization?

I'd like to chime in with my own thoughts on this.

> Not a reliable one.
>
> OfflineIMAP would have to have a way to know that a new message that suddenly
> appeared in a new folder was originally a message with UID x in folder y,
> and that this message with UID x was still in folder y on the server --
> which may not be the case if you had accessed your IMAP box by other means
> and deleted it.

So long as UIDVALIDITY holds, message with UID x in the folder either 
exists or is gone, but never has different contents.  So the simple way to 
handle this is "Use COPY if possible; if not, fall back to a lame message 
insertion like before".

This addresses the reliability concern completely, I believe.

> IMAP does not have a MOVE command.  It does have a COPY, which OfflineIMAP
> does not use.

> OfflineIMAP also operates on each folder individually.  It would be possible
> to make it detect and use COPY in this situation, but would be a fairly
> significant code change.  Consider: what if you moved a message from folder
> A to folder B, and also a message from folder B to folder A?  You would have
> to scan all folders for changes before you started processing any at all,
> lest you delete on the server the source for a COPY.

That's true, it would take some code.  But OTOH it would be easy if 
offlineimap stored a table of the MD5 or SHA1 hash of each file it 
is aware of.  Then for new messages, you would consult that table and see 
what possible candidates are for COPY; and it doesn't matter which one you 
COPY since they're the same!

That way, you can dramatically decrease network usage, as hoped by the 
thread starter.

> In short, I think that this optimization adds quite a bit of complexity for
> relatively little gain.

"Relatively little gain" depends on your usage model.  This is why I 
thought I'd reply:

offlineimap has been quite bandwidth-intensive for me lately as I've moved 
my mail handling style from "Store everything in INBOX and use it as the 
archive" to "Once you read a message, handle the message and then move it 
to the archive folder".  It means that syncs take a long time, especially 
because people email me large files from time to time.  It means I can't 
just quickly sync up before I head out the door but have to plan to do it 
30 minutes before.

With GMail users of offlineimap, messages get copied and moved all the 
time as I understand it as you add labels to them.

To be clear, I'm not saying that it's worth the Johns's time to do it, nor 
that you have an obligation to either me or the GMail users of 
offlineimap, but I wanted to chime in with a different slant.

-- Asheesh.

-- 
It just doesn't seem right to go over the river and through the woods
to Grandmother's condo.