Bug: X-OfflineIMAP header in wrong place

[Bruce Campbell <bru-offlineimap@xxxxxxxxxxxxxxxxxxxxxx>]

Hi,

Just started using offlineimap, and have noticed a somewhat annoying bug 
in syncing between two IMAP servers.  This bug results in mail messages on 
the destination side being noticably different from the same message on 
the source side.

The problem is the 'X-OfflineIMAP' meta header.  Ideally, the same message 
on both sides, after this header is removed, should be identical. 
However, in 4.0.8 and onwards to 4.0.12, these header is inserted in a 
manner which makes it awkward to actually remove it properly; an annoyance 
if additional mail-handling programs are being used on the destination 
side (eg, SpamAssassin).

To illustrate the problem, a message on the source mail spool may look 
like:

        Received: from nagbeast by XXXX with local
                id 1GLRiX-00054f-GG
                for XXXX@XXXX; Fri, 08 Sep 2006 07:47:25 +1000
        To: XXXX@XXXX
        Subject: Host UP alert for XXXX!

On the destination mail spool after the gentle ministrations of 
offlineimap, two IMAP servers and savemessage_addheader(), these same 
headers become:

        Received: from nagbeast by XXXX with local
        X-OfflineIMAP-878846305-6c6f63616c696d6170-494e424f58: 
1157665717-0884545389509-v4.0.8
                id 1GLRiX-00054f-GG
                for XXXX@XXXX; Fri, 08 Sep 2006 07:47:25 +1000
        To: XXXX
        Subject: Host UP alert for XXXX!

Suddenly, instead of a 'Received' header of 3 lines, I now have a 
'Received' header of one line, and a 'X-OfflineIMAP' header of 3 lines. 
If I wanted to remove the X-OfflineIMAP header later, I would have to 
remove a _line_, rather than the more elegant (and preferred) method of 
removing a _header_ .

The problem is caused by the savemessage_addheader routine in IMAP.py, 
unchanged since at least 4.0.8 .  This inserts the 'X-OfflineIMAP' header 
after the first '\r\n' sequence.  In this case, this is in the middle of a 
header and thus offlineimap has broken the header.  Eeek.  A quick trip 
around the rosary of rfc 822 (3.1.1) may be in order.

There are two solutions.  One is to search for a suitable place that is 
not in the middle of a folded header.  The second is to treat the 
'X-OfflineIMAP' header as a trace header, and simply add it to the start 
of the headers, not bothering to search for a spot.

Here is a tweak for the first suggestion (in IMAP.py).  The tweak for the 
second suggestion is to simply replace the call to find() with an 
assignment to 0:

     def savemessage_addheader(self, content, headername, headervalue):
         ui = UIBase.getglobalui()
         ui.debug('imap',
                  'savemessage_addheader: called to add %s: %s' % (headername,
                                                                   headervalue))
         insertionpoint = content.find("\r\n")

        # poor-man's detection of folded headers. loop until we don't find a
        # space or tab immediately afterwards a "\r\n" combination.
         while content.find( " ", insertionpoint, insertionpoint + 3 ) > -1 or 
content.find( "   ", insertionpoint, insertionpoint + 3 ) > -1 :
             ui.debug('imap', 'savemessage_addheader: insertionpoint found 
folded at %d' % insertionpoint)
             insertionpoint = content.find("\r\n", insertionpoint + 1 )

         # End tweak.

         while content.find( " ", insertionpoint, insertionpoint + 3 ) != -1 
and content.find( "        ", insertionpoint, insertionpoint + 3 ) != -1 :
             insertionpoint = content.find("\r\n", insertionpoint + 1 )

         ui.debug('imap', 'savemessage_addheader: insertionpoint = %d' % 
insertionpoint)

(etc)

Kind regards,

-- 
   Bruce Campbell.