Re: Google Chats with offlineimap

[Ng Oon-Ee <ngoonee@xxxxxxxxx>]

On Thu, 2009-08-13 at 10:42 +0800, Ng Oon-Ee wrote:
> On Thu, 2009-08-06 at 06:56 +0800, Ng Oon-Ee wrote:
> > On Wed, 2009-08-05 at 15:08 -0500, John Goerzen wrote:
> > > Ng Oon-Ee wrote:
> > > > Hi all,
> > > > 
> > > > Sometimes my friends forward me google chats, which have additional
> > > > headers which look like this:-
> > > > 
> > > > X-Google-Chat-Start: 1249355382000
> > > > X-Google-Chat-End: 1249360592000
> > > > X-Google-Chat-With: someone's.email@xxxxxxxxx
> > > > 
> > > > Offlineimap's python parser chokes on uploading these messages to the
> > > > IMAP server. Removing those lines fixes the issue, but I have to
> > > > manually do that everytime I get one of these emails. Note, there is no
> > > > problem with downloading from the server, but I tend to file my emails
> > > > to another folder, and it can't upload from that folder to the server.> 
> > > 
> > > I think we'd have to see the last bits of a debug log and the specific
> > > error messages to have any idea what's going on.
> > 
> > Ah, silly me, forgot to change the header previously, sorry for messing
> > up people's threading.
> > 
> > I will include the debug logs as soon as I get another google chat
> > email, I manually correct them when I get them, since I do need to get
> > other mails. Will follow up.
> > 
> 
> Whaddaya know, when I finally get a google chat, it works flawlessly
> now. I have not updated offlineimap in the meantime (there was a release
> today, wasn't there?) so that won't explain it. I did check the mail
> itself, it had those three lines as above but synced flawlessly.
> 
> Will continue to keep an eye out, if it happens again will remember to
> reproduce and send.
> 
Finally I got an email with the same error message. While cutting it
down to minimal size (basically removing the email) I noticed a few
things. Firstly, if I get it to the condition that it IS working
(removing X-Google-Chat lines does this, as well as changing the names
of the sender/chatted-with-person), the next sync will be successful as
well, even though I revert my local copy to the non-working one. This
seems to indicate that there is no problem if offlineimap's UID matches
the one on Gmail's servers. The problem ONLY appears when I have a 'new'
mail which has been copied into my maildir (for example, when moving a
mail from my Inbox to its storage folder in Evolution).

Anyway, am attaching the minimal message as far as I can figure it out,
which always gives me the error if I paste it into a maildir (does not
exist on my server-side gmail), as well as the log of the output from
offlineimap with all the errors.


-- Attached file included as plaintext by Ecartis --
-- File: 1251248551.M558502P8706.ngoonee-laptop:2,S

Delivered-To: ngoonee@xxxxxxxxx
Received: by 10.150.152.20 with SMTP id z20cs326810ybd;
        Mon, 24 Aug 2009 23:36:37 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.115.98.40 with SMTP id a40mr7307680wam.97.1251182196973; Mon, 
        24 Aug 2009 23:36:36 -0700 (PDT)
X-Google-Chat-Start: 1251179660000
X-Google-Chat-End: 1251181930000
X-Google-Chat-With: someoneelse@xxxxxxxxx
In-Reply-To: <32551448.28040341251181930451.chat@xxxxxxxxx>
References: <32551448.28040341251181930451.chat@xxxxxxxxx>
Date: Tue, 25 Aug 2009 14:36:36 +0800
Message-ID: <f80c2e820908242336y2f9020bbhf1cea9affaa3b8d7@xxxxxxxxxxxxxx>
Subject: Chat with Kwan Zhen Wei
From: Shu Yi <someone@xxxxxxxxx>
To: Oon-Ee <ngoonee@xxxxxxxxx>
Content-Type: multipart/alternative; boundary=0016e648f0546662860471f18ecb

--0016e648f0546662860471f18ecb
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

--0016e648f0546662860471f18ecb--


-- Attached file included as plaintext by Ecartis --
-- File: offlineimapdebug

OfflineIMAP 6.2.0
Copyright (C) 2002 - 2009 John Goerzen <jgoerzen@xxxxxxxxxxxx>
This software comes with ABSOLUTELY NO WARRANTY; see the file
COPYING for details.  This is free software, and you are welcome
to distribute it under the conditions laid out in COPYING.
***** Processing account MyAccount
Copying folder structure from Gmail to Maildir
Establishing connection to imap.gmail.com:993.
Syncing INBOX: Gmail -> Maildir
Syncing [Gmail]/Drafts: Gmail -> Maildir
Syncing [Gmail]/Friends: Gmail -> Maildir
Establishing connection to imap.gmail.com:993.
Establishing connection to imap.gmail.com:993.
Syncing [Gmail]/Friends/C2G 2003: Gmail -> Maildir
Establishing connection to imap.gmail.com:993.
Syncing [Gmail]/Friends/Eryl Hor: Gmail -> Maildir
Establishing connection to imap.gmail.com:993.
Syncing [Gmail]/Friends/Gang of 7: Gmail -> Maildir
Syncing [Gmail]/Friends/IPots BUZZ: Gmail -> Maildir
Syncing [Gmail]/Friends/Mechatron: Gmail -> Maildir
Syncing [Gmail]/Friends/Post-grad: Gmail -> Maildir
Syncing [Gmail]/Friends/Sharon Tan: Gmail -> Maildir
Syncing [Gmail]/Friends/Shu Yi: Gmail -> Maildir
Syncing [Gmail]/General: Gmail -> Maildir
Syncing [Gmail]/General/Asus: Gmail -> Maildir
Copy message -1 Maildir[root/Friends/Shu Yi] -> Gmail[[Gmail]/Friends/Shu Yi], 
LocalStatus[root.Friends.Shu Yi]
Syncing [Gmail]/General/Mailing Lists: Gmail -> Maildir
Syncing [Gmail]/General/Malaysia: Gmail -> Maildir
Syncing [Gmail]/General/Non-serious: Gmail -> Maildir
Syncing [Gmail]/General/Stories: Gmail -> Maildir
Copy message -1 Maildir[root/Friends/Shu Yi] -> LocalStatus[root.Friends.Shu Yi]
Syncing [Gmail]/Prayer: Gmail -> Maildir
Thread 'New msg sync from root/Friends/Shu Yi' terminated with exception:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/offlineimap/threadutil.py", line 149, 
in run
    Thread.run(self)
  File "/usr/lib/python2.6/threading.py", line 477, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.6/site-packages/offlineimap/folder/Base.py", line 212, 
in syncmessagesto_neguid_msg
    successuid = tryappend.savemessage(uid, message, flags, rtime)
  File "/usr/lib/python2.6/site-packages/offlineimap/folder/IMAP.py", line 297, 
in savemessage
    date, content)[0] == 'OK')
AssertionError


Last 11 debug messages logged for New msg sync from root/Friends/Shu Yi prior 
to exception:
imap: savemessage: called
imap: savemessage: using date "26-Aug-2009 09:48:53 +0800"
imap: savemessage: initial content is: 'Delivered-To: 
ngoonee@xxxxxxxxx\r\nReceived: by 10.150.152.20 with SMTP id 
z20cs326810ybd;\r\n        Mon, 24 Aug 2009 23:36:37 -0700 
(PDT)\r\nMIME-Version: 1.0\r\nReceived: by 10.115.98.40 with SMTP id 
a40mr7307680wam.97.1251182196973; Mon, \r\n\t24 Aug 2009 23:36:36 -0700 
(PDT)\r\nX-Google-Chat-Start: 1251179660000\r\nX-Google-Chat-End: 
1251181930000\r\nX-Google-Chat-With: someoneelse@xxxxxxxxx\r\nIn-Reply-To: 
<32551448.28040341251181930451.chat@xxxxxxxxx>\r\nReferences: 
<32551448.28040341251181930451.chat@xxxxxxxxx>\r\nDate: Tue, 25 Aug 2009 
14:36:36 +0800\r\nMessage-ID: 
<f80c2e820908242336y2f9020bbhf1cea9affaa3b8d7@xxxxxxxxxxxxxx>\r\nSubject: Chat 
with Kwan Zhen Wei\r\nFrom: Shu Yi <someone@xxxxxxxxx>\r\nTo: Oon-Ee 
<ngoonee@xxxxxxxxx>\r\nContent-Type: multipart/alternative; 
boundary=0016e648f0546662860471f18ecb\r\n\r\n--0016e648f0546662860471f18ecb\r\nContent-Type:
 text/plain; charset=ISO-8859-1\r\nContent-Transfer-Encoding: 
7bit\r\n\r\n--0016e648f0546662860471f18ecb--\r\n'
imap: savemessage: new headers are: 
X-OfflineIMAP-1023824374-4d79476d61696c-5b476d61696c5d2f467269656e64732f536875205969:
 1251251795-0705667402651-v6.2.0
imap: savemessage_addheader: called to add 
X-OfflineIMAP-1023824374-4d79476d61696c-5b476d61696c5d2f467269656e64732f536875205969:
 1251251795-0705667402651-v6.2.0
imap: savemessage_addheader: insertionpoint = 31
imap: savemessage_addheader: leader = 'Delivered-To: ngoonee@xxxxxxxxx'
imap: savemessage_addheader: newline = 
'\r\nX-OfflineIMAP-1023824374-4d79476d61696c-5b476d61696c5d2f467269656e64732f536875205969:
 1251251795-0705667402651-v6.2.0'
imap: savemessage_addheader: trailer = '\r\nReceived: by 10.150.152.20 with 
SMTP id z20cs326810ybd;\r\n        Mon, 24 Aug 2009 23:36:37 -0700 
(PDT)\r\nMIME-Version: 1.0\r\nReceived: by 10.115.98.40 with SMTP id 
a40mr7307680wam.97.1251182196973; Mon, \r\n\t24 Aug 2009 23:36:36 -0700 
(PDT)\r\nX-Google-Chat-Start: 1251179660000\r\nX-Google-Chat-End: 
1251181930000\r\nX-Google-Chat-With: someoneelse@xxxxxxxxx\r\nIn-Reply-To: 
<32551448.28040341251181930451.chat@xxxxxxxxx>\r\nReferences: 
<32551448.28040341251181930451.chat@xxxxxxxxx>\r\nDate: Tue, 25 Aug 2009 
14:36:36 +0800\r\nMessage-ID: 
<f80c2e820908242336y2f9020bbhf1cea9affaa3b8d7@xxxxxxxxxxxxxx>\r\nSubject: Chat 
with Kwan Zhen Wei\r\nFrom: Shu Yi <someone@xxxxxxxxx>\r\nTo: Oon-Ee 
<ngoonee@xxxxxxxxx>\r\nContent-Type: multipart/alternative; 
boundary=0016e648f0546662860471f18ecb\r\n\r\n--0016e648f0546662860471f18ecb\r\nContent-Type:
 text/plain; charset=ISO-8859-1\r\nContent-Transfer-Encoding: 
7bit\r\n\r\n--0016e648f0546662860471f18ecb--\r\n'
imap: savemessage: new content is: 'Delivered-To: 
ngoonee@xxxxxxxxx\r\nX-OfflineIMAP-1023824374-4d79476d61696c-5b476d61696c5d2f467269656e64732f536875205969:
 1251251795-0705667402651-v6.2.0\r\nReceived: by 10.150.152.20 with SMTP id 
z20cs326810ybd;\r\n        Mon, 24 Aug 2009 23:36:37 -0700 
(PDT)\r\nMIME-Version: 1.0\r\nReceived: by 10.115.98.40 with SMTP id 
a40mr7307680wam.97.1251182196973; Mon, \r\n\t24 Aug 2009 23:36:36 -0700 
(PDT)\r\nX-Google-Chat-Start: 1251179660000\r\nX-Google-Chat-End: 
1251181930000\r\nX-Google-Chat-With: someoneelse@xxxxxxxxx\r\nIn-Reply-To: 
<32551448.28040341251181930451.chat@xxxxxxxxx>\r\nReferences: 
<32551448.28040341251181930451.chat@xxxxxxxxx>\r\nDate: Tue, 25 Aug 2009 
14:36:36 +0800\r\nMessage-ID: 
<f80c2e820908242336y2f9020bbhf1cea9affaa3b8d7@xxxxxxxxxxxxxx>\r\nSubject: Chat 
with Kwan Zhen Wei\r\nFrom: Shu Yi <someone@xxxxxxxxx>\r\nTo: Oon-Ee 
<ngoonee@xxxxxxxxx>\r\nContent-Type: multipart/alternative; 
boundary=0016e648f0546662860471f18ecb\r\n\r\n--0016e648f0546662860471f18ecb\r\nContent-Type:
 text/plain; charset=ISO-8859-1\r\nContent-Transfer-Encoding: 
7bit\r\n\r\n--0016e648f0546662860471f18ecb--\r\n'
imap: savemessage: new content length is 1063