Re: Offlineimap crashes when trying to sync a folder containing messages from the vim-users list

[hdv@xxxxxxxxx (J.A. de Vries)]

On 2006-07-01 @ 04:52:35 (week 26) John Goerzen wrote:

> On Sat, Jul 01, 2006 at 09:42:13AM +0000, J.A. de Vries wrote:
> > Hi,
> >   File "/usr/lib/python2.3/site-packages/offlineimap/folder/IMAP.py", line 
> > 114, in getmessage
> >     return initialresult[1][0][1].replace("\r\n", "\n")
> > TypeError: unsubscriptable object
> 
> Sounds like the IMAP server is sending something odd across, so look
> towards the bottom of your debug log and see if you can see anything
> interesting.

Hi John,

Sorry. It sure took a long time for me to find some time to delve
deeper into this...

After a thorough look at the logs I couldn't find anything more than
what I already posted. However the output at the cli gave me something
to work on:

#-------------------------------------------------------------------------------
Thread 'Copy message 2504 from Applications.[vim]' terminated with exception:
Traceback (most recent call last):
  File "/var/lib/python-support/python2.4/offlineimap/threadutil.py", line 153, 
in run
    Thread.run(self)
  File "threading.py", line 422, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/var/lib/python-support/python2.4/offlineimap/folder/Base.py", line 
272, in copymessageto
    message = self.getmessage(uid)
  File "/var/lib/python-support/python2.4/offlineimap/folder/IMAP.py", line 
114, in getmessage
    return initialresult[1][0][1].replace("\r\n", "\n")
TypeError: unsubscriptable object


Last 1 debug messages logged for Copy message 2504 from Applications.[vim] 
prior to exception:
imap: Returned object from fetching 2504: ('OK', [None])
#-------------------------------------------------------------------------------

So that got me started to find out how I could see what message 2504 contained
and if there was something unexpected about one or more line-endings in it. At
first I didn't know how to do that, but some poking around led me to a file
called dovecot-uidlist in the Maildir directory for the Applications.[vim]
folder. (dovecot version 0.99.14-1sarge0 is the IMAP daemon I am using on this
Debian system.) In that file I found a line like this:

#-------------------------------------------------------------------------------
# grep '^2504' ~/Maildir/.Applications.\[vim\]/dovecot-uidlist
2504 1141914051.M484859P8695V0000000000000309Ip0007553_0.draupnir,S=3707
#-------------------------------------------------------------------------------

However when I try to find that file in the cur directory something weird
happens:

#-------------------------------------------------------------------------------
# cd ~/Maildir/.Applications.\[vim\]/cur
# ls 1141914051.M484859P8695V0000000000000309Ip0007553_0*
ls: 1141914051.M484859P8695V0000000000000309Ip0007553_0.draupnir,S=3707:2,S: No 
such file or directory
#-------------------------------------------------------------------------------

Just to be sure I did the same for some other files and those were found just
fine.

So the systems tells me the file is non-existant, but it does know the full
name of that same file without me telling it. I don't understand how that is
possible, but that probably is just due to lack of knowledge on my part.
(The problem persists over reboots, though. So it is not due to some
form of caching.)

Problem for me is that offlineimap probably gets told by dovecot that
such a file is present, but when it tries to sync that file it dies
because it is not available. At least that's what I am making of it,
based on the idea that trying to replace "\r\n" with "\n" without actual
input (unscriptable object) could make offline imap choke. 

So I tried deleting that line from the dovecot-uidlist and restarted
dovecot. Alas, to no avail: the problem persists.

After that I did a recursive grep on the ~/.offlineimapfolder on the
local system for 2504, hoping maybe that UID was cached somewhere, but
that didn't help either.

You most probably don't need me telling you this (just trying to be
useful, so please don't take this the wrong way), but could it be that
some form of added exception-handling at the right place could help
prevent the program from choking when trying to read from a non-existant
message?

I'd like to temporarily remove just this 1 folder from the synclist, so
I can at least sync the rest of the mail. However I am not familiar with
Python, so I am not sure what I need to change in the search clause for
this account. Currently I have this in my ~/.offlineimaprc:

#-------------------------------------------------------------------------------
#Specify which folders to sync
folderfilter = lambda foldername: \
               re.search('[^~]$', foldername)
#-------------------------------------------------------------------------------

I do know regexes, but do not know how to combine the current one with
an exclusion for the vim folder in Python. Do I need to use multiple
re.search clauses, or do I need to combine it all in one regexp? Would
this work?

#-------------------------------------------------------------------------------
#Specify which folders to sync
folderfilter = lambda foldername: \
               not re.search('~$|^Applications\.\[vim\]$', foldername)
#-------------------------------------------------------------------------------

Thanks for your time.

Sincerely,

J.A. de Vries