[agriffis@xxxxxxxxxx: problems with threads]

[Aron Griffis <agriffis@xxxxxxxxxx>]

Hi,

I continue to see the problem reported below.  It happens about once
per month and forces me to purge the state files and redownload the
INBOX from the server.  I have noticed that it only appears to happen
with Cyrus IMAP server (but I don't have control over what is run on
that server).

Full debug logs (minus passwords and such) are available at

    http://gentoo.org/~agriffis/offlineimap_bug/

Regards,
Aron

----- Forwarded message from Aron Griffis <agriffis@xxxxxxxxxx> -----

From: Aron Griffis <agriffis@xxxxxxxxxx>
Date: Mon, 3 May 2004 20:02:20 -0400
To: offlineimap@xxxxxxxxxxxx
User-Agent: Mutt/1.5.6i
Subject: problems with threads

Hello,

I have been using offlineimap for a few months now and enjoy it
immensely.  I use it from two workstations and one laptop to
synchronize with two courier and one cyrus IMAP servers.  I run it
from cron on all three machines with some checking to determine if
it's truly necessary to run (i.e. is mutt running, is there mail in
the Outbox, etc)

When I first started using offlineimap, I avoided the threading
feature for the sake of sanity.  Today I added maxsyncaccounts=4 (I
have four accounts total) and maxconnections=2 to each account.
I did this on only one of the client machines as an experiment.

Soon after, I saw the attached logs on another client machine (which
is still running maxsyncaccounts=1 and maxconnections=1).  I
understand that I can probably blow away my local folders and
LocalStatus files to fix the current corruption, but this experience
makes me very concerned that the threading feature has the potential
to cause corruption.

Regards,
Aron

--
Aron Griffis
Gentoo Linux Developer


0 time ~ $ offlineimap 
OfflineIMAP 4.0.0 (Rev 509)
Copyright (C) 2002, 2003 John Goerzen <jgoerzen@xxxxxxxxxxxx>
This software comes with ABSOLUTELY NO WARRANTY; see the file
COPYING for details.  This is free software, and you are welcome
to distribute it under the conditions laid out in COPYING.
***** Processing account HomeRecv
Copying folder structure from IMAP to Maildir
Establishing connection to tunnel:ssh -q griffis1.net /usr/sbin/courier-imapd 
.maildir.
INFO: LOGIN, user=user, ip=[127.0.0.1], protocol=IMAP
Syncing INBOX: IMAP -> Maildir
Deleting 2 messages (60201, 60202) in Maildir[INBOX], LocalStatus[INBOX]
Syncing INBOX.Trash: IMAP -> Maildir
Syncing INBOX.family: IMAP -> Maildir
Syncing INBOX.gentoo: IMAP -> Maildir
Deleting 1 messages (5062) in Maildir[gentoo], LocalStatus[gentoo]
Copy message 5070 IMAP[INBOX.gentoo] -> Maildir[gentoo], LocalStatus[gentoo]
Copy message 5071 IMAP[INBOX.gentoo] -> Maildir[gentoo], LocalStatus[gentoo]
Adding flags S to 1 messages  on Maildir[gentoo]
Adding flags S to 1 messages  on LocalStatus[gentoo]
Syncing INBOX.gentoo-alpha: IMAP -> Maildir
Syncing INBOX.gentoo-core: IMAP -> Maildir
Syncing INBOX.gentoo-dev: IMAP -> Maildir
Syncing INBOX.mutt-users: IMAP -> Maildir
Syncing INBOX.ruby-talk: IMAP -> Maildir
Copy message 6128 IMAP[INBOX.ruby-talk] -> Maildir[ruby-talk], 
LocalStatus[ruby-talk]
Syncing INBOX.vim-dev: IMAP -> Maildir
INFO: LOGOUT, user=user, ip=[127.0.0.1], headers=0, body=7648
***** Finished processing account HomeRecv
***** Processing account HomeSend
Copying folder structure from IMAP to Maildir
Establishing connection to tunnel:ssh -q griffis1.net \
'env SENDMAIL=/usr/sbin/sendmail OUTBOX=.Outbox /usr/sbin/courier-imapd 
.maildir'.
INFO: LOGIN, user=user, ip=[127.0.0.1], protocol=IMAP
Syncing INBOX.Outbox: IMAP -> Maildir
***** Finished processing account HomeSend
INFO: LOGOUT, user=user, ip=[127.0.0.1], headers=0, body=0
***** Processing account WorkSend
Copying folder structure from IMAP to Maildir
Establishing connection to kaf.zk3.dec.com:993.
Syncing INBOX.Outbox: IMAP -> Maildir
***** Finished processing account WorkSend
***** Processing account WorkRecv
Copying folder structure from IMAP to Maildir
Establishing connection to imaphub.zk3.dec.com:143.
Syncing INBOX: IMAP -> Maildir
Deleting 2 messages (315438, 315444) in IMAP[INBOX], LocalStatus[INBOX]
Adding flags S to 2 messages  on IMAP[INBOX]
Adding flags S to 2 messages  on LocalStatus[INBOX]
Copy message 315446 IMAP[INBOX] -> Maildir[INBOX], LocalStatus[INBOX]
Adding flags e to 1 messages  on Maildir[INBOX]
Adding flags ) to 1 messages  on Maildir[INBOX]
Adding flags ( to 1 messages  on Maildir[INBOX]
Adding flags n to 1 messages  on Maildir[INBOX]
Adding flags S to 1 messages  on Maildir[INBOX]
Adding flags R to 2 messages  on Maildir[INBOX]
Adding flags \ to 1 messages  on Maildir[INBOX]
Adding flags e to 1 messages  on LocalStatus[INBOX]
Adding flags ) to 1 messages  on LocalStatus[INBOX]
Adding flags ( to 1 messages  on LocalStatus[INBOX]
Adding flags n to 1 messages  on LocalStatus[INBOX]
Adding flags S to 1 messages  on LocalStatus[INBOX]
Adding flags R to 2 messages  on LocalStatus[INBOX]
Adding flags \ to 1 messages  on LocalStatus[INBOX]
Syncing INBOX.clinux: IMAP -> Maildir
Copy message 251 IMAP[INBOX.clinux] -> Maildir[clinux], LocalStatus[clinux]
Syncing INBOX.ia64_rnd: IMAP -> Maildir
Syncing INBOX.linux-ia64: IMAP -> Maildir
Syncing INBOX.sent: IMAP -> Maildir
Copy message 13050 IMAP[INBOX.sent] -> Maildir[sent], LocalStatus[sent]
Syncing INBOX.trading: IMAP -> Maildir
***** Finished processing account WorkRecv

0 time ~ $ offlineimap 
OfflineIMAP 4.0.0 (Rev 509)
Copyright (C) 2002, 2003 John Goerzen <jgoerzen@xxxxxxxxxxxx>
This software comes with ABSOLUTELY NO WARRANTY; see the file
COPYING for details.  This is free software, and you are welcome
to distribute it under the conditions laid out in COPYING.
***** Processing account HomeRecv
Copying folder structure from IMAP to Maildir
Establishing connection to tunnel:ssh -q griffis1.net /usr/sbin/courier-imapd 
.maildir.
INFO: LOGIN, user=user, ip=[127.0.0.1], protocol=IMAP
Syncing INBOX: IMAP -> Maildir
Syncing INBOX.Trash: IMAP -> Maildir
Syncing INBOX.family: IMAP -> Maildir
Syncing INBOX.gentoo: IMAP -> Maildir
Copy message 5072 IMAP[INBOX.gentoo] -> Maildir[gentoo], LocalStatus[gentoo]
Copy message 5073 IMAP[INBOX.gentoo] -> Maildir[gentoo], LocalStatus[gentoo]
Syncing INBOX.gentoo-alpha: IMAP -> Maildir
Syncing INBOX.gentoo-core: IMAP -> Maildir
Syncing INBOX.gentoo-dev: IMAP -> Maildir
Syncing INBOX.mutt-users: IMAP -> Maildir
Syncing INBOX.ruby-talk: IMAP -> Maildir
Syncing INBOX.vim-dev: IMAP -> Maildir
INFO: LOGOUT, user=user, ip=[127.0.0.1], headers=0, body=4539
***** Finished processing account HomeRecv
***** Processing account HomeSend
Copying folder structure from IMAP to Maildir
Establishing connection to tunnel:ssh -q griffis1.net \
'env SENDMAIL=/usr/sbin/sendmail OUTBOX=.Outbox /usr/sbin/courier-imapd 
.maildir'.
INFO: LOGIN, user=user, ip=[127.0.0.1], protocol=IMAP
Syncing INBOX.Outbox: IMAP -> Maildir
***** Finished processing account HomeSend
INFO: LOGOUT, user=user, ip=[127.0.0.1], headers=0, body=0
***** Processing account WorkSend
Copying folder structure from IMAP to Maildir
Establishing connection to kaf.zk3.dec.com:993.
Syncing INBOX.Outbox: IMAP -> Maildir
***** Finished processing account WorkSend
***** Processing account WorkRecv
Copying folder structure from IMAP to Maildir
Establishing connection to imaphub.zk3.dec.com:143.
Syncing INBOX: IMAP -> Maildir
Deleting flags e to 1 messages on IMAP[INBOX]
Syncing INBOX.clinux: IMAP -> Maildir
Thread 'Folder sync WorkRecv[INBOX]' terminated with exception:
Traceback (most recent call last):
  File "/usr/lib/python2.3/site-packages/offlineimap/threadutil.py", line 153, 
in run
    Thread.run(self)
  File "/usr/lib/python2.3/threading.py", line 416, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.3/site-packages/offlineimap/accounts.py", line 222, in 
syncfolder
    localfolder.syncmessagesto(statusfolder, [remotefolder, statusfolder])
  File "/usr/lib/python2.3/site-packages/offlineimap/folder/Base.py", line 388, 
in syncmessagesto
    self.syncmessagesto_flags(dest, applyto)
  File "/usr/lib/python2.3/site-packages/offlineimap/folder/Base.py", line 368, 
in syncmessagesto_flags
    object.deletemessagesflags(delflaglist[flag], [flag])
  File "/usr/lib/python2.3/site-packages/offlineimap/folder/IMAP.py", line 275, 
in deletemessagesflags
    self.processmessagesflags('-', uidlist, flags)
  File "/usr/lib/python2.3/site-packages/offlineimap/folder/IMAP.py", line 288, 
in processmessagesflags
    imaputil.flagsmaildir2imap(flags))
  File "/usr/lib/python2.3/site-packages/offlineimap/imaplib.py", line 673, in 
uid
    typ, dat = apply(self._simple_command, (name, command) + args)
  File "/usr/lib/python2.3/site-packages/offlineimap/imaplib.py", line 976, in 
_simple_command
    return self._command_complete(name, apply(self._command, (name,) + args))
  File "/usr/lib/python2.3/site-packages/offlineimap/imaplib.py", line 813, in 
_command_complete
    raise self.error('%s command error: %s %s' % (name, typ, data))
error: UID command error: BAD ['Invalid flag name  in UID Store command']


Last 50 debug messages logged for Folder sync WorkRecv[INBOX] prior to 
exception:
imap: options2hash called with input: ['FLAGS', '(\\Seen)', 'UID', '315432']
imap: options2hash returning: {'FLAGS': '(\\Seen)', 'UID': '315432'}
imap: imapsplit() called with input: FLAGS () UID 315433
imap: imapsplit() returning: ['FLAGS', '()', 'UID', '315433']
imap: options2hash called with input: ['FLAGS', '()', 'UID', '315433']
imap: options2hash returning: {'FLAGS': '()', 'UID': '315433'}
imap: imapsplit() called with input: FLAGS (\Seen) UID 315434
imap: imapsplit() returning: ['FLAGS', '(\\Seen)', 'UID', '315434']
imap: options2hash called with input: ['FLAGS', '(\\Seen)', 'UID', '315434']
imap: options2hash returning: {'FLAGS': '(\\Seen)', 'UID': '315434'}
imap: imapsplit() called with input: FLAGS () UID 315435
imap: imapsplit() returning: ['FLAGS', '()', 'UID', '315435']
imap: options2hash called with input: ['FLAGS', '()', 'UID', '315435']
imap: options2hash returning: {'FLAGS': '()', 'UID': '315435'}
imap: imapsplit() called with input: FLAGS () UID 315436
imap: imapsplit() returning: ['FLAGS', '()', 'UID', '315436']
imap: options2hash called with input: ['FLAGS', '()', 'UID', '315436']
imap: options2hash returning: {'FLAGS': '()', 'UID': '315436'}
imap: imapsplit() called with input: FLAGS (\Seen) UID 315437
imap: imapsplit() returning: ['FLAGS', '(\\Seen)', 'UID', '315437']
imap: options2hash called with input: ['FLAGS', '(\\Seen)', 'UID', '315437']
imap: options2hash returning: {'FLAGS': '(\\Seen)', 'UID': '315437'}
imap: imapsplit() called with input: FLAGS (\Answered \Seen) UID 315439
imap: imapsplit() returning: ['FLAGS', '(\\Answered \\Seen)', 'UID', '315439']
imap: options2hash called with input: ['FLAGS', '(\\Answered \\Seen)', 'UID', 
'315439']
imap: options2hash returning: {'FLAGS': '(\\Answered \\Seen)', 'UID': '315439'}
imap: imapsplit() called with input: FLAGS (\Seen) UID 315440
imap: imapsplit() returning: ['FLAGS', '(\\Seen)', 'UID', '315440']
imap: options2hash called with input: ['FLAGS', '(\\Seen)', 'UID', '315440']
imap: options2hash returning: {'FLAGS': '(\\Seen)', 'UID': '315440'}
imap: imapsplit() called with input: FLAGS (\Answered \Seen) UID 315441
imap: imapsplit() returning: ['FLAGS', '(\\Answered \\Seen)', 'UID', '315441']
imap: options2hash called with input: ['FLAGS', '(\\Answered \\Seen)', 'UID', 
'315441']
imap: options2hash returning: {'FLAGS': '(\\Answered \\Seen)', 'UID': '315441'}
imap: imapsplit() called with input: FLAGS () UID 315442
imap: imapsplit() returning: ['FLAGS', '()', 'UID', '315442']
imap: options2hash called with input: ['FLAGS', '()', 'UID', '315442']
imap: options2hash returning: {'FLAGS': '()', 'UID': '315442'}
imap: imapsplit() called with input: FLAGS (\Seen) UID 315443
imap: imapsplit() returning: ['FLAGS', '(\\Seen)', 'UID', '315443']
imap: options2hash called with input: ['FLAGS', '(\\Seen)', 'UID', '315443']
imap: options2hash returning: {'FLAGS': '(\\Seen)', 'UID': '315443'}
imap: imapsplit() called with input: FLAGS (\Seen) UID 315445
imap: imapsplit() returning: ['FLAGS', '(\\Seen)', 'UID', '315445']
imap: options2hash called with input: ['FLAGS', '(\\Seen)', 'UID', '315445']
imap: options2hash returning: {'FLAGS': '(\\Seen)', 'UID': '315445'}
imap: imapsplit() called with input: FLAGS (\Seen) UID 315446
imap: imapsplit() returning: ['FLAGS', '(\\Seen)', 'UID', '315446']
imap: options2hash called with input: ['FLAGS', '(\\Seen)', 'UID', '315446']
imap: options2hash returning: {'FLAGS': '(\\Seen)', 'UID': '315446'}


----- End forwarded message -----