Re: LocalStatus as SQLlite

[John Goerzen <jgoerzen@xxxxxxxxxxxx>]

On Tue April 1 2008 4:00:10 am Stewart Smith wrote:
> On Mon, 2008-03-31 at 10:16 -0500, John Goerzen wrote:
> > First off, thank you for your continued work and persistence on this.
>
> Not a problem - just maintaining what i find useful. mmmm... free
> software.
>
> > Referring back to the original thread [1], it looks like there may be a
> > couple of regressions:
> >
> > 1) That we cannot use threading to simultaneously download multiple
> > messages in a single folder;
>
> Yep. One possible way around this would be to add locking to LocalStatus
> so that only one thread can use the object at once (but every other bit
> of code can run in parallel).

That would be ideal.

> This is intentional. Commit == fsync() - so it's a relatively expensive
> operation. When there's been a lot of changes (e.g. marking lots of
> mails as read... common for me on heavy traffic mailing lists), it's the
> difference between being able to sync about 100/s and a lot more /sec...
> and also reduces file system journal traffic and leaves more disk ops
> (sync) for other apps you're probably running at the same time.
>
> If you crash during this part of the sync, it'll just have to re-sync up
> to 100 messages flags.. so I don't think it's much of an issue.

I could see it causing message duplication as well.  OfflineIMAP is designed 
to "crash well" -- to be as consistent as possible in all situations.

I wonder how much of a performance difference you would see if you simply 
change self.doautosave = 1 to self.doautosave = 0 in 
offlineimap/folder/LocalStatus.py

-- John