Re: OfflineIMAP and KMail

[Josh Rickmar <joshua_rickmar@xxxxxxxxxxx>]

Thanks for your response.

I tried to get this fixed using your recommendations, but its not quite 
working the way I expected it to. :)

My ~/.offlineimaprc is all set up to connect to Lavabit over SSL on port 
993, and to sync it to ~/maildir.  The directory layout of my ~/maildir 
looks like this:

Folder1/
Folder1.Subfolder1/
Folder1.Subfolder2/
Folder2/
Inbox/
Inbox/cur   <-- Subfolders in Inbox/
Inbox/new
Inbox/tmp
Folder3/
Folder3.Subfolder1/
..more directories

In addition to these directories made by OfflineIMAP, I also have cur/ 
new/ and tmp/ directories in ~/maildir made by Dovecot (probably 
thinking that ~/maildir was the location of my inbox).

I created my /usr/local/etc/dovecot.conf file by copying it from 
dovecot-example.conf.  Here is the output of 'dovecot -n' (forgive me if 
I missed something important):

FGD135# dovecot -n
# 1.1.1: /usr/local/etc/dovecot.conf
log_path: /var/log/dovecot
log_timestamp: %Y-%m-%d %H:%M:%S
protocols: imap
listen: *:143
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /var/run/dovecot/login
login_executable: /usr/local/libexec/dovecot/imap-login
verbose_proctitle: yes
first_valid_gid: 0
mail_privileged_group: mail
mail_debug: yes
imap_client_workarounds: delay-newmail netscape-eoh tb-extra-mailbox-sep
auth default:
   passdb:
     driver: passwd-file
     args: /home/joshua/dovecot-passdb
   passdb:
     driver: pam
   userdb:
     driver: passwd
     args: /home/joshua/dovecot-passdb


And here is my /home/joshua/dovecot-passdb

joshua:{plain}xxxxx:1001:1001::/home/joshua::userdb_mail=maildir
:/home/joshua/maildir:INBOX=/home/joshua/maildir/Inbox

I have tried it with both the INBOX= and the end and without it, but it 
does not seem to make any difference when finding the real inbox.

When I try to connect to localhost:143 in KMail, it is able to connect 
to dovecot (was able to check for security support, and login worked), 
so the authentication is working.  However, it finds neither the Inbox 
nor the rest of my mail in ~/maildir, but simply says "No New Messages".

 From my perspective, there seems like there is two ways to fix this, 
either do some OfflineIMAP magic and make it sync the Inbox to 
~/maildir/{cur,new,tmp} where Dovecot thinks it is, or do some namespace 
magic in Dovecot (which I'm not sure how to do with the dovecot-passdb 
file).

P.S. Yes, something is wrong with the encoding in the quote, I see a 
bunch of extra stuff (=20, =2D...) in your message.  Not sure why...


David L. Emerson wrote:
> I use kmail and offlineimap together, and -- after doing a lot of=20
> reading/research, there are two key things I remember about kmail:
> 
> 1. It does not handle maildirs correctly -- you do not want to hook it=20
> into a shared maildir. Its implementation is often described=20
> as "broken".
> 
> 2. Its locally cached imap implementation is buggy as well --=20
> specifically, it has been known to accidentally and permanently delete=20
> large quantities of messages.
> 
> =46rom a "setup" perspective, this boils down to something very simple:=20
> When you are creating a new Account, you should NEVER Add Account of=20
> type
> =2D Disconnected IMAP
> OR
> =2D Maildir mailbox
> 
> I'm actually not sure about the regular IMAP, either (not really an=20
> option for me since I am often offline and need the local cache)
> 
> Anyway, I do know that John has done some truly amazing and commendable=20
> work in making offlineimap compatible with many different broken imap=20
> server implementations, so offlineimap will almost definitely be better=20
> at communicating with most imap servers than kmail will be -- even=20
> without local cacheing.
> 
> I don't know about that "local mailbox" option, either ... I assume=20
> that's for, e.g., reading an old mozilla mailbox. I think I even tried=20
> it once.
> 
> 
> Anyway, the way I managed to get everything up and running is like so:
> 
> SERVER <---> offlineimap <---> dovecot <---> kmail
> 
> It really does seem like overkill. I really and truly wish I could just=20
> do: SERVER <---> kmail or even SERVER <---> offlineimap <---> kmail,=20
> but sadly neither of these seems to be a very good idea.
> 
> This is actually a little bit deceptive -- offlineimap and dovecot do=20
> not actually "connect" to each other, they simple access and modify the=20
> same directories and files. So it would really be more appropriate to=20
> draw something more like this:
> 
> SERVER <---> offlineimap <---> a bunch of maildir directories
> 
> a bunch of maildir directories <---> dovecot <---> kmail
> 
> 
> So I have a locally running dovecot serve me messages from the=20
> offlineimap maildir folders, since kmail's maildir handling is broken=20
> (in addition to its imap cacheing).
> 
> I hate to be sarcastic, but ... happy hacking? It's awfully=20
> resource-intensive, and there seem to be several bugs in kmail's imap=20
> handling that make kmail crash or lock up from time to time. (dovecot=20
> runs just fine, I never have to do anything with it.) Specifically,=20
> searching for messages doesn't work very well at all in kmail over=20
> imap. But, I like kmail's interface very much, so somehow all this=20
> difficulty is worth it so I have a comfortable working environment.
> 
> Ah, but I haven't answered your real question, how to get all this stuff=20
> talking together? Let's see what I can pull up. I'm using debian 4.0,=20
> with a hacked offlineimap 5.99.4 (since AFAIK John didn't accept my=20
> patch and I've been too lazy to update and fix 6.0 to work properly) as=20
> well as dovecot-common and dovecot-imapd 1.0.13 (from debian backports)=20
> and kmail 3.5.5.
> 
> my /etc/dovecot/dovecot.conf looks like this:
> 
> protocols =3D imap
> listen =3D 127.0.0.1:143
> log_path =3D /var/log/dovecot
> log_timestamp =3D "%Y-%m-%d %H:%M:%S "
> ssl_disable =3D yes
> login_process_per_connection =3D no
> mail_extra_groups =3D mail
> mail_process_size =3D 80
> maildir_copy_with_hardlinks =3D yes
> protocol imap {
> }
> auth default {
>   mechanisms =3D plain
>   passdb passwd-file {
>    args =3D /path/to/my/dovecot-passdb
>   }
>   userdb passwd-file {
>    args =3D /path/to/my/dovecot-passdb
>   }
>   passdb pam {
>   }
>   user =3D root
> }
> dict {
> }
> plugin {
> }
> 
> 
> Here is one (line-broken) line from my dovecot-passdb:
> 
> d3x:{plain}mysecretpasswordinplaintext:3078
> :3078::/home/david::userdb_mail=3Dmaildir:/path/to/offlineimap/d3x
> 
> 3078 is my user id as seen in /etc/passwd. If you ever change your user=20
> ID you'll have to go in and change this, otherwise dovecot won't=20
> work :)
> 
> I have several lines that are similar to this, with different password,=20
> account name, and account directories in the /path/to/offlineimap dir.
> 
> Note that the password here is the password kmail uses to connect to=20
> dovecot. It need not be the same password that offlineimap uses to=20
> connect to  for dovecot to connect to offlineimap. It need not be the=20
> same as your actual email address password. Even so, dovecot-passdb has=20
> perms go-r (and so does dovecot.conf, which is owned by root)
> 
> When I set up an account in kmail, I make it a regular imap account, and=20
> I have it connect to localhost as the imap server on port 143 -- to=20
> match the dovecot settings. Many people have been known to have=20
> offlineimap+dovecot on a "server" machine and connect to dovecot=20
> via "client" machines, which may or may not suit you.
> 
> 
> Here are some snippets from my ~/.offlineimaprc :
> 
> [general]
> accounts =3D aowm, d3x, [...]
> # if you have 9 accounts then this needs to be 9 if you ever want to run
> # continuously.
> maxsyncaccounts =3D 9
> # folder name translations...
> pythonfile =3D /path/to/offlineimap-config/offlineimap.py
> # metadata folder...
> metadata=3D/path/to/offlineimap-Metadata
> socktimeout =3D 45
> 
> # account name corresponds to that in the [general] accounts list.
> # It happens to be the same account name I use with dovecot, but dovecot
> # and offlineimap don't really have any awareness of each other, so this
> # is optional. The account name you connect to in kmail is the one you
> # specify with dovecot.
> 
> [Account d3x]
> 
> localrepository =3D local-d3x
> remoterepository =3D remote-d3x
> autorefresh =3D 3
> quick =3D 3
> 
> 
> [Repository local-d3x]
> type =3D Maildir
> localfolders =3D /path/to/offlineimap/d3x   # also in dovecot-passdb
> 
> [Repository remote-d3x]
> type =3D IMAP
> remotehost =3D mail.myserver.org
> remoteuser =3D d3x@xxxxxxxxxxxx
> remotepassfile =3D /path/to/offlineimap-config/pw-d3x
> ssl =3D no
> #axconnections: 1 is safe; 2 or 3 MAY be quicker; never more than 5.
> maxconnections =3D 2
> nametrans =3D fix_dovecot_kmail   # defined in my pythonfile
> foldersort=3Dmycmp   # also defined in my pythonfile
> 
> note that the remotepassfile =3D /path/to/offlineimap-config/pw-d3x holds=20
> a plaintext version of the password used to access the email server --=20
> this is the one that's a "really big" secret. Of course the file is=20
> go-r.
> 
> My pythonfile (/path/to/offlineimap-config/offlineimap.py) is adapted=20
> from Asheesh's.
> 
> I have dovecot set up to run automatically with the rc*.d scripts. If I=20
> need to restart it for some reason (e.g. to reread the passdb file and=20
> start serving a new mail account) "/etc/init.d/dovecot restart" does=20
> the trick.
> 
> I run offlineimap manually from the command line, and let it sit there=20
> with its pretty Curses.Blinkenlights :)  I also set statuschar to '*'=20
> because I find it to be more visible than '.'
> 
> offlineimap crashes A LOT when connecting to one of my email servers --=20
> usually it crashes once for each message it downloads -- so I have a=20
> bash wrapper script that automatically restarts it for me in a=20
> semi-intelligent manner. Hopefully offlineimap won't crash at all for=20
> you (I think it works great for most people without crashing, I have=20
> just been too lazy to give John the reports and info he would need to=20
> work out this problem, and I am hoping to switch servers sometime soon=20
> anyway -- that's what I've been saying for nearly a year now,=20
> methinks.)
> 
> Good luck!  Now I really can say "happy hacking" without feeling too=20
> guilty about not helping much :)
> 
> ~David.
> 
> 
> On Thursday 03 July 2008 5:05 pm, joshua_rickmar@xxxxxxxxxxx wrote:
>> Hello.
>> =20
>> A while ago, I tried to use KMail with my email provider, Lavabit.
>> However, because of KMail's strange IMAP implementation, there are
>> numerous bugs, like messages disappearing when moving emails and the=20
> like=3D
>> .
>>  Here (http://www.emaildiscussions.com/showthread.php?p=3D3D447209) is a
>> thread I started on Emaildiscussions.com trying to solve this problem
>> using OfflineIMAP and a local IMAP server.  I wasn't very successful,=20
> but
>> since I have re-installed FreeBSD on my computer, I decided to give it=20
> a
>> second shot.
>> =20
>> On the OfflineIMAP FAQ, it says:
>> =20
>>  > However, I have made KMail version 3 work well with OfflineIMAP by
>>  > installing an IMAP server on my local machine, having OfflineIMAP=20
> sync
>>  > to that, and pointing KMail at the same server.
>> =20
>> I'm just wondering how this is set up?  Specifically, what IMAP client=20
> is
>> this referring to?  And if dovecot will work, and I was just doing
>> something wrong in that thread, what has to be fixed?
>> =20
>> =20
>> =20
>> =20
>> =20
> 
>