[aclug-L] Re: MTAs & maildir vs mbox (was Re: Virtual POP3 and IMAP server)

[Jonathan Hall <jonhall@xxxxxxxxxxxx>]

Adding now to my own commentary...

I don't know how all POP3 servers work... but the ones I'm familiar with
copy a user's mbox file from /var/spool/mail to /var/spool/pop (or similar)
during the download... then after the fetch, essentially "cat >>" to the
original mail spool again...

This will add significantly to the overhead of mbox--at least when messages
are left after a POP check (regardless of if or how many messages are
downloaded).

I guess maybe it comes down to this:

In perfectly ideal situation, mbox would be more efficient.  I.e. when all
messages are downloaded at once, every time.

In the real world, tho (particularly with dialup users who's POP3 sessiosn
may frequently be inerrupted midway), maildir may have significantly less
overhead.

I know I was kinda speaking against the crowed when I suggesed mbox would be
more efficient... but through my own reasoning, perhaps I've changed my
mind.  Is my thought process here correct?

-- Jonathan


On Fri, Oct 19, 2001 at 01:28:34PM -0500, Jonathan Hall wrote:
> 
> I know sometimes what seems like common sense doesn't necissarily pan out in
> the real world...
> 
> But isn't it faster to read (and then delete) a single large file rather
> than reading then deleting a number of small files?
> 
> I put together a quick 'n dirty bash script to test this theory... altho I
> realize this isn't the same as a real-world MTA/POP3 style test...  here are
> my results:
> 
> (Test platform, if it matters to anyone, is Debian/Linux on a P133 w/ 64mb
> RAM and 6gb IDE HD)
>                                       Real    User    Sys
> Creating 1000 10kb files (with dd):   14.725  7.330   7.200
> Deleting 1000 10kb files:             10.733  5.230   5.410
> Creating 1 10,000kb file (with dd):   0.893   0.080   0.740
> Deleting 1 10,000kb file:             0.102   0.050   0.050
> Creating 10,000kb file, 10kb at a time:       12.567  6.600   5.830
> Deleting 10,000kb file, 10kb at a time:       After 14m51s I aborted this test
> 
> The last test (deleting a partial file) is obviously where mbox suffers the
> greatest performance hit.
> 
> But in a situation where most users are using POP3, and download their
> entire mailbox in one sitting.
> 
> A quick scan of my syslog over the last week shows that out of 26,562 POP3
> connections, 802 connections resulted in downloaded mail (the others were
> checks when no new mail was available).  Of those 802 downloads, only 20
> them involved a partial download--the rest of the time either no mail or all
> mail was downloaded.  This means that in less than 2.5% of the instances are
> mailboxes rewritten (or partially deleted).  And of course, those 20 cases
> were not nearly as bad as my test case above.  For one thing, the mail boxes
> weren't 10,000kb in size, and second, it wasn't just a single message
> deleted at a time.
> 
> Can anyone point me to actual real-world benchmarks between mbox and
> maildir?
> 
> My quick test here seems to suggest (as my own commen sense would) that for
> most sequential mail reads (as with POP3), mbox is faster...
> 
> That is to say nothing of the reliability factor tho.
> 
> -- Jonathan
> 
> 
> On Fri, Oct 19, 2001 at 11:58:00AM -0500, Steven Saner wrote:
> > 
> > On Fri, Oct 19, 2001 at 10:31:38AM -0500, Jonathan Hall wrote:
> > > 
> > > Another question I have is maildir vs. mbox?  I'd discussed this with some
> > > others, and came to the conclusion that mbox would be more efficient for 
> > > me
> > > since 99% of my users will dowload their mail w/ POP3, then delete it.  
> > > mbox
> > > uses far fewer inodes... and sequential messsage access should be much
> > > faster for mbox over maildir... right?
> > 
> > 
> > I will weigh in on this issue, since I haven't seen anyone else do
> > it. The word on the street is that Maildir rocks, period, end of
> > statement. There have been benchmarking tests done, and really there
> > isn't much difference in terms of speed unless you are dealing with
> > huge amounts of email. Yes Maildir does require more inodes, but in
> > today's cheap hardware world, that really isn't an issue (again,
> > unless you are are doing huge things). The major advantage of Maildir
> > is reliability. One message can not clobber the entire mail box. Also,
> > if a customer does have a bad message, you can delete the file, rather
> > than try to edit the mbox. If you want to think about moving your mail
> > spool to an NFS mounted box at some point, then this is about the only
> > way to do it to avoid bad file locking issues.
> > 
> > A cool trick that I learned about that you can do with Maildir is a
> > flexible pop bulletin like thing (I know that was one of your
> > requirements). You can create a file that has the message, and then
> > write a simple script to add a symbolic link to that file in each
> > customer's Maildir that you want to get it. When they check the mail,
> > they download it and then the POP server deletes the symbolic link, as
> > if it was a normal message.
> > 
> > I have had my time as a pro mbox person, but I see advantages in
> > changing that.
> > 
> > In my working with these things, I have looked at the following:
> > 
> > sendmail for MTA
> > procmail as a delivery agent that will deliver to Maildir
> > Courier IMAP for a Maildir IMAP server
> > Solid POP for a simple Maildir POP server
> > 
> > However, if you really want to do virtual email, I'm afraid that
> > looking at the QMail solution, may be the best (only?) way of doing it.
> > 
> > Steve
> > -- This is the discussion@xxxxxxxxx list.  To unsubscribe,
> > visit http://tmp2.complete.org/cgi-bin/listargate-aclug.cgi
> 
> --
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>   Jonathan Hall  *  jonhall@xxxxxxxxxxxx  *  PGP public key available
>  Systems Admin, Future Internet Services; Goessel, KS * (620) 367-2487
>          http://www.futureks.net/  *  PGP Key ID: FE 00 FD 51
>                   -=  Running Debian GNU/Linux  =-
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> -- This is the discussion@xxxxxxxxx list.  To unsubscribe,
> visit http://tmp2.complete.org/cgi-bin/listargate-aclug.cgi

--
--
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  Jonathan Hall  *  jonhall@xxxxxxxxxxxx  *  PGP public key available
 Systems Admin, Future Internet Services; Goessel, KS * (620) 367-2487
         http://www.futureks.net/  *  PGP Key ID: FE 00 FD 51
                  -=  Running Debian GNU/Linux  =-
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-- This is the discussion@xxxxxxxxx list.  To unsubscribe,
visit http://tmp2.complete.org/cgi-bin/listargate-aclug.cgi