[aclug-L] Re: MTAs & maildir vs mbox (was Re: Virtual POP3 and IMAP server)

[Jonathan Hall <jonhall@xxxxxxxxxxxx>]

I know sometimes what seems like common sense doesn't necissarily pan out in
the real world...

But isn't it faster to read (and then delete) a single large file rather
than reading then deleting a number of small files?

I put together a quick 'n dirty bash script to test this theory... altho I
realize this isn't the same as a real-world MTA/POP3 style test...  here are
my results:

(Test platform, if it matters to anyone, is Debian/Linux on a P133 w/ 64mb
RAM and 6gb IDE HD)
                                        Real    User    Sys
Creating 1000 10kb files (with dd):     14.725  7.330   7.200
Deleting 1000 10kb files:               10.733  5.230   5.410
Creating 1 10,000kb file (with dd):     0.893   0.080   0.740
Deleting 1 10,000kb file:               0.102   0.050   0.050
Creating 10,000kb file, 10kb at a time: 12.567  6.600   5.830
Deleting 10,000kb file, 10kb at a time: After 14m51s I aborted this test

The last test (deleting a partial file) is obviously where mbox suffers the
greatest performance hit.

But in a situation where most users are using POP3, and download their
entire mailbox in one sitting.

A quick scan of my syslog over the last week shows that out of 26,562 POP3
connections, 802 connections resulted in downloaded mail (the others were
checks when no new mail was available).  Of those 802 downloads, only 20
them involved a partial download--the rest of the time either no mail or all
mail was downloaded.  This means that in less than 2.5% of the instances are
mailboxes rewritten (or partially deleted).  And of course, those 20 cases
were not nearly as bad as my test case above.  For one thing, the mail boxes
weren't 10,000kb in size, and second, it wasn't just a single message
deleted at a time.

Can anyone point me to actual real-world benchmarks between mbox and
maildir?

My quick test here seems to suggest (as my own commen sense would) that for
most sequential mail reads (as with POP3), mbox is faster...

That is to say nothing of the reliability factor tho.

-- Jonathan


On Fri, Oct 19, 2001 at 11:58:00AM -0500, Steven Saner wrote:
> 
> On Fri, Oct 19, 2001 at 10:31:38AM -0500, Jonathan Hall wrote:
> > 
> > Another question I have is maildir vs. mbox?  I'd discussed this with some
> > others, and came to the conclusion that mbox would be more efficient for me
> > since 99% of my users will dowload their mail w/ POP3, then delete it.  mbox
> > uses far fewer inodes... and sequential messsage access should be much
> > faster for mbox over maildir... right?
> 
> 
> I will weigh in on this issue, since I haven't seen anyone else do
> it. The word on the street is that Maildir rocks, period, end of
> statement. There have been benchmarking tests done, and really there
> isn't much difference in terms of speed unless you are dealing with
> huge amounts of email. Yes Maildir does require more inodes, but in
> today's cheap hardware world, that really isn't an issue (again,
> unless you are are doing huge things). The major advantage of Maildir
> is reliability. One message can not clobber the entire mail box. Also,
> if a customer does have a bad message, you can delete the file, rather
> than try to edit the mbox. If you want to think about moving your mail
> spool to an NFS mounted box at some point, then this is about the only
> way to do it to avoid bad file locking issues.
> 
> A cool trick that I learned about that you can do with Maildir is a
> flexible pop bulletin like thing (I know that was one of your
> requirements). You can create a file that has the message, and then
> write a simple script to add a symbolic link to that file in each
> customer's Maildir that you want to get it. When they check the mail,
> they download it and then the POP server deletes the symbolic link, as
> if it was a normal message.
> 
> I have had my time as a pro mbox person, but I see advantages in
> changing that.
> 
> In my working with these things, I have looked at the following:
> 
> sendmail for MTA
> procmail as a delivery agent that will deliver to Maildir
> Courier IMAP for a Maildir IMAP server
> Solid POP for a simple Maildir POP server
> 
> However, if you really want to do virtual email, I'm afraid that
> looking at the QMail solution, may be the best (only?) way of doing it.
> 
> Steve
> -- This is the discussion@xxxxxxxxx list.  To unsubscribe,
> visit http://tmp2.complete.org/cgi-bin/listargate-aclug.cgi

--
--
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  Jonathan Hall  *  jonhall@xxxxxxxxxxxx  *  PGP public key available
 Systems Admin, Future Internet Services; Goessel, KS * (620) 367-2487
         http://www.futureks.net/  *  PGP Key ID: FE 00 FD 51
                  -=  Running Debian GNU/Linux  =-
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-- This is the discussion@xxxxxxxxx list.  To unsubscribe,
visit http://tmp2.complete.org/cgi-bin/listargate-aclug.cgi