[aclug-L] Re: MTAs & maildir vs mbox (was Re: Virtual POP3 and IMAP server)

["Ryan Hunt" <rhunt@xxxxxxxxxxx>]

They each have their pros and cons, of course.

Adding a new message to either is pretty cheap, but in the case of
mbox, locking is required to ensure that multiple simultaneous updates
don't conflict.

Deleting a message from an mbox requires reading the entire file and
writing it all back out, except for the deleted message. Deleting a
message from a maildir only requires unlinking the file containing the
message. Again, updating the mbox requires locking.

Indexing an mbox mailbox requires reading the entire mailbox and
parsing the messages--parsing is undesirable. Indexing a maildir
mailbox requires opening each file and reading the header--that means
lots of directory accesses and open() system calls.

Large mbox mailboxes are huge, unwieldy files. Large maildir mailboxes
are huge, unwieldy directories--most filesystems don't handle them
efficiently after they grow to a few thousand files.

This of course is a huge debate that has been going on since the begining of
Maildirs..

No one has came up with conclusive proof that one is mostdefinatly the
better way to go, its all personal opinion..

My personal opinion is Maildirs are better than mbox's

-Ryan


----- Original Message -----
From: "Jonathan Hall" <jonhall@xxxxxxxxxxxx>
To: <discussion@xxxxxxxxx>
Sent: Friday, October 19, 2001 1:28 PM
Subject: [aclug-L] Re: MTAs & maildir vs mbox (was Re: Virtual POP3 and IMAP
server)


>
> I know sometimes what seems like common sense doesn't necissarily pan out
in
> the real world...
>
> But isn't it faster to read (and then delete) a single large file rather
> than reading then deleting a number of small files?
>
> I put together a quick 'n dirty bash script to test this theory... altho I
> realize this isn't the same as a real-world MTA/POP3 style test...  here
are
> my results:
>
> (Test platform, if it matters to anyone, is Debian/Linux on a P133 w/ 64mb
> RAM and 6gb IDE HD)
> Real User Sys
> Creating 1000 10kb files (with dd): 14.725 7.330 7.200
> Deleting 1000 10kb files: 10.733 5.230 5.410
> Creating 1 10,000kb file (with dd): 0.893 0.080 0.740
> Deleting 1 10,000kb file: 0.102 0.050 0.050
> Creating 10,000kb file, 10kb at a time: 12.567 6.600 5.830
> Deleting 10,000kb file, 10kb at a time: After 14m51s I aborted this test
>
> The last test (deleting a partial file) is obviously where mbox suffers
the
> greatest performance hit.
>
> But in a situation where most users are using POP3, and download their
> entire mailbox in one sitting.
>
> A quick scan of my syslog over the last week shows that out of 26,562 POP3
> connections, 802 connections resulted in downloaded mail (the others were
> checks when no new mail was available).  Of those 802 downloads, only 20
> them involved a partial download--the rest of the time either no mail or
all
> mail was downloaded.  This means that in less than 2.5% of the instances
are
> mailboxes rewritten (or partially deleted).  And of course, those 20 cases
> were not nearly as bad as my test case above.  For one thing, the mail
boxes
> weren't 10,000kb in size, and second, it wasn't just a single message
> deleted at a time.
>
> Can anyone point me to actual real-world benchmarks between mbox and
> maildir?
>
> My quick test here seems to suggest (as my own commen sense would) that
for
> most sequential mail reads (as with POP3), mbox is faster...
>
> That is to say nothing of the reliability factor tho.
>
> -- Jonathan
>
>
> On Fri, Oct 19, 2001 at 11:58:00AM -0500, Steven Saner wrote:
> >
> > On Fri, Oct 19, 2001 at 10:31:38AM -0500, Jonathan Hall wrote:
> > >
> > > Another question I have is maildir vs. mbox?  I'd discussed this with
some
> > > others, and came to the conclusion that mbox would be more efficient
for me
> > > since 99% of my users will dowload their mail w/ POP3, then delete it.
mbox
> > > uses far fewer inodes... and sequential messsage access should be much
> > > faster for mbox over maildir... right?
> >
> >
> > I will weigh in on this issue, since I haven't seen anyone else do
> > it. The word on the street is that Maildir rocks, period, end of
> > statement. There have been benchmarking tests done, and really there
> > isn't much difference in terms of speed unless you are dealing with
> > huge amounts of email. Yes Maildir does require more inodes, but in
> > today's cheap hardware world, that really isn't an issue (again,
> > unless you are are doing huge things). The major advantage of Maildir
> > is reliability. One message can not clobber the entire mail box. Also,
> > if a customer does have a bad message, you can delete the file, rather
> > than try to edit the mbox. If you want to think about moving your mail
> > spool to an NFS mounted box at some point, then this is about the only
> > way to do it to avoid bad file locking issues.
> >
> > A cool trick that I learned about that you can do with Maildir is a
> > flexible pop bulletin like thing (I know that was one of your
> > requirements). You can create a file that has the message, and then
> > write a simple script to add a symbolic link to that file in each
> > customer's Maildir that you want to get it. When they check the mail,
> > they download it and then the POP server deletes the symbolic link, as
> > if it was a normal message.
> >
> > I have had my time as a pro mbox person, but I see advantages in
> > changing that.
> >
> > In my working with these things, I have looked at the following:
> >
> > sendmail for MTA
> > procmail as a delivery agent that will deliver to Maildir
> > Courier IMAP for a Maildir IMAP server
> > Solid POP for a simple Maildir POP server
> >
> > However, if you really want to do virtual email, I'm afraid that
> > looking at the QMail solution, may be the best (only?) way of doing it.
> >
> > Steve
> > -- This is the discussion@xxxxxxxxx list.  To unsubscribe,
> > visit http://tmp2.complete.org/cgi-bin/listargate-aclug.cgi
>
> --
> --
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>   Jonathan Hall  *  jonhall@xxxxxxxxxxxx  *  PGP public key available
>  Systems Admin, Future Internet Services; Goessel, KS * (620) 367-2487
>          http://www.futureks.net/  *  PGP Key ID: FE 00 FD 51
>                   -=  Running Debian GNU/Linux  =-
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> -- This is the discussion@xxxxxxxxx list.  To unsubscribe,
> visit http://tmp2.complete.org/cgi-bin/listargate-aclug.cgi
>

-- This is the discussion@xxxxxxxxx list.  To unsubscribe,
visit http://tmp2.complete.org/cgi-bin/listargate-aclug.cgi