Re: [aclug-L] ntmag

[John Goerzen <jgoerzen@xxxxxxxxxxxx>]

Here is a point-by-point response to the article.

His first large mistake is putting far too much emphasis on
benchmarks.  One industry leader, Steve Gibson, once wrote that
"benchmarks are worth nothing more than the disks they're stored on."
While I probably wouldn't go that far, if you are using something for
extremely high-end computing, you definately would be much better off
to test the proposed solution yourself with your own situation than to 
use the artificial situation created by benchmarks.

He then launches into a discussion of "kernel-mode threads" and
describes an "obvious solution" wherein a client pre-initializes a
number of threads.  This is precisely what Apache does.  He then talks 
about "drawbacks", the first being that "the server must know at the
time it initializes what kind of. . . load".  This is incorrect.
Apache will fire up new "threads" (processes in our jargon) when
necessary, and destroy extra ones after load drops down significantly.
He then discusses resources, saying that they "might not be adequate
to create all the threads".  This is a weak argument in the first
place: if you're talking about a quad-processor system, you darn well
better have enough resources anyway.  Furthermore, fork() on Linux is
copy-on-write, meaning that most of the "threads" will *share* memory
space, causing a very efficient use of memory.  He mentions things
about managing the threads using processor time, and switching using
"significant" overhead.  While true to a point, it is not *nearly* as
bad as he suggests; Apache, while serving a document, mmap()'s the
file, and sits in an extremely simple loop: just writing data out to
the network until it's all written or there's a network error.  He
must be delusional if he thinks this uses significant CPU time :-)

Then, he talks about asynchronous I/O.  He COMPLETELY misses the mark
here.  First, with the thread model he was discussing above, this is
irrelevant; by the time a thread gets the request, the connection is
already open.  Furthermore, all that has to be done is mmap() the
file.  I have no idea how he manages to say that synchronous I/O
causes these designs to perform poorly.  In fact, the opposite is
true: the thread would have to busy-wait otherwise, which would use
huge amounts of CPU time.

He then states that Linux 2.2 cannot efficiently wait for multiple
events and that it doesn't support asynchronous I/O.  Both are
*blatantly* false.

From the manpages:

NAME
       select,  FD_CLR,  FD_ISSET,  FD_SET, FD_ZERO - synchronous
       I/O multiplexing

That's the first.

Here's the second, asynchronous mode:

       O_NONBLOCK or O_NDELAY
               The  file  is opened in non-blocking mode. Neither
               the open nor any subsequent operations on the file
               descriptor  which is returned will cause the call­
               ing process to wait.

That's from the open(2) manpage.

What's interesting is that he then does mention select(2), but
incorrectly states its history.  He then completely misunderstands how 
select() works.

He then complains that select "doesn't scale."  This is a valid point, 
but silly.  select is not *supposed* to scale.  If you need to watch
over 1024 file descriptors, you've probably got problems with your
code anyway.  Furthermore, Linux 2.2.x-ac has no 1024-fd limit.

Then, he describes a "new mechanism" but at the same time demonstrates 
a lack of understanding of how TCP/IP programming is done in Unix.  Of 
*course* this mechanism would not cover "new client connection"s -- to 
do so would go against the Unix socket mechanism.  This isn't really a 
problem, either.  Then he again doesn't understand how threads work.

Again, he says that "the OS currently has no asynchronous APIs."
Again, untrue.  He tries to draw a distinction between "nonblocking"
[sic] and asynchronous I/O.  He conveniently "forgets" about select(2) 
here, because if select() were used, you get EXACTLY the behavior that 
he is describing.

He then talks about a "thread-throttling" idea, but talks about it
only in the most abstract of terms, so I'm not sure exactly what he's
talking about.  Apparently what he is saying is that if there are lots 
of threads available to process something, that only one thread should 
do so.  Well, the answer there is simple: create only one thread.

He then completely misses the mark and says that "write paths still
serialize".  This cannot possibly be true; if it were, then, for
instance, while my system is writing to a floppy disk, it wouldn't be
able to read from the network.  The last time I saw that behavior was, 
hmm, in Windows.

He then talks about a sendfile() API.  He states: "Without sendfile, a 
Web server. . . must first read the contents of the file into its
memory".  This is FALSE.  Apache and similar servers use mmap(), which 
does not require them to read it into memory first.  Had he looked up
the manpage for mmap, he would have seen this.  This makes the rest of 
his rant on this moot.

-- John

phrostie <phrostie@xxxxxxxxxxxxx> writes:

> [1  <text/plain; us-ascii (7bit)>]
> a man i used to work with in austin is considering learning linux.
> his day job is as an nt administrator.
> today he sent me this article asking me what i thought .
> in truth most of what the author is talking about is over my head.
> my understanding is that linux has always had support for SMP,
> with 2.2 it is more efficient.
> i would like to give him some constructive feed back, but
> once again this is over my head.
> not too much nt bashing please :-).
> thankyou
> 
> http://www.winntmag.com/Magazine/Article.cfm?ArticleID=5048
> 
> --
> phrostie@xxxxxxxxxxxxx
> Oh I've slipped the surly bonds of dos
> and danced the skies on LINUX silvered wings.
> http://www.cottagesoft.com/~phrostie/cad-tastrafy
> 
> 
> [2  <text/html; us-ascii (7bit)>]
> 

-- 
John Goerzen   Linux, Unix consulting & programming   jgoerzen@xxxxxxxxxxxx |
Developer, Debian GNU/Linux (Free powerful OS upgrade)       www.debian.org |
----------------------------------------------------------------------------+
The 180,104,072nd prime number is 3,782,300,581.