On Sat, 24 Apr 1999, Andi Kleen wrote:
> > > You have multiple threads doing an accept on a single listen socket. As
> > > soon as a thread finished work it calls accept and gets the next ready 
> > > connection handed from the kernel.
> > 
> >  ...or will be awakened on the connection that was handled by another
> > thread (because of "wake everyone" handling), and accept() will fail,
> > causing the infamous "thundering herd".
> 
> If the load is high enough it doesn't matter, because there will
> be always enough connections to be returned to an accept after a wakeup.

right, but before you get to this point, there is a performance drop.

> If it isn't the threads pool should adapt and use less threads which
> avoids the problem (and a few lost wakeups in the transitions don't harm, 
> because the machine has enough free cycles).

this is sounding more complicated by the minute.  you also want to tune
this so that you have just the right number of threads active to keep the
L1/L2 caches working at their most efficient.  is there any guarantee that
waiting in accept() won't cause round-robin behavior rather than just
picking the first couple of threads on the list?

in other words, it's best to have the number of worker threads be close to
the number of physical CPUs; otherwise, if the threads are scheduled in
round-robin fashion, they could constantly knock each others' working set
out of the CPU caches.

if waiting in accept() does cause the thundering herd problem, that might
be a good thing - the thread that wins will probably have the best cache
foot-print.

> Do you have any real data that this doesn't happen?

i'm wondering if anyone has studied the performance difference between
using this kind of model, and using Windows NT completion ports?  i've
heard lots of speculation that using a completion port model is
significantly more efficient than accept().

	- Chuck Lever
--
corporate:	<chuckl@netscape.com>
personal:	<chucklever@netscape.net> or <cel@monkey.org>

The Linux Scalability project:
	http://www.citi.umich.edu/projects/linux-scalability/


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/