lkml.org 
[lkml]   [1999]   [Apr]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Linus on Linux, Apache and Threads
On Sun, Apr 25, 1999 at 12:43:52AM +0200, Chuck Lever wrote:
> On Sat, 24 Apr 1999, Andi Kleen wrote:
> > > > You have multiple threads doing an accept on a single listen socket. As
> > > > soon as a thread finished work it calls accept and gets the next ready
> > > > connection handed from the kernel.
> > >
> > > ...or will be awakened on the connection that was handled by another
> > > thread (because of "wake everyone" handling), and accept() will fail,
> > > causing the infamous "thundering herd".
> >
> > If the load is high enough it doesn't matter, because there will
> > be always enough connections to be returned to an accept after a wakeup.
>
> right, but before you get to this point, there is a performance drop.

Then you have enough cycles left, so it doesn't matter. The server always
has to be a bit oversized to handle traffic peaks, in non-peak situation
you can afford to be a bit less efficient (to complicate code to optimize
this would be wasted time).

Also if the threads pool size is adapting quickly enough it shouldn't
be that bad.

>
> > If it isn't the threads pool should adapt and use less threads which
> > avoids the problem (and a few lost wakeups in the transitions don't harm,
> > because the machine has enough free cycles).
>
> this is sounding more complicated by the minute. you also want to tune
> this so that you have just the right number of threads active to keep the
> L1/L2 caches working at their most efficient. is there any guarantee that
> waiting in accept() won't cause round-robin behavior rather than just
> picking the first couple of threads on the list?

There is no such guarantee (except perhaps if you play with nice values[1]),
but it does not matter when you keep statistics about number of requests/time.
As soon as the average time a thread has to wait in the accept goes below
some time add more threads. If it goes gets above the time kill threads and
lower the time. Costs you a few gettimeofdays() if you don't use a time
keeper thread (a thread that updates a timestamp counter in shared memory)


>
> in other words, it's best to have the number of worker threads be close to
> the number of physical CPUs; otherwise, if the threads are scheduled in
> round-robin fashion, they could constantly knock each others' working set
> out of the CPU caches.
>
> if waiting in accept() does cause the thundering herd problem, that might
> be a good thing - the thread that wins will probably have the best cache
> foot-print.

Only real tests can show. Anyways, if the kernel accept() behaviour
should really cause problems (I would guess it doesn't by intuition, but I
have no data), then accept() should be fixed, not complicated code added
to th e user application.


-Andi

[1] I wouldn't suggest that because of the possible nasty interactions
with other server processes running on the same machine.

--
This is like TV. I don't like TV.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:51    [from the cache]
©2003-2011 Jasper Spaans