[lkml]   [1999]   [Apr]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: Linus on Linux, Apache and Threads
    On Sun, Apr 25, 1999 at 12:43:52AM +0200, Chuck Lever wrote:
    > On Sat, 24 Apr 1999, Andi Kleen wrote:
    > > > > You have multiple threads doing an accept on a single listen socket. As
    > > > > soon as a thread finished work it calls accept and gets the next ready
    > > > > connection handed from the kernel.
    > > >
    > > > ...or will be awakened on the connection that was handled by another
    > > > thread (because of "wake everyone" handling), and accept() will fail,
    > > > causing the infamous "thundering herd".
    > >
    > > If the load is high enough it doesn't matter, because there will
    > > be always enough connections to be returned to an accept after a wakeup.
    > right, but before you get to this point, there is a performance drop.

    Then you have enough cycles left, so it doesn't matter. The server always
    has to be a bit oversized to handle traffic peaks, in non-peak situation
    you can afford to be a bit less efficient (to complicate code to optimize
    this would be wasted time).

    Also if the threads pool size is adapting quickly enough it shouldn't
    be that bad.

    > > If it isn't the threads pool should adapt and use less threads which
    > > avoids the problem (and a few lost wakeups in the transitions don't harm,
    > > because the machine has enough free cycles).
    > this is sounding more complicated by the minute. you also want to tune
    > this so that you have just the right number of threads active to keep the
    > L1/L2 caches working at their most efficient. is there any guarantee that
    > waiting in accept() won't cause round-robin behavior rather than just
    > picking the first couple of threads on the list?

    There is no such guarantee (except perhaps if you play with nice values[1]),
    but it does not matter when you keep statistics about number of requests/time.
    As soon as the average time a thread has to wait in the accept goes below
    some time add more threads. If it goes gets above the time kill threads and
    lower the time. Costs you a few gettimeofdays() if you don't use a time
    keeper thread (a thread that updates a timestamp counter in shared memory)

    > in other words, it's best to have the number of worker threads be close to
    > the number of physical CPUs; otherwise, if the threads are scheduled in
    > round-robin fashion, they could constantly knock each others' working set
    > out of the CPU caches.
    > if waiting in accept() does cause the thundering herd problem, that might
    > be a good thing - the thread that wins will probably have the best cache
    > foot-print.

    Only real tests can show. Anyways, if the kernel accept() behaviour
    should really cause problems (I would guess it doesn't by intuition, but I
    have no data), then accept() should be fixed, not complicated code added
    to th e user application.


    [1] I wouldn't suggest that because of the possible nasty interactions
    with other server processes running on the same machine.

    This is like TV. I don't like TV.

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 13:51    [W:0.021 / U:3.432 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site