lkml.org 
[lkml]   [2005]   [Nov]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: Realtime-preempt performs worse for many threads?
    On Tue, 1 Nov 2005, Carlos Antunes wrote:

    > Hi!
    >
    > I've been developing some code for the OpenPBX project
    > (http://www.openpbx.org) and wrote a program to test how the system,
    > responds when hundreds of threads are spawned. These threads run at
    > high priority (SCHED_FIFO) and use clock_nanocleep with absolute
    > timeouts on a 20ms loop cycle.
    >
    > With the stock 2.6.14 kernel, I get latencies in the order of several
    > milliseconds (but less than 20ms) when running 1250 threads
    > simultaneously. However, when I switch to a kernel patched with
    > realtime-preempt latency increases to several hundred milliseconds in
    > many cases.

    There is only one explanation:
    Some of the operations (task switch, nanosleep etc.) are more expensive in
    the RT kernel. Thus your 1250 threads spend 100% CPU doing what they do.
    You therefore get very bad latencies.

    >
    > When I only only spawn 10 or so threads, realtime-preempt gives me
    > latencies of less than 1ms while the stock kernel still gives me a few
    > milliseconds. However, when the number of threads sleeping on
    > clock_nanosleep increases to several hundred, things just break.
    >
    > Should I assume that realtime-preempt at this time is not ready to
    > deal with hundreds of realtime threads sleeping most of the time on
    > clock_nanosleep?
    >

    Well, apparently not. But whoever wants to do that for a _real_
    application?
    In practise, I would say that you can not really gurantie latencies for
    more than the 10-20 highest priority threads. The probabilty that
    those 10-20 threads have runs for an non-determiniticly long time
    becomes very high, especially if they are event-triggered: The events
    could come in bursts. Therefore you can't consider the next threads "RT".
    This is what happens in your case: When you increase the number of threads
    the CPU gets more and more to do on the highest priority. At some point
    you hit 100% workload for 100s of ms in row. The next threads thus can't
    meet their deadlines. As you put all your threads on the same priority,
    nobody is really among highest priority threads and nobody optains
    deterministic latencies.

    > Any ideas on how to maybe debug this and see if there is some kind of problem?
    >

    No bug, only extra overhead. The RT kernel has a lower _throughput_.
    Apparently 1250 threads doing nanosleep for 2ms is it's limit on your
    machine.

    Esben


    > Thanks!
    >
    > Carlos
    >
    >
    >
    > --
    > "We hold [...] that all men are created equal; that they are
    > endowed [...] with certain inalienable rights; that among
    > these are life, liberty, and the pursuit of happiness"
    > -- Thomas Jefferson
    > -
    > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    > the body of a message to majordomo@vger.kernel.org
    > More majordomo info at http://vger.kernel.org/majordomo-info.html
    > Please read the FAQ at http://www.tux.org/lkml/
    >

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-11-02 00:30    [W:0.022 / U:35.500 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site