lkml.org 
[lkml]   [2008]   [Nov]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: [ltt-dev] [PATCH] Poll : introduce poll_wait_exclusive() new function
Date

Mathieu Desnoyers <compudj@krystal.dyndns.org> writes:

[...]

>> > Mathieu Desnoyers explained it cause following problem to LTTng.
>> >
>> > In LTTng, all lttd readers are polling all the available debugfs files
>> > for data. This is principally because the number of reader threads is
>> > user-defined and there are typical workloads where a single CPU is
>> > producing most of the tracing data and all other CPUs are idle,
>> > available to consume data. It therefore makes sense not to tie those
>> > threads to specific buffers. However, when the number of threads grows,
>> > we face a "thundering herd" problem where many threads can be woken up
>> > and put back to sleep, leaving only a single thread doing useful work.
>>
>> Why do you need to have so many threads banging a single device/file?
>> Have one (or any other very little number) puller thread(s), that
>> activates with chucks of pulled data the other processing threads. That
>> way there's no need for a new wakeup abstraction.
>>
>>
>>
>> - Davide
>
> One of the key design rule of LTTng is to do not depend on such
> system-wide data structures, or entity (e.g. single manager thread).
> Everything is per-cpu, and it does scale very well.
>
> I wonder how badly the approach you propose can scale on large NUMA
> systems, where having to synchronize everything through a single thread
> might become an important point of contention, just due to the cacheline
> bouncing and extra scheduler activity involved.

But at the end of the day these threads end up writing to the (possibly)
single spindle. Isn't that the biggest bottlneck here?

--
andy


\
 
 \ /
  Last update: 2008-11-26 12:25    [W:0.364 / U:1.188 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site