lkml.org 
[lkml]   [2008]   [Nov]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: RT sched: cpupri_vec lock contention with def_root_domain and no load balance
Peter Zijlstra wrote:
> On Mon, 2008-11-03 at 15:07 -0600, Dimitri Sivanich wrote:
>
>> When load balancing gets switched off for a set of cpus via the
>> sched_load_balance flag in cpusets, those cpus wind up with the
>> globally defined def_root_domain attached. The def_root_domain is
>> attached when partition_sched_domains calls detach_destroy_domains().
>> A new root_domain is never allocated or attached as a sched domain
>> will never be attached by __build_sched_domains() for the non-load
>> balanced processors.
>>
>> The problem with this scenario is that on systems with a large number
>> of processors with load balancing switched off, we start to see the
>> cpupri->pri_to_cpu->lock in the def_root_domain becoming contended.
>> This starts to become much more apparent above 8 waking RT threads
>> (with each RT thread running on it's own cpu, blocking and waking up
>> continuously).
>>
>> I'm wondering if this is, in fact, the way things were meant to work,
>> or should we have a root domain allocated for each cpu that is not to
>> be part of a sched domain? Note the the def_root_domain spans all of
>> the non-load-balanced cpus in this case. Having it attached to cpus
>> that should not be load balancing doesn't quite make sense to me.
>>
>
> It shouldn't be like that, each load-balance domain (in your case a
> single cpu) should get its own root domain. Gregory?
>

Yeah, this sounds broken. I know that the root-domain code was being
developed coincident to some upheaval with the cpuset code, so I suspect
something may have been broken from the original intent. I will take a
look.

-Greg

>
>> Here's where we've often seen this lock contention occur:
>>
>
> what's this horrible output from?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>


[unhandled content-type:application/pgp-signature]
\
 
 \ /
  Last update: 2008-11-04 04:51    [W:0.074 / U:1.168 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site