lkml.org 
[lkml]   [2012]   [Feb]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: sched: Avoid SMT siblings in select_idle_sibling() if possible
From
Date
On Mon, 2012-02-20 at 20:33 +0530, Srivatsa Vaddagiri wrote: 
> * Peter Zijlstra <peterz@infradead.org> [2012-02-20 15:41:01]:
>
> > On Fri, 2011-11-18 at 16:14 +0100, Mike Galbraith wrote:
> >
> > > ---
> > > kernel/sched_fair.c | 10 ++--------
> > > 1 file changed, 2 insertions(+), 8 deletions(-)
> > >
> > > Index: linux-3.0-tip/kernel/sched_fair.c
> > > ===================================================================
> > > --- linux-3.0-tip.orig/kernel/sched_fair.c
> > > +++ linux-3.0-tip/kernel/sched_fair.c
> > > @@ -2276,17 +2276,11 @@ static int select_idle_sibling(struct ta
> > > for_each_cpu_and(i, sched_domain_span(sd), tsk_cpus_allowed(p)) {
> > > if (idle_cpu(i)) {
> > > target = i;
> > > + if (sd->flags & SD_SHARE_CPUPOWER)
> > > + continue;
> > > break;
> > > }
> > > }
> > > -
> > > - /*
> > > - * Lets stop looking for an idle sibling when we reached
> > > - * the domain that spans the current cpu and prev_cpu.
> > > - */
> > > - if (cpumask_test_cpu(cpu, sched_domain_span(sd)) &&
> > > - cpumask_test_cpu(prev_cpu, sched_domain_span(sd)))
> > > - break;
> > > }
> > > rcu_read_unlock();
> >
> > Mike, Suresh, did we ever get this sorted? I was looking at
> > select_idle_sibling() and it looks like we dropped this.
> >
> > Also, did anybody ever get an answer from a HW guy on why sharing stuff
> > over SMT threads is so much worse than sharing it over proper cores? Its
> > not like this workload actually does anything concurrently.
> >
> > I was looking at this code due to vatsa wanting to do SD_BALANCE_WAKE.
>
> From a quick scan of that code, it seems to prefer selecting an idle cpu
> in the same cache domain (vs selecting prev_cpu in absence of a core
> that is fully idle).

Yes, that was the sole purpose of select_idle_sibling() from square one.
If you can mobilize a CPU without eating cache penalty, this is most
excellent for load ramp-up. The gain is huge over affine wakeup if
there is any overlap to regain, ie it's not a 100% synchronous load.

> I can give that a try for my benchmark and see how much it helps. My
> suspicion is it will not fully solve the problem I have on hand.

I doubt it will either. Your problem is when it doesn't succeed, but
you have an idle core available in another domain. That's a whole
different ball game. Yeah, you can reap benefit by doing wakeup
balancing, but you'd better look very closely at the cost. I haven't
been able to do that lately, so dunno what cost is in the here and now,
but it used to be _way_ too expensive to consider, just as unrestricted
idle balancing is, or high frequency load balancing in general is.

-Mike



\
 
 \ /
  Last update: 2012-02-20 19:29    [W:0.538 / U:0.188 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site