Messages in this thread | | | Subject | Re: sched: Avoid SMT siblings in select_idle_sibling() if possible | From | Mike Galbraith <> | Date | Mon, 20 Feb 2012 19:25:43 +0100 |
| |
On Mon, 2012-02-20 at 20:33 +0530, Srivatsa Vaddagiri wrote: > * Peter Zijlstra <peterz@infradead.org> [2012-02-20 15:41:01]: > > > On Fri, 2011-11-18 at 16:14 +0100, Mike Galbraith wrote: > > > > > --- > > > kernel/sched_fair.c | 10 ++-------- > > > 1 file changed, 2 insertions(+), 8 deletions(-) > > > > > > Index: linux-3.0-tip/kernel/sched_fair.c > > > =================================================================== > > > --- linux-3.0-tip.orig/kernel/sched_fair.c > > > +++ linux-3.0-tip/kernel/sched_fair.c > > > @@ -2276,17 +2276,11 @@ static int select_idle_sibling(struct ta > > > for_each_cpu_and(i, sched_domain_span(sd), tsk_cpus_allowed(p)) { > > > if (idle_cpu(i)) { > > > target = i; > > > + if (sd->flags & SD_SHARE_CPUPOWER) > > > + continue; > > > break; > > > } > > > } > > > - > > > - /* > > > - * Lets stop looking for an idle sibling when we reached > > > - * the domain that spans the current cpu and prev_cpu. > > > - */ > > > - if (cpumask_test_cpu(cpu, sched_domain_span(sd)) && > > > - cpumask_test_cpu(prev_cpu, sched_domain_span(sd))) > > > - break; > > > } > > > rcu_read_unlock(); > > > > Mike, Suresh, did we ever get this sorted? I was looking at > > select_idle_sibling() and it looks like we dropped this. > > > > Also, did anybody ever get an answer from a HW guy on why sharing stuff > > over SMT threads is so much worse than sharing it over proper cores? Its > > not like this workload actually does anything concurrently. > > > > I was looking at this code due to vatsa wanting to do SD_BALANCE_WAKE. > > From a quick scan of that code, it seems to prefer selecting an idle cpu > in the same cache domain (vs selecting prev_cpu in absence of a core > that is fully idle).
Yes, that was the sole purpose of select_idle_sibling() from square one. If you can mobilize a CPU without eating cache penalty, this is most excellent for load ramp-up. The gain is huge over affine wakeup if there is any overlap to regain, ie it's not a 100% synchronous load.
> I can give that a try for my benchmark and see how much it helps. My > suspicion is it will not fully solve the problem I have on hand.
I doubt it will either. Your problem is when it doesn't succeed, but you have an idle core available in another domain. That's a whole different ball game. Yeah, you can reap benefit by doing wakeup balancing, but you'd better look very closely at the cost. I haven't been able to do that lately, so dunno what cost is in the here and now, but it used to be _way_ too expensive to consider, just as unrestricted idle balancing is, or high frequency load balancing in general is.
-Mike
| |