Messages in this thread Patch in this message | | | Subject | Re: [patch 0/7] sched: change nohz idle load balancing logic to push model | From | Peter Zijlstra <> | Date | Mon, 31 May 2010 11:17:12 +0200 |
| |
On Fri, 2010-05-21 at 17:09 -0700, Suresh Siddha wrote:
> From: Venkatesh Pallipadi <venki@google.com> > Subject: sched: Change nohz ilb logic from pull to push model > > In the new push model, all idle CPUs indeed go into nohz mode. There is > still the concept of idle load balancer (performing the load balancing > on behalf of all the idle cpu's in the system). Busy CPU kicks the nohz > balancer when any of the nohz CPUs need idle load balancing. > The kickee CPU does the idle load balancing on behalf of all idle CPUs > instead of the normal idle balance. > > This addresses the below two problems with the current nohz ilb logic: > * the idle load balancer continued to have periodic ticks during idle and > wokeup frequently, even though it did not have any rebalancing to do on > behalf of any of the idle CPUs. > * On x86 and CPUs that have APIC timer stoppage on idle CPUs, this > periodic wakeup can result in a periodic additional interrupt on a CPU > doing the timer broadcast. > > Also currently we are migrating the unpinned timers from an idle to the cpu > doing idle load balancing (when all the cpus in the system are idle, > there is no idle load balancing cpu and timers get added to the same idle cpu > where the request was made. So the existing optimization works only on semi idle > system). > > And In semi idle system, we no longer have periodic ticks on the idle load > balancer CPU. Using that cpu will add more delays to the timers than intended > (as that cpu's timer base may not be uptodate wrt jiffies etc). This was > causing mysterious slowdowns during boot etc. > > For now, in the semi idle case, use the nearest busy cpu for migrating timers > from an idle cpu. This is good for power-savings anyway. > > Signed-off-by: Venkatesh Pallipadi <venki@google.com> > Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com> > ---
I sorted the conflict with Venki's update_cpu_load() patch as below.
Thanks!
--- Index: linux-2.6/kernel/sched_fair.c =================================================================== --- linux-2.6.orig/kernel/sched_fair.c +++ linux-2.6/kernel/sched_fair.c @@ -3446,6 +3446,9 @@ static void nohz_idle_balance(int this_c break; } + raw_spin_lock_irq(&rq->lock); + update_cpu_load(rq); + raw_spin_unlock_irq(&rq->lock); rebalance_domains(balance_cpu, CPU_IDLE); rq = cpu_rq(balance_cpu); @@ -3518,40 +3521,12 @@ static void run_rebalance_domains(struct rebalance_domains(this_cpu, idle); -#ifdef CONFIG_NO_HZ /* - * If this cpu is the owner for idle load balancing, then do the + * If this cpu has a pending nohz_balance_kick, then do the * balancing on behalf of the other idle cpus whose ticks are * stopped. */ - if (this_rq->idle_at_tick && - atomic_read(&nohz.load_balancer) == this_cpu) { - struct rq *rq; - int balance_cpu; - - for_each_cpu(balance_cpu, nohz.cpu_mask) { - if (balance_cpu == this_cpu) - continue; - - /* - * If this cpu gets work to do, stop the load balancing - * work being done for other cpus. Next load - * balancing owner will pick it up. - */ - if (need_resched()) - break; - - rq = cpu_rq(balance_cpu); - raw_spin_lock_irq(&rq->lock); - update_cpu_load(rq); - raw_spin_unlock_irq(&rq->lock); - rebalance_domains(balance_cpu, CPU_IDLE); - - if (time_after(this_rq->next_balance, rq->next_balance)) - this_rq->next_balance = rq->next_balance; - } - } -#endif + nohz_idle_balance(this_cpu, idle); } static inline int on_null_domain(int cpu)
| |