lkml.org 
[lkml]   [2012]   [Jul]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRe: [RFC] optimize the locking in the rebalance_domains()
    Date
    Hi, Vlad

    On Sun, 22 Jul 2012 19:23:55 +0300, Vlad Zolotarov wrote:
    > Ingo, we've noticed that rebalance_domains() will try to take a lock
    > every time it's called (every jiffy) if SD_SERIALIZE is set (which is a
    > default configuration). This is done regardless the fact that maybe
    > there hasn't passed enough time since the last rebalancing in which case
    > there is no need to take a lock the first place.
    >
    > The above creates a heavy false sharing problem on the "balancing"
    > spin-lock on large SMP systems: try_lock() is implemented with an
    > (atomic) xchng instruction which invalidates the cache line "balancing"
    > belongs to and therefore creates an intensive cross-NUMA-nodes traffic.
    >
    > The below patch will minimize the above phenomena to the time slots it's
    > really needed, namely when the "interval" has really passed.
    >
    > Pls., comment.
    >
    > thanks,
    > vlad
    >
    > ---
    > kernel/sched/fair.c | 20 +++++++++++---------
    > 1 file changed, 11 insertions(+), 9 deletions(-)
    >
    > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
    > index c099cc6..6777d38 100644
    > --- a/kernel/sched/fair.c
    > +++ b/kernel/sched/fair.c
    > @@ -4689,6 +4689,9 @@ static void rebalance_domains(int cpu, enum cpu_idle_type idle)
    > interval = msecs_to_jiffies(interval);
    > interval = clamp(interval, 1UL, max_load_balance_interval);
    >
    > + if (!time_after_eq(jiffies, sd->last_balance + interval))
    > + goto out;
    > +

    First line looks like white-space-damaged.
    Anyway, wouldn't it be better using time_before() here?

    Thanks,
    Namhyung


    > need_serialize = sd->flags & SD_SERIALIZE;
    >
    > if (need_serialize) {
    > @@ -4696,16 +4699,15 @@ static void rebalance_domains(int cpu, enum cpu_idle_type idle)
    > goto out;
    > }
    >
    > - if (time_after_eq(jiffies, sd->last_balance + interval)) {
    > - if (load_balance(cpu, rq, sd, idle, &balance)) {
    > - /*
    > - * We've pulled tasks over so either we're no
    > - * longer idle.
    > - */
    > - idle = CPU_NOT_IDLE;
    > - }
    > - sd->last_balance = jiffies;
    > + if (load_balance(cpu, rq, sd, idle, &balance)) {
    > + /*
    > + * We've pulled tasks over so either we're no
    > + * longer idle.
    > + */
    > + idle = CPU_NOT_IDLE;
    > }
    > + sd->last_balance = jiffies;
    > +
    > if (need_serialize)
    > spin_unlock(&balancing);
    > out:


    \
     
     \ /
      Last update: 2012-07-23 05:01    [W:0.026 / U:29.524 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site