lkml.org 
[lkml]   [2012]   [Apr]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    SubjectRe: load balancing regression since commit 367456c7
    From
    Date
    On Fri, 2012-04-20 at 19:33 +0200, Peter Zijlstra wrote:
    >
    > OK, I'll go stare at the cgroup part then.. Thanks!
    >
    Ok, I could reproduce when using cgroups, the below fixes it for me, can
    you confirm?

    ---
    Subject: sched: Fix more load-balance fallout
    From: Peter Zijlstra <a.p.zijlstra@chello.nl>
    Date: Tue Apr 17 13:38:40 CEST 2012

    Commits 367456c756a6 ("sched: Ditch per cgroup task lists for
    load-balancing") and 5d6523ebd ("sched: Fix load-balance wreckage")
    left some more wreckage.

    By setting loop_max unconditionally to ->nr_running load-balancing
    could take a lot of time on very long runqueues (hackbench!). So keep
    the sysctl as max limit of the amount of tasks we'll iterate.

    Furthermore, the min load filter for migration completely fails with
    cgroups since inequality in per-cpu state can easily lead to such
    small loads :/

    Furthermore the change to add new tasks to the tail of the queue
    instead of the head seems to have some effect.. not quite sure I
    understand why.

    Combined these fixes solve the huge hackbench regression reported by
    Tim when hackbench is ran in a cgroup.

    Reported-by: Tim Chen <tim.c.chen@linux.intel.com>
    Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
    ---
    kernel/sched/fair.c | 19 ++++++++++++++-----
    1 file changed, 14 insertions(+), 5 deletions(-)

    --- a/kernel/sched/fair.c
    +++ b/kernel/sched/fair.c
    @@ -784,7 +784,7 @@ account_entity_enqueue(struct cfs_rq *cf
    update_load_add(&rq_of(cfs_rq)->load, se->load.weight);
    #ifdef CONFIG_SMP
    if (entity_is_task(se))
    - list_add_tail(&se->group_node, &rq_of(cfs_rq)->cfs_tasks);
    + list_add(&se->group_node, &rq_of(cfs_rq)->cfs_tasks);
    #endif
    cfs_rq->nr_running++;
    }
    @@ -3215,6 +3215,14 @@ static int move_one_task(struct lb_env *

    static unsigned long task_h_load(struct task_struct *p);

    +static const unsigned int sched_nr_migrate_break =
    +#ifdef CONFIG_PREEMPT
    + 8
    +#else
    + 32
    +#endif
    + ;
    +
    /*
    * move_tasks tries to move up to load_move weighted load from busiest to
    * this_rq, as part of a balancing operation within domain "sd".
    @@ -3242,7 +3250,7 @@ static int move_tasks(struct lb_env *env

    /* take a breather every nr_migrate tasks */
    if (env->loop > env->loop_break) {
    - env->loop_break += sysctl_sched_nr_migrate;
    + env->loop_break += sched_nr_migrate_break;
    env->flags |= LBF_NEED_BREAK;
    break;
    }
    @@ -3252,7 +3260,7 @@ static int move_tasks(struct lb_env *env

    load = task_h_load(p);

    - if (load < 16 && !env->sd->nr_balance_failed)
    + if (sched_feat(LB_MIN) && load < 16 && !env->sd->nr_balance_failed)
    goto next;

    if ((load / 2) > env->load_move)
    @@ -4407,7 +4415,7 @@ static int load_balance(int this_cpu, st
    .dst_cpu = this_cpu,
    .dst_rq = this_rq,
    .idle = idle,
    - .loop_break = sysctl_sched_nr_migrate,
    + .loop_break = sched_nr_migrate_break,
    };

    cpumask_copy(cpus, cpu_active_mask);
    @@ -4448,7 +4456,8 @@ static int load_balance(int this_cpu, st
    env.load_move = imbalance;
    env.src_cpu = busiest->cpu;
    env.src_rq = busiest;
    - env.loop_max = busiest->nr_running;
    + env.loop_max = min_t(unsigned long,
    + sysctl_sched_nr_migrate, busiest->nr_running);

    more_balance:
    local_irq_save(flags);


    \
     
     \ /
      Last update: 2012-04-25 17:03    [W:0.036 / U:0.872 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site