lkml.org 
[lkml]   [2007]   [May]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    SubjectRe: [patch] CFS scheduler, -v8
    Ingo Molnar wrote:
    > * Balbir Singh <balbir@linux.vnet.ibm.com> wrote:
    >
    >> With -v7 I would run the n/n+1 test. Basically on a system with n
    >> cpus, I would run n+1 tasks and see how their load is distributed. I
    >> usually find that the last two tasks would get stuck on one CPU on the
    >> system and would get half the cpu time as their other peers. I think
    >> this issue has been around for long even before CFS. But while I was
    >> investigating that, I found that with -v8, all the n+1 tasks are stuck
    >> on the same cpu.
    >
    > i believe this problem is specific to powerpc - load is distributed fine
    > on i686/x86_64 and your sched_debug shows a cpu_load[0] == 0 on CPU#2
    > which is 'impossible'. (I sent a few suggestions off-Cc about how to
    > debug this.)
    >
    > Ingo

    Hi, Ingo

    The suggestions helped, here is a fix tested on PowerPC only.

    Patch and Description
    =====================


    Load balancing on PowerPC is broken. Running 5 tasks on a 4 cpu system
    results in all 5 tasks running on the same CPU. Based on Ingo's feedback,
    I instrumented and debugged update_load_fair().

    The problem is with comparing a s64 values with (s64)ULONG_MAX, which
    evaluates to -1. Then we check if exec_delta64 and fair_delta64 are greater
    than (s64)ULONG_MAX (-1), if so we assign (s64)ULONG_MAX to the respective
    values.

    The fix is to compare these values against (s64)LONG_MAX and assign
    (s64)LONG_MAX to exec_delta64 and fair_delta64 if they are greater than
    (s64)LONG_MAX.

    Tested on PowerPC, the regression is gone, tasks are load balanced as they
    were in v7.

    Output of top

    5614 root 20 0 4912 784 252 R 52 0.0 3:27.49 3 bash

    5620 root 20 0 4912 784 252 R 47 0.0 3:07.38 2 bash

    5617 root 20 0 4912 784 252 R 47 0.0 3:08.18 0 bash

    5624 root 20 0 4912 784 252 R 26 0.0 1:42.97 1 bash

    5621 root 20 0 4912 784 252 R 26 0.0 1:43.14 1 bash


    Tasks 5624 and 5621 getting half of their peer values is a separate issue
    altogether.

    Signed-off-by: Balbir Singh <balbir@linux.vnet.ibm.com>
    ---

    kernel/sched.c | 10 +++++-----
    1 file changed, 5 insertions(+), 5 deletions(-)

    diff -puN kernel/sched.c~cfs-fix-load-balancing-arith kernel/sched.c
    --- linux-2.6.21/kernel/sched.c~cfs-fix-load-balancing-arith 2007-05-02
    16:16:20.000000000 +0530
    +++ linux-2.6.21-balbir/kernel/sched.c 2007-05-02 16:16:47.000000000 +0530
    @@ -1533,19 +1533,19 @@ static void update_load_fair(struct rq *
    this_rq->prev_exec_clock = this_rq->exec_clock;
    WARN_ON_ONCE(exec_delta64 <= 0);

    - if (fair_delta64 > (s64)ULONG_MAX)
    - fair_delta64 = (s64)ULONG_MAX;
    + if (fair_delta64 > (s64)LONG_MAX)
    + fair_delta64 = (s64)LONG_MAX;
    fair_delta = (unsigned long)fair_delta64;

    - if (exec_delta64 > (s64)ULONG_MAX)
    - exec_delta64 = (s64)ULONG_MAX;
    + if (exec_delta64 > (s64)LONG_MAX)
    + exec_delta64 = (s64)LONG_MAX;
    exec_delta = (unsigned long)exec_delta64;
    if (exec_delta > TICK_NSEC)
    exec_delta = TICK_NSEC;

    idle_delta = TICK_NSEC - exec_delta;

    - tmp = SCHED_LOAD_SCALE * exec_delta / fair_delta;
    + tmp = (SCHED_LOAD_SCALE * exec_delta) / fair_delta;
    tmp64 = (u64)tmp * (u64)exec_delta;
    do_div(tmp64, TICK_NSEC);
    this_load = (unsigned long)tmp64;
    _

    --
    Warm Regards,
    Balbir Singh
    Linux Technology Center
    IBM, ISTL
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2007-05-02 13:03    [W:0.042 / U:0.484 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site