lkml.org 
[lkml]   [2009]   [Aug]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [tip:sched/core] sched: cpuacct: Use bigger percpu counter batch values for stats counters
    On Thu, 20 Aug 2009 15:10:38 +1000
    Anton Blanchard <anton@samba.org> wrote:

    >
    > Hi,
    >
    > Looks like this issue is still present. I tested on a 32 core box and
    > the patch improved maximum context switch rate from from 76k/sec to 9.5M/sec.
    > Thats over 100x faster, or 50x per line of code. That's got to be some sort of
    > record :)
    >
    > Any chance we can get a fix in for 2.6.31? Don't make me find an even bigger
    > box so I can break the 200x mark :)
    >
    I like the patch itself, I think it does a correct fix.

    Hmm, but, IIRC, there was a discssion "what happens at reading
    this counter once per 1ms ?" or some.

    What overhead at read on big system ? Can you show it ?

    Thanks,
    -Kame


    > Anton
    >
    > > --
    > >
    > > When CONFIG_VIRT_CPU_ACCOUNTING is enabled we can call cpuacct_update_stats
    > > with values much larger than percpu_counter_batch. This means the
    > > call to percpu_counter_add will always add to the global count which is
    > > protected by a spinlock.
    > >
    > > Since reading of the CPU accounting cgroup counters is not performance
    > > critical, we can use a maximum size batch of INT_MAX and use
    > > percpu_counter_sum on the read side which will add all the percpu
    > > counters.
    > >
    > > With this patch an 8 core POWER6 with CONFIG_VIRT_CPU_ACCOUNTING and
    > > CONFIG_CGROUP_CPUACCT shows an improvement in aggregate context switch rate of
    > > 397k/sec to 3.9M/sec, a 10x improvement.
    > >
    > > Signed-off-by: Anton Blanchard <anton@samba.org>
    > > ---
    > >
    > > Index: linux.trees.git/kernel/sched.c
    > > ===================================================================
    > > --- linux.trees.git.orig/kernel/sched.c 2009-07-16 10:11:02.000000000 +1000
    > > +++ linux.trees.git/kernel/sched.c 2009-07-16 10:16:41.000000000 +1000
    > > @@ -10551,7 +10551,7 @@
    > > int i;
    > >
    > > for (i = 0; i < CPUACCT_STAT_NSTATS; i++) {
    > > - s64 val = percpu_counter_read(&ca->cpustat[i]);
    > > + s64 val = percpu_counter_sum(&ca->cpustat[i]);
    > > val = cputime64_to_clock_t(val);
    > > cb->fill(cb, cpuacct_stat_desc[i], val);
    > > }
    > > @@ -10621,7 +10621,7 @@
    > > ca = task_ca(tsk);
    > >
    > > do {
    > > - percpu_counter_add(&ca->cpustat[idx], val);
    > > + __percpu_counter_add(&ca->cpustat[idx], val, INT_MAX);
    > > ca = ca->parent;
    > > } while (ca);
    > > rcu_read_unlock();
    >



    \
     
     \ /
      Last update: 2009-08-20 07:21    [W:2.981 / U:0.024 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site