lkml.org 
[lkml]   [2009]   [Mar]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [v2 PATCH 0/4] timers: framework for migration between CPU

* Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com> wrote:

> * Ingo Molnar <mingo@elte.hu> [2009-03-04 18:33:21]:
>
> >
> > * Arun R Bharadwaj <arun@linux.vnet.ibm.com> wrote:
> >
> > > $taskset -c 4,5,6,7 make -j4
> > >
> > > my_driver queuing timers continuously on CPU 10.
> > >
> > > idle load balancer currently on CPU 15
> > >
> > >
> > > Case1: Without timer migration Case2: With timer migration
> > >
> > > -------------------- --------------------
> > > | Core | LOC Count | | Core | LOC Count |
> > > | 4 | 2504 | | 4 | 2503 |
> > > | 5 | 2502 | | 5 | 2503 |
> > > | 6 | 2502 | | 6 | 2502 |
> > > | 7 | 2498 | | 7 | 2500 |
> > > | 10 | 2501 | | 10 | 35 |
> > > | 15 | 2501 | | 15 | 2501 |
> > > -------------------- --------------------
> > >
> > > --------------------- --------------------
> > > | Core | Sleep time | | Core | Sleep time |
> > > | 4 | 0.47168 | | 4 | 0.49601 |
> > > | 5 | 0.44301 | | 5 | 0.37153 |
> > > | 6 | 0.38979 | | 6 | 0.51286 |
> > > | 7 | 0.42829 | | 7 | 0.49635 |
> > > | 10 | 9.86652 | | 10 | 10.04216 |
> > > | 15 | 0.43048 | | 15 | 0.49056 |
> > > --------------------- ---------------------
> > >
> > > Here, all the timers queued by the driver on CPU10 are moved to CPU15,
> > > which is the idle load balancer.
> >
> > The numbers with this automatic method based on the ilb-cpu look
> > pretty convincing. Is this what you expected it to be?
>
> Yes Ingo, this is the expected results and looks pretty good. However
> there are two parameters controlled in this experiment:
>
> 1) The system is moderately loaded with kernbench so that there are
> some busy CPUs and some idle cpus, and the no_hz mask is does not
> change often. This leads to stable ilb-cpu selection. If the
> system is either completely idle or loaded too little leading to
> ilb nominations, then timers keep following the ilb cpu and it is
> very difficult to experimentally observe the benefits.
>
> Even if the ilb bounces, consolidating timers should increase
> overlap between timers and reduce the wakeup from idle.
>
> Optimising the ilb selection should significantly improve
> experimental results for this patch.
>
> 2) The timer test driver creates quite large timer load so that the
> effect of migration is observable as sleep time difference on the
> expected target cpu. This kind of timer load may not be uncommon
> with lots of application stack loaded in an enterprise system

the important thing to watch out for is to not have _worse_
performance due to ilb jumping too much. So as long as you can
prove that numbers dont get worse you are golden.

Power-saving via migration will only work if there's a
concentrated workload to begin with.

So the best results will be in combination with scheduler
power-saving patches. (which too make the ilb jump less in
essence)

So by getting scheduler power saving enhancements your method
will work better too - there's good synergy and no dependency on
any user-space component.

Btw., could you please turn the runtime switch into a /proc/sys
sysctl, and only when CONFIG_SCHED_DEBUG=y. Otherwise it should
be default-enabled with no ability to turn it off.

Ingo


\
 
 \ /
  Last update: 2009-03-04 19:33    [from the cache]
©2003-2014 Jasper Spaans. hosted at Digital Ocean