Messages in this thread Patch in this message | | | Date | Tue, 8 Jan 2008 23:08:15 +0530 | From | Vaidyanathan Srinivasan <> | Subject | Analysis of sched_mc_power_savings |
| |
Hi,
The following experiments were conducted on a two socket dual core intel processor based machine in order to understand the impact of sched_mc_power_savings scheduler heuristics.
Kernel linux-2.6.24-rc6:
CONFIG_SCHED_SMT=y CONFIG_SCHED_MC=y CONFIG_TICK_ONESHOT=y CONFIG_NO_HZ=y CONFIG_HIGH_RES_TIMERS=y CONFIG_GENERIC_CLOCKEVENTS_BUILD=y CONFIG_HPET_TIMER=y CONFIG_HPET_EMULATE_RTC=y
tick-sched.c has been instrumented to collect idle entry and exit time stamps.
Instrumentation patch:
Instrument tick-sched nohz code and generate time stamp trace data.
Signed-off-by: Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com> --- kernel/time/tick-sched.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
--- linux-2.6.24-rc6.orig/kernel/time/tick-sched.c +++ linux-2.6.24-rc6/kernel/time/tick-sched.c @@ -20,6 +20,7 @@ #include <linux/profile.h> #include <linux/sched.h> #include <linux/tick.h> +#include <linux/ktrace.h> #include <asm/irq_regs.h> @@ -200,7 +201,10 @@ void tick_nohz_stop_sched_tick(void) if (ts->tick_stopped) { delta = ktime_sub(now, ts->idle_entrytime); ts->idle_sleeptime = ktime_add(ts->idle_sleeptime, delta); - } + ktrace_log2(KT_FUNC_tick_nohz_stop_sched_tick, KT_EVENT_INFO1, + ktime_to_ns(now), ts->idle_calls); >>>>>>>>>>>>>>>>Tracepoint A
+ } else + ktrace_log2(KT_FUNC_tick_nohz_stop_sched_tick, KT_EVENT_FUNC_ENTER, ktime_to_ns(now), 0); >>>>>>>>>>>>>>>>Tracepoint B
ts->idle_entrytime = now; ts->idle_calls++; @@ -391,6 +395,8 @@ void tick_nohz_restart_sched_tick(void) tick_do_update_jiffies64(now); now = ktime_get(); } + ktrace_log2(KT_FUNC_tick_nohz_restart_sched_tick, KT_EVENT_FUNC_EXIT, + ktime_to_ns(now), ts->idle_calls); >>>>>>>>>>>>>>>>Tracepoint C local_irq_enable(); }
The idle time collected are time stamp at (C) minus (B). This is the time interval between stopping ticks and restarting ticks in an idle system. Complete patch series: http://svaidy.googlepages.com/1-klog.patch http://svaidy.googlepages.com/1-trace-sched.patch
Userspace program to extract trace data: http://svaidy.googlepages.com/1-klog.c
Python script to post process binary trace data: http://svaidy.googlepages.com/1-sched-stats.py
Gnuplot scripts that was used to generate the graphs: http://svaidy.googlepages.com/1-multiplot.gp
The scheduler heuristics for multi core system /sys/devices/system/cpu/sched_mc_power_savings should ideally extend the cpu tickless idle time atleast on few CPU in an SMP machine.
However in the experiment it was found that turning on sched_mc_power_savings marginally increased the idle time in only some of the CPUs.
Experiment 1: ------------- Setup:
* yum-updated and irqbalance daemon was stopped to reduce the idle wakeup rate * All irqs manually routed to CPU0 only (hoping this will keep other CPUs idle) (http://svaidy.googlepages.com/1-irq-config.txt) * Powertop shows around 35 wakeups per second during idle (http://svaidy.googlepages.com/1-powertop-screen.txt) * The trace of idle time stamps was collected for 120 seconds with the system in idle state
Results:
There are 4 png files that plots the idle time for each CPU in the system.
Please get the graphs from the following URLs
http://svaidy.googlepages.com/1-idle-cpu0.png http://svaidy.googlepages.com/1-idle-cpu1.png http://svaidy.googlepages.com/1-idle-cpu2.png http://svaidy.googlepages.com/1-idle-cpu3.png
Each png file has 4 graphs plotted that is relevant to one CPU
* Right-top plot is the idle time sample obtained during the experiment * Left-top graph is histogram of right top plot * The bottom graphs corresponding to idle times when sched_mc_power_savings=1
Observations with sched_mc_power_savings=1:
* No major impact of sched_mc_power_savings on CPU0 and CPU1 * Significant idle time improvement on CPU2 * However, significant idle time reduction on CPU3
Experiment 2: ------------- Setup:
* USB stopped * Most daemons like yum-updatesd, hal, autofs, syslog, crond, irqbalance, sendmail, pcscd were stopped * Interrupt routing left to default but irqbalance daemon stopped * Powertop shows around 4 wakeups per second during idle (http://svaidy.googlepages.com/2-powertop-screen.txt) * The trace of idle time stamps was collected for 120 seconds with the system in idle state
Results:
There are 4 png files that plots the idle time for each CPU in the system.
http://svaidy.googlepages.com/2-idle-cpu0.png http://svaidy.googlepages.com/2-idle-cpu1.png http://svaidy.googlepages.com/2-idle-cpu2.png http://svaidy.googlepages.com/2-idle-cpu3.png
The details of the plot are same as the previous experiment.
Observations with sched_mc_power_savings=1:
* No major impact of sched_mc_power_savings on CPU0 and CPU1 * Good idle time improvement on CPU2 and CPU3
Please review the experiment and comment on how the effectiveness of sched_mc_power_savings can be analysed.
At very very low wakeup count of ~4 per second on 4 CPU system gave good idle time result when sched_mc_power_savings is enabled.
However the results are not as expected even at a marginal wakeup count of ~35 per second.
Please let us know your comments and suggestions on the experiment or results.
Do we have similar analysis and data on scheduler heuristics for power savings.
Thanks, Vaidy
| |