lkml.org 
[lkml]   [2011]   [Dec]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
Subject[patch v3 5/6] sched, ttwu_queue: queue remote wakeups only when crossing cache domains
From: Mike Galbraith <efault@gmx.de>

Context-switch intensive microbenchmark on a 8-socket system had
~600K times more resched IPI's on each logical CPU because of the
TTWU_QUEUE sched feature, which queues the task on the remote cpu's
queue and completes the wakeup locally using an IPI.

As the TTWU_QUEUE sched feature is for minimizing the cache-misses
associated with the remote wakeups, use the IPI only when the local and
the remote cpu's are from different cache domains. Otherwise use the
traditional remote wakeup.

With this, context-switch microbenchmark performed 5 times better on the
8-socket NHM-EX system.

Signed-off-by: Mike Galbraith <efault@gmx.de>
Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
---
kernel/sched/core.c | 25 ++++++++++++++++++++++++-
1 file changed, 24 insertions(+), 1 deletion(-)

Index: tip/kernel/sched/core.c
===================================================================
--- tip.orig/kernel/sched/core.c
+++ tip/kernel/sched/core.c
@@ -1481,12 +1481,35 @@ static int ttwu_activate_remote(struct t
#endif /* __ARCH_WANT_INTERRUPTS_ON_CTXSW */
#endif /* CONFIG_SMP */

+static int ttwu_share_cache(int this_cpu, int cpu)
+{
+#ifndef CONFIG_X86
+ struct sched_domain *sd;
+ int ret = 0;
+
+ rcu_read_lock();
+ for_each_domain(this_cpu, sd) {
+ if (!cpumask_test_cpu(cpu, sched_domain_span(sd)))
+ continue;
+
+ ret = (sd->flags & SD_SHARE_PKG_RESOURCES);
+ break;
+ }
+ rcu_read_unlock();
+
+ return ret;
+#else
+ return per_cpu(cpu_llc_id, this_cpu) == per_cpu(cpu_llc_id, cpu);
+#endif
+}
+
static void ttwu_queue(struct task_struct *p, int cpu)
{
struct rq *rq = cpu_rq(cpu);

#if defined(CONFIG_SMP)
- if (sched_feat(TTWU_QUEUE) && cpu != smp_processor_id()) {
+ if (sched_feat(TTWU_QUEUE) &&
+ !ttwu_share_cache(smp_processor_id(), cpu)) {
sched_clock_cpu(cpu); /* sync clocks x-cpu */
ttwu_queue_remote(p, cpu);
return;



\
 
 \ /
  Last update: 2011-12-02 02:15    [W:0.145 / U:0.056 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site