lkml.org 
[lkml]   [2010]   [May]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
SubjectRe: [PATCH] Make sure timers have migrated before killing migration_thread
On Wed, May 19, 2010 at 11:31:55AM +0200, Peter Zijlstra wrote:
> On Wed, 2010-05-19 at 14:35 +0530, Amit K. Arora wrote:
> > + cpuset_lock();
> > + rq = cpu_rq(cpu);
> > + kthread_stop(rq->migration_thread);
> > + put_task_struct(rq->migration_thread);
> > + rq->migration_thread = NULL;
> > + cpuset_unlock();
> > + break;
> > +
>
> The other problem is more urgent though, CPU_POST_DEAD runs outside of
> the hotplug lock and thus the above becomes a race where we could
> possible kill off the migration thread of a newly brought up cpu:
>
> cpu0 - down 2
> cpu1 - up 2 (allocs a new migration thread, and leaks the old one)
> cpu0 - post_down 2 - frees the migration thread -- oops!

<Adding Tejun Heo to CC list .. >

Hi Peter,

In an offline discussion with Tejun, he suggested that the above race
can not happen, since _cpu_up() and _cpu_down() can never run in
parallel, because of cpu_add_remove_lock. Looking at the code we can see
that cpu_up() and cpu_down() call "_" variants with cpu_add_remove_lock
mutex held (using cpu_maps_update_begin()).

Here is exactly what he had to say:

"I don't think that's possible. There are two locks involved here.
cpu_add_remove_lock and cpu_hotplug.lock. The former wraps around the
second and already provides full exclusion between all cpu hotplug/unplug
operations.
The latter is there for reader/writer type exclusion via
get/put_online_cpus().

CPU_POST_DEAD is outside of cpu_hotplug.lock allowing get_online_cpus()
to proceed in parallel but it's still inside cpu_add_remove_lock so other
cpu up/down operations cannot begin before it finishes. "


Thus, since above race can never happen, is there any other issue with
this patch ?

Thanks!
--
Regards,
Amit Arora


Signed-off-by: Amit Arora <aarora@in.ibm.com>
Signed-off-by: Gautham R Shenoy <ego@in.ibm.com>
---
diff -Nuarp linux-2.6.34.org/kernel/sched.c linux-2.6.34/kernel/sched.c
--- linux-2.6.34.org/kernel/sched.c 2010-05-18 22:56:21.000000000 -0700
+++ linux-2.6.34/kernel/sched.c 2010-05-18 22:58:31.000000000 -0700
@@ -5942,14 +5942,26 @@ migration_call(struct notifier_block *nf
cpu_rq(cpu)->migration_thread = NULL;
break;

+ case CPU_POST_DEAD:
+ /*
+ * Bring the migration thread down in CPU_POST_DEAD event,
+ * since the timers should have got migrated by now and thus
+ * we should not see a deadlock between trying to kill the
+ * migration thread and the sched_rt_period_timer.
+ */
+ cpuset_lock();
+ rq = cpu_rq(cpu);
+ kthread_stop(rq->migration_thread);
+ put_task_struct(rq->migration_thread);
+ rq->migration_thread = NULL;
+ cpuset_unlock();
+ break;
+
case CPU_DEAD:
case CPU_DEAD_FROZEN:
cpuset_lock(); /* around calls to cpuset_cpus_allowed_lock() */
migrate_live_tasks(cpu);
rq = cpu_rq(cpu);
- kthread_stop(rq->migration_thread);
- put_task_struct(rq->migration_thread);
- rq->migration_thread = NULL;
/* Idle task back to normal (off runqueue, low prio) */
raw_spin_lock_irq(&rq->lock);
update_rq_clock(rq);

\
 
 \ /
  Last update: 2010-05-24 12:03    [W:0.086 / U:1.932 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site