lkml.org 
[lkml]   [2015]   [Dec]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    Subject[PATCH 2/4] sched: Fix a race in try_to_wake_up() vs schedule()
    Oleg noticed that its possible to falsely observe p->on_cpu == 0 such
    that we'll prematurely continue with the wakeup and effectively run p on
    two CPUs at the same time.

    Even though the overlap is very limited; the task is in the middle of
    being scheduled out; it could still result in corruption of the
    scheduler data structures.


    CPU0 CPU1

    set_current_state(...)

    <preempt_schedule>
    context_switch(X, Y)
    prepare_lock_switch(Y)
    Y->on_cpu = 1;
    finish_lock_switch(X)
    store_release(X->on_cpu, 0);

    try_to_wake_up(X)
    LOCK(p->pi_lock);

    t = X->on_cpu; // 0

    context_switch(Y, X)
    prepare_lock_switch(X)
    X->on_cpu = 1;
    finish_lock_switch(Y)
    store_release(Y->on_cpu, 0);
    </preempt_schedule>

    schedule();
    deactivate_task(X);
    X->on_rq = 0;

    if (X->on_rq) // false

    if (t) while (X->on_cpu)
    cpu_relax();

    context_switch(X, ..)
    finish_lock_switch(X)
    store_release(X->on_cpu, 0);


    Avoid the load of X->on_cpu being hoisted over the X->on_rq load.

    Reported-by: Oleg Nesterov <oleg@redhat.com>
    Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
    ---
    kernel/sched/core.c | 19 +++++++++++++++++++
    1 file changed, 19 insertions(+)

    --- a/kernel/sched/core.c
    +++ b/kernel/sched/core.c
    @@ -2084,6 +2084,25 @@ try_to_wake_up(struct task_struct *p, un

    #ifdef CONFIG_SMP
    /*
    + * Ensure we load p->on_cpu _after_ p->on_rq, otherwise it would be
    + * possible to, falsely, observe p->on_cpu == 0.
    + *
    + * One must be running (->on_cpu == 1) in order to remove oneself
    + * from the runqueue.
    + *
    + * [S] ->on_cpu = 1; [L] ->on_rq
    + * UNLOCK rq->lock
    + * RMB
    + * LOCK rq->lock
    + * [S] ->on_rq = 0; [L] ->on_cpu
    + *
    + * Pairs with the full barrier implied in the UNLOCK+LOCK on rq->lock
    + * from the consecutive calls to schedule(); the first switching to our
    + * task, the second putting it to sleep.
    + */
    + smp_rmb();
    +
    + /*
    * If the owning (remote) cpu is still in the middle of schedule() with
    * this task as prev, wait until its done referencing the task.
    */



    \
     
     \ /
      Last update: 2015-12-03 14:01    [W:4.366 / U:0.824 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site