[lkml]   [2012]   [Sep]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    Patch in this message
    Subject[PATCH 1/2] rcu: Exit RCU extended QS on user preemption
    When exceptions or irq are about to resume userspace, if
    the task needs to be rescheduled, the arch low level code
    calls schedule() directly.

    If we call it, it is because we have the TIF_RESCHED flag:

    - It can be set after random local calls to set_need_resched()
    (RCU, drm, ...)

    - A wake up happened and the CPU needs preemption. This can
    happen in several ways:

    * Remotely: the remote waking CPU has set TIF_RESCHED and send the
    wakee an IPI to schedule the new task.
    * Remotely enqueued: the remote waking CPU sends an IPI to the target
    and the wake up is made by the target.
    * Locally: waking CPU == wakee CPU and the wakeup is done locally.
    set_need_resched() is called without IPI.

    In the case of local and remotely enqueued wake ups, the tick can
    be restarted when we enqueue the new task and RCU can exit the
    extended quiescent state at the same time. Then by the time we reach
    irq exit path and we call schedule, we are not in RCU user mode.

    But if we call schedule() only because something called set_need_resched(),
    RCU may still be in user mode when we reach schedule.

    Also if a wake up is done remotely, the CPU might see the TIF_RESCHED
    flag and call schedule while the IPI has not yet happen to restart the
    tick and exit RCU user mode.

    We need to manually protect against these corner cases.

    Create a new API schedule_user() that calls schedule() inside
    rcu_user_exit()-rcu_user_enter() in order to protect it. Archs
    will need to rely on it now to implement user preemption safely.

    Signed-off-by: Frederic Weisbecker <>
    Cc: Alessio Igor Bogani <>
    Cc: Andrew Morton <>
    Cc: Avi Kivity <>
    Cc: Chris Metcalf <>
    Cc: Christoph Lameter <>
    Cc: Geoff Levand <>
    Cc: Gilad Ben Yossef <>
    Cc: Hakan Akkan <>
    Cc: H. Peter Anvin <>
    Cc: Ingo Molnar <>
    Cc: Josh Triplett <>
    Cc: Kevin Hilman <>
    Cc: Max Krasnyansky <>
    Cc: Peter Zijlstra <>
    Cc: Stephen Hemminger <>
    Cc: Steven Rostedt <>
    Cc: Sven-Thorsten Dietrich <>
    Cc: Thomas Gleixner <>
    Signed-off-by: Paul E. McKenney <>
    Reviewed-by: Josh Triplett <>
    kernel/sched/core.c | 15 +++++++++++++++
    1 files changed, 15 insertions(+), 0 deletions(-)

    diff --git a/kernel/sched/core.c b/kernel/sched/core.c
    index 0bd599b..7298dbf 100644
    --- a/kernel/sched/core.c
    +++ b/kernel/sched/core.c
    @@ -3463,6 +3463,21 @@ asmlinkage void __sched schedule(void)

    +#ifdef CONFIG_RCU_USER_QS
    +asmlinkage void __sched schedule_user(void)
    + /*
    + * If we come here after a random call to set_need_resched(),
    + * or we have been woken up remotely but the IPI has not yet arrived,
    + * we haven't yet exited the RCU idle mode. Do it here manually until
    + * we find a better solution.
    + */
    + rcu_user_exit();
    + schedule();
    + rcu_user_enter();
    * schedule_preempt_disabled - called with preemption disabled

     \ /
      Last update: 2012-09-14 19:21    [W:0.026 / U:17.232 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site