lkml.org 
[lkml]   [2006]   [Aug]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRe: Is stopmachine() preempt safe?
    Date
    Rusty Russell (on Mon, 28 Aug 2006 16:17:11 +1000) wrote:
    >On Mon, 2006-08-28 at 12:55 +1000, Keith Owens wrote:
    >> Rusty Russell (on Mon, 28 Aug 2006 09:38:55 +1000) wrote:
    >> >On Sun, 2006-08-27 at 19:42 +1000, Keith Owens wrote:
    >> >> I cannot convince myself that stopmachine() is preempt safe. What
    >> >> prevents this race with CONFIG_PREEMPT=y?
    >> >
    >> >Nothing. Read side is preempt_disable. Write side is stopmachine.
    >>
    >> That is very worrying. The whole point of stopmachine is to get all
    >> cpus to a known state with no locally cached global data, so the caller
    >> of stopmachine can safely fiddle with some global data (like updating
    >> the module lists). But CONFIG_PREEMPT defeats this and turns any code
    >> that relies on stopmachine into a race.
    >
    >Hi Keith,
    >
    > Do not panic: this is not a bug, and was never how stop_machine was to
    >be used. try_module_get() disables preemption, and you are supposed to
    >disable preemption

    Disabling preemption only guarantees that the current task will not be
    preempted. It says nothing about the state of tasks that were already
    running on the cpus when stopmachine was called. It is these tasks
    that were already running and were preempted by stopmachine that have
    to be flushed before stopmachine can continue. Remember that this is
    the race:

    cpu 0 cpu 1
    stop_machine()
    Process <n> reads a global resource
    do_stop()
    kernel_thread(stopmachine, 1)
    Process <n> is preempted
    stopmachine() runs on cpu 1
    STOPMACHINE_PREPARE
    STOPMACHINE_DISABLE_IRQ
    do_stop() calls smdata->fn
    smdata->fn changes global data
    restart_machine()
    STOPMACHINE_EXIT
    stopmachine() exits
    Scheduler resumes process <n>
    The global resource is out of sync

    >(or take cpu_hotplug_lock) when relying on
    >cpu_online_map.

    There is a lot of code in the kernel that runs cpu_online_map without
    taking any locks and without disabling preemption. Obviously we do not
    want all that code to lock or disable preemption, it will kill
    scalability. The lack of locking on the read side means that changes
    to cpu_online_map have to be done under stopmachine, which brings in
    the race again.

    I will have a look at your scheduler patch and see if it fixes the
    race.

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2006-08-28 08:39    [W:7.285 / U:0.096 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site