lkml.org 
[lkml]   [2009]   [Jun]   [27]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    Subject[Patch] stop_machine stalls for a considerable period on large cpu count machines.
    I forgot to include the linux-kernel mailing list.

    Robin

    ----- Forwarded message from Robin Holt <holt@sgi.com> -----

    Date: Sat, 27 Jun 2009 05:53:11 -0500
    From: Robin Holt <holt@sgi.com>
    To: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Rusty Russell <rusty@rustcorp.com.au>, stable@kernel.org,
    mike travis <travis@sgi.com>
    Subject: [Patch] stop_machine stalls for a considerable period on large cpu
    count machines.

    Mike Travis noted that a 2048 cpu machine booting would take hours
    to get through its modprobes. We would get numerous back traces from
    stop_cpu indicating they had not serviced interrupts.

    A quick code review indicated we have a situation of heavy cacheline contention due to the 'state' (read-mostly) and 'thread_ack'
    (write-mostly) variables being located in the same cacheline.


    Signed-off-by: Robin Holt <holt@sgi.com>

    ---
    kernel/stop_machine.c | 4 ++--
    1 file changed, 2 insertions(+), 2 deletions(-)
    Index: stop_machine_false_sharing/kernel/stop_machine.c
    ===================================================================
    --- stop_machine_false_sharing.orig/kernel/stop_machine.c 2009-06-27 05:26:57.000000000 -0500
    +++ stop_machine_false_sharing/kernel/stop_machine.c 2009-06-27 05:34:45.182637682 -0500
    @@ -36,8 +36,8 @@ struct stop_machine_data {

    /* Like num_online_cpus(), but hotplug cpu uses us, so we need this. */
    static unsigned int num_threads;
    -static atomic_t thread_ack;
    -static DEFINE_MUTEX(lock);
    +static atomic_t thread_ack ____cacheline_aligned;
    +static DEFINE_MUTEX(lock) ____cacheline_aligned;
    /* setup_lock protects refcount, stop_machine_wq and stop_machine_work. */
    static DEFINE_MUTEX(setup_lock);
    /* Users of stop_machine. */
    ----- End forwarded message -----

    \
     
     \ /
      Last update: 2009-06-27 12:59    [W:0.022 / U:31.920 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site