lkml.org 
[lkml]   [2012]   [Aug]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks on v3.6
    On 08/07/2012 07:40 AM, John Stultz wrote:
    > On 08/06/2012 11:28 AM, Sasha Levin wrote:
    >> On 08/06/2012 08:20 PM, John Stultz wrote:
    >>> On 08/06/2012 10:21 AM, John Stultz wrote:
    >>>> On 08/05/2012 09:55 AM, Sasha Levin wrote:
    >>>>> On 07/30/2012 03:17 PM, Avi Kivity wrote:
    >>>>>> Possible causes:
    >>>>>> - the APIC calibration in the guest failed, so it is programming too
    >>>>>> low values into the timer
    >>>>>> - it actually needs 1 us wakeups and then can't keep up (esp. as kvm
    >>>>>> interrupt injection is slowing it down)
    >>>>>>
    >>>>>> You can try to find out by changing
    >>>>>> arch/x86/kvm/lapic.c:start_lapic_timer() to impose a minimum wakeup of
    >>>>>> (say) 20 microseconds which will let the guest live long enough for you
    >>>>>> to ftrace it and see what kind of timers it is programming.
    >>>>> I've kept trying to narrow it down, and found out It's triggerable using adjtimex().
    >>> Sorry, one more question: Could you provide details on how is it trigger-able using adjtimex?
    >> It triggers after a while of fuzzing using trinity of just adjtimex ('./trinity --quiet -l off -cadjtimex').
    >>
    >> Trinity is available here: http://git.codemonkey.org.uk/?p=trinity.git .
    >>
    >> Let me know if I can help further with reproducing this, I can probably copy over my testing environment to some other host if you'd like.
    >
    > Ok. Finally I *think* got it reproduced. (Had some trouble initially, as I think since the first time I ran it as a normal user, the socket cache isn't the same as if you run it the first time as root? Anyway, after doing a make clean and rebuilding it started to trigger).
    >
    > I'm not seeing the rcu stall message, but I do manage to trigger two other behaviors: a hard hang and a sort of zombie state where memory isn't properly being freed & everything starts segfaulting. So this may not be the exact same issue, but it triggers quickly as you described (within a few seconds of running trinity as root).
    >
    > It looks like both of these issues are caused by adjtimex(ADJ_SETOFFSET), which adds or subtracts a huge offset and that either goes negative or gets clamped to a ktime_t at KTIME_MAX (if you get clamped the system hangs, if it goes negative, the system barely functions, but sort of drags along).
    >
    > An updated version of my KTIME_MAX sanity checking patch to handle both of these conditions is below.
    >
    > Would you mind giving this patch a shot and letting me know if you still see problems?

    This fixes the problem for me. Thanks!


    \
     
     \ /
      Last update: 2012-08-07 13:01    [W:0.045 / U:89.396 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site