lkml.org 
[lkml]   [2010]   [Oct]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    SubjectRe: [PATCH][GIT PULL] tracing: Fix compile issue for trace_sched_wakeup.c
    On Thu, Oct 21, 2010 at 01:26:14PM +0200, Ingo Molnar wrote:
    > * Steven Rostedt <rostedt@goodmis.org> wrote:
    >
    > > On Wed, 2010-10-20 at 17:40 +0200, Ingo Molnar wrote:
    > > > FYI, there's a new mystery hang (sometimes crash) that triggers in -tip - and which
    > > > seems to be tracing related. See the crashlog below - config attached.
    > > >
    > > > It's not bisectable - small changes in the kernel make the bug come/go. (might be a
    > > > race of some sorts)
    > > >
    > >
    > >
    > > > [ 42.324027] Testing all events:
    > > > [ 245.668090] INFO: task swapper:1 blocked for more than 120 seconds.
    > > > [ 245.672051] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    > > > [ 245.676026] swapper D f6420b40 6544 1 0 0x00000000
    > > > [ 245.684051] f6437dac 00000046 f694aac0 f6420b40 f6438000 f6437d74 f6438294 f6438290
    > > > [ 245.692237] c2192ac0 c204e6c0 c2192ac0 c2192ac0 f6438290 00000000 f6438000 ff2ffa7d
    > > > [ 245.701068] 00000009 f6420b40 f6437e5c 7fffffff f6438000 f6437dfc f6437e5c 7fffffff
    > > > [ 245.709071] Call Trace:
    > > > [ 245.711551] [<c1a7f561>] schedule_timeout+0x1c/0x1e7
    > > > [ 245.712036] [<c1a818b6>] ? _raw_spin_unlock_irq+0x2d/0x43
    > > > [ 245.716037] [<c1027f2d>] ? sub_preempt_count+0x4/0x98
    > > > [ 245.720061] [<c1a818b6>] ? _raw_spin_unlock_irq+0x2d/0x43
    > > > [ 245.724036] [<c1027fb4>] ? sub_preempt_count+0x8b/0x98
    > > > [ 245.728036] [<c1a7e76b>] wait_for_common+0xc1/0x11a
    > > > [ 245.732062] [<c102de32>] ? default_wake_function+0x0/0x12
    > > > [ 245.736041] [<c1a7e863>] wait_for_completion+0x17/0x19
    > > > [ 245.740069] [<c10667a2>] __stop_cpus+0xdd/0x103
    > > > [ 245.744072] [<c1a7e6db>] ? wait_for_common+0x31/0x11a
    > > > [ 245.748040] [<c10665a4>] ? stop_machine_cpu_stop+0x0/0x9a
    > > > [ 245.752040] [<c106683d>] stop_cpus+0x2c/0x3f
    > > > [ 245.756069] [<c10668af>] __stop_machine+0x5f/0x67
    > > > [ 245.760186] [<c1006240>] ? stop_machine_text_poke+0x0/0x43
    > > > [ 245.764040] [<c1006240>] ? stop_machine_text_poke+0x0/0x43
    > > > [ 245.768071] [<c19f0a73>] ? cfdgml_create+0x2b/0xde
    > > > [ 245.772040] [<c10060fd>] text_poke_smp+0x3a/0x42
    > > > [ 245.776039] [<c19f0a73>] ? cfdgml_create+0x2b/0xde
    > >
    > >
    > > > [ 245.780098] [<c1005b9c>] arch_jump_label_transform+0x53/0x67
    > > > [ 245.784042] [<c104ef0d>] jump_label_update+0x49/0x98
    > >
    > > Looks like this code had jump labels enabled. Do you have a dump where
    > > they are not enabled?
    >
    > No. Good find - and the timeline agrees too, these crashes started triggering when i
    > pulled jump labels from you.
    >
    > Thanks,
    >
    > Ingo

    Hi,

    (adding Tejun to the 'cc list)

    I finally found that we actually continue to run after the above
    apparent 'hang'. That is, we continue to make progress updating the jump
    labels. And doing a dump of all the system tasks at the time of the hang
    showed the processes in various places besides the stop machine threads.
    Thus, I thought that perhaps, for some reason the stop machine threads
    weren't being scheduled.

    Thus, I tried commenting out the special scheduling that is set up for
    stop machine threads, and that fixed the hang. I haven't yet looked into
    what might be going wrong with that scheduling...but maybe somebody else
    knows...

    thanks,

    -Jason


    diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
    index 090c288..3013b85 100644
    --- a/kernel/stop_machine.c
    +++ b/kernel/stop_machine.c
    @@ -307,7 +307,7 @@ static int __cpuinit cpu_stop_cpu_callback(struct notifier_block *nfb,
    return NOTIFY_BAD;
    get_task_struct(p);
    kthread_bind(p, cpu);
    - sched_set_stop_task(cpu, p);
    + //sched_set_stop_task(cpu, p);
    stopper->thread = p;
    break;

    @@ -326,7 +326,7 @@ static int __cpuinit cpu_stop_cpu_callback(struct notifier_block *nfb,
    {
    struct cpu_stop_work *work;

    - sched_set_stop_task(cpu, NULL);
    + //sched_set_stop_task(cpu, NULL);
    /* kill the stopper */
    kthread_stop(stopper->thread);
    /* drain remaining works */

    \
     
     \ /
      Last update: 2010-10-22 03:47    [W:0.053 / U:92.008 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site