lkml.org 
[lkml]   [2013]   [Jan]   [10]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRE: [PATCH] hardlockup: detect hard lockups without NMIs using secondary cpus
Date


> -----Original Message-----
> From: Colin Cross [mailto:ccross@android.com]
> Sent: Thursday, January 10, 2013 9:58 AM
> To: linux-kernel@vger.kernel.org
> Cc: Andrew Morton; Don Zickus; Ingo Molnar; Thomas Gleixner; Liu,
> Chuansheng; linux-arm-kernel@lists.infradead.org; Colin Cross
> Subject: [PATCH] hardlockup: detect hard lockups without NMIs using
> secondary cpus
>
> Emulate NMIs on systems where they are not available by using timer
> interrupts on other cpus. Each cpu will use its softlockup hrtimer
> to check that the next cpu is processing hrtimer interrupts by
> verifying that a counter is increasing.
>
> This patch is useful on systems where the hardlockup detector is not
> available due to a lack of NMIs, for example most ARM SoCs.
> Without this patch any cpu stuck with interrupts disabled can
> cause a hardware watchdog reset with no debugging information,
> but with this patch the kernel can detect the lockup and panic,
> which can result in useful debugging info.
>
> Signed-off-by: Colin Cross <ccross@android.com>
> +static void watchdog_check_hardlockup_other_cpu(void)
> +{
> + int cpu;
> + cpumask_t cpus = watchdog_cpus;
> +
> + /*
> + * Test for hardlockups every 3 samples. The sample period is
> + * watchdog_thresh * 2 / 5, so 3 samples gets us back to slightly over
> + * watchdog_thresh (over by 20%).
> + */
> + if (__this_cpu_read(hrtimer_interrupts) % 3 != 0)
> + return;
> +
> + /* check for a hardlockup on the next cpu */
> + cpu = cpumask_next(smp_processor_id(), &cpus);
> + if (cpu >= nr_cpu_ids)
> + cpu = cpumask_first(&cpus);
> + if (cpu == smp_processor_id())
> + return;
> +
> + smp_rmb();
> +
> + if (per_cpu(watchdog_nmi_touch, cpu) == true) {
> + per_cpu(watchdog_nmi_touch, cpu) = false;
> + return;
> + }
> +
> + if (is_hardlockup_other_cpu(cpu)) {
> + /* only warn once */
One possible case for new hotplug CPU that false hardlockup case.
1/ Assume CPU1, CPU2 are online, CPU3 is being hotplug:
CPU3: CPU2:
watchdog_nmi_enable()
per_cpu(watchdog_nmi_touch, cpu) = true;
cpumask_set_cpu(cpu, &watchdog_cpus);
watchdog_check_hardlockup_other_cpu()
per_cpu(watchdog_nmi_touch, cpu) = false; == > Here cpu is CPU3

2/ Before CPU3's first hrtimer interrupt coming, CPU2 is been offlined.
Then CPU1's next CPU is CPU3. But we can not use CPU3's watchdog_nmi_touch to defense
false CPU3 hardlock more. When CPU1's hrtimer interrupt coming, it is possible report CPU3
false hard lockup.

Is it the case?




\
 
 \ /
  Last update: 2013-01-11 03:21    [W:0.512 / U:0.740 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site