Messages in this thread Patch in this message | | | Date | Fri, 10 Sep 2010 17:17:35 +0200 | From | Robert Richter <> | Subject | Re: [PATCH 0/3 v2] nmi perf fixes |
| |
On 10.09.10 10:46:34, Ingo Molnar wrote: > > * Don Zickus <dzickus@redhat.com> wrote: > > > On Fri, Sep 10, 2010 at 01:41:40PM +0200, Peter Zijlstra wrote: > > > On Thu, 2010-09-02 at 15:07 -0400, Don Zickus wrote: > > > > Fixes to allow unknown nmis to pass through the perf nmi handler instead > > > > of being swallowed. Contains patches that are already in Ingo's tree. Added > > > > here for completeness. Based on ingo/tip > > > > > > > > Tested on intel/amd > > > > > > > > v2: patch cleanups and consolidation, no code changes > > > > > > > > Don Zickus (1): > > > > perf, x86: Fix accidentally ack'ing a second event on intel perf > > > > counter > > > > > > > > Peter Zijlstra (1): > > > > perf, x86: Fix handle_irq return values > > > > > > > > Robert Richter (1): > > > > perf, x86: Try to handle unknown nmis with an enabled PMU > > > > > > > > arch/x86/kernel/cpu/perf_event.c | 59 +++++++++++++++++++++++++------- > > > > arch/x86/kernel/cpu/perf_event_intel.c | 15 +++++--- > > > > arch/x86/kernel/cpu/perf_event_p4.c | 2 +- > > > > 3 files changed, 56 insertions(+), 20 deletions(-) > > > > > > Both Ingo and I are getting Dazed and confused on our AMD machines, it > > > started before yesterday (that is, after backing out all my recent > > > changes it still gets dazed), so I suspect this set. > > > > > > I'll look at getting a trace of the thing, but if any of you has a > > > bright idea... > > > > What are you running to create the problem? I can try and duplicate > > it here. > > It happens easily here - just running something like: > > perf record -g ./hackbench 10
I try to reproduce it, which systems are affected?
> > a couple of times triggers it. Note, unlike with the earlier bug, the > NMIs are not permanently 'stuck' - and everything continues working. > Obviously the messages are nasty looking so this is a regression we need > to fix.
The patch below adds ratelimits.
-Robert
--
From 1747710d684302b806b145e5acb590ab2088e5ca Mon Sep 17 00:00:00 2001 From: Robert Richter <robert.richter@amd.com> Date: Thu, 5 Aug 2010 18:04:31 +0200 Subject: [PATCH] x86: ratelimit NMI messages
In case of a storm of unknown NMIs the cpu get stucked. This patch adds ratelimits to avoid this.
Signed-off-by: Robert Richter <robert.richter@amd.com> --- arch/x86/kernel/traps.c | 16 +++++++++------- 1 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 60788de..97a492d 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -366,15 +366,17 @@ unknown_nmi_error(unsigned char reason, struct pt_regs *regs) return; } #endif - printk(KERN_EMERG - "Uhhuh. NMI received for unknown reason %02x on CPU %d.\n", - reason, smp_processor_id()); - - printk(KERN_EMERG "Do you have a strange power saving mode enabled?\n"); + if (printk_ratelimit()) { + printk(KERN_EMERG "Uhhuh. NMI received for unknown reason" + " %02x on CPU %d.\n", reason, smp_processor_id()); + printk(KERN_EMERG + "Do you have a strange power saving mode enabled?\n"); + } if (panic_on_unrecovered_nmi) panic("NMI: Not continuing"); - - printk(KERN_EMERG "Dazed and confused, but trying to continue\n"); + if (printk_ratelimit()) + printk(KERN_EMERG + "Dazed and confused, but trying to continue\n"); } static notrace __kprobes void default_do_nmi(struct pt_regs *regs) -- 1.7.1.1
-- Advanced Micro Devices, Inc. Operating System Research Center
| |