lkml.org 
[lkml]   [2012]   [Jun]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH] x86: Avoid intermixing cpu dump_stack output on multi-processor systems
On Fri, Jun 01, 2012 at 05:56:03PM -0500, Russ Anderson wrote:
> > I am curious, your NMI handler has locking wrapped around dump_stack,
> > shouldn't that serialize the output the way you want it? Why isn't that
> > working?
>
> Yes, you're right, it does. It is working. I'd forgotten that
> the community kernel has uv_nmi_lock in uv_handle_nmi. Must
> be working too much with distro kernels. :-) But that doesn't
> help for all the other code paths than call dump_stack.

Sure, I agree. But not every caller of dump_stack needs to dump for all
cpus. I was just trying to avoid the ugly complicated code of
cmp_xchange you were requesting.

>
>
> > > FWIW, "Wait for up to 10 seconds for all CPUs to do the backtrace" on
> > > a 4096 cpu system isn't long enough. :-)
> >
> > Good point. :-)
> >
> > >
> > > > Whereas the lock you are proposing can be called in a mixture of NMI and
> > > > IRQ which could cause deadlocks I believe.
> > >
> > > Since this is a lock just around the dump_stack printk, would
> > > checking for forward progress and a timeout to catch any possible
> > > deadlock be sufficient? In the unlikely case of a deadlock the
> > > lock gets broken and some of the cpu backtraces get intermixed.
> > > That is still a huge improvement over the current case where
> > > all of the backtraces get intermixed.
> >
> > I saw your new patch based on Frederick's input. It seems to take care of
> > deadlock situations though you run into the starving lock problem that
> > ticketed spinlocks solved. Which is why I am curious why moving the
> > locking one layer up to the NMI handler (which is where it is currently),
> > didn't fix your problem.
>
> Locking in dump_stack would remove the need for uv_nmi_lock.

I agree. I was just wondering if the added complexities were worth it for
the normal case.

If the uv_nmi_lock works, then I feel comfortable that your second patch
based on Frederic's suggestion will work too. I just feel uncomfortable
with locking on a stack dump that should be reliable. It is one thing to
do the locking in only NMI space or only IRQ space, but now we are
traversing both. I don't think there will be any deadlocks (based on the
else path).

It could just be an overhyped paranoia of mine. But that was my biggest
hestitation.

Cheers,
Don


\
 
 \ /
  Last update: 2012-06-04 17:01    [W:2.381 / U:0.384 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site