Messages in this thread |  | | | Date | Sat, 21 Mar 2009 21:09:56 +0100 | | From | Frederic Weisbecker <> | | Subject | Re: [PATCH 0/5] [GIT PULL] updates for tip/tracing/ftrace |
| |
On Sat, Mar 21, 2009 at 07:18:58PM +0100, Ingo Molnar wrote: > > * Frederic Weisbecker <fweisbec@gmail.com> wrote: > > > > [<ffffffff8020c79d>] return_to_handler+0x0/0x73 > > > [<ffffffff8029ea13>] rcu_pending+0x2c/0x5e > > > [<ffffffff8020c79d>] return_to_handler+0x0/0x73 > > > [<ffffffff8026abef>] update_process_times+0x3c/0x77 > > > [<ffffffff8020c79d>] return_to_handler+0x0/0x73 > > > [<ffffffff802875dd>] tick_periodic+0x6e/0x70 > > > > > > Still hanging in the timer interrupt. > > I guess it makes the timer interrupt servicing too slow and then > > once it is serviced, another one is raised. > > > > But the cause is perhaps more complex > > > > I think you have had too much hanging of this type. I'm preparing > > a fix that checks periodically if the function graph tracer is > > spending too much time in an interrupt. > > > > I guess I could count the number of function executed between the > > irq entry and its exit. > > > > That's the best: if we are hanging in an interrupt, it could be > > whatever interrupt and the jiffies could not be progressing so I > > can't rely on time but only on number of functions executed. > > > > May be 10000 calls is a good threshold before killing the function > > graph inside an interrupt? > > i think the problem isnt even the IRQ handler - but the fact that > the (timer) irq handler gets re-triggered - so all we do is > processing timer IRQs. > > Your patch would detect a timer IRQ hanging - but it would not > detect the 'system makes no progress because there's always anoter > pending timer IRQ to execute' situation.
Ah, you're right.
> So i think we need a "function trace watchdog" - which kills the > tracer if we do more than 100,000,000 entries since we started the > self-test, or so. > > Ingo
The problem is that it can happen also on other contexts than selftests. For example with ftrace=function_graph or by simply enabling the tracer later.
Sometimes it can happen during the selftests, sometimes it's only revealed by manually enabling it. I just remind another hang that you reported earlier and which I half-solved by fixing a pointless softirq call...
|  |