[lkml]   [2009]   [Oct]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH RFC tip/core/rcu 1/3] rcu: The Bloatwatch Edition, v7
On Wed, Oct 14, 2009 at 08:37:18AM +0800, Lai Jiangshan wrote:
> Paul E. McKenney wrote:
> >> It's an old issue.
> >> It's not only about RCUTINY, it's also about other rcu implementations:
> >>
> >> rcu_enter_nohz()/rcu_exit_nohz() are not called in pairs.
> >>
> >> irq_exit() calls tick_nohz_stop_sched_tick() which calls rcu_enter_nohz(),
> >> where is the corresponding rcu_exit_nohz()?
> >> (or tick_nohz_restart_sched_tick())?
> >
> > The tick_nohz_restart_sched_tick() function is called from the various
> > per-architecture cpu_idle() functions (or default_idle() or whatever
> > name that the architecture uses). For example, in:
> >
> > arch/x86/kernel/process_64.c
> >
> > the cpu_idle() function invokes tick_nohz_restart_sched_tick() just
> > before invoking schedule() to exit the idle loop.
> >
> > And, as you say, tick_nohz_restart_sched_tick() invokes rcu_exit_nohz().
> These tick_nohz_restart_sched_tick() which are called from the various
> per-architecture cpu_idle() functions are not the opposite of
> the tick_nohz_stop_sched_tick() in *irq_exit()*. So I figure that
> rcu_enter_nohz()/rcu_exit_nohz() are not called in pairs.

OK, let's start with rcu_enter_nohz(), which tells RCU that the running
CPU is going into dyntick-idle mode, and thus should be ignored by RCU.
Let's do the idle loop first:

o Upon entry to the idle() loop (using cpu_idle() in
arch/x86/kernel/process_64.c for this exercise),
we invoke tick_nohz_stop_sched_tick(1), which says we
are in an idle loop. (This is in contrast to the call
from irq_exit(), where we are not in the idle loop.)

o tick_nohz_stop_sched_tick() invokes rcu_enter_nohz(),
does a bunch of timer checking, and returns. If anything
indicated that entering dyntick-idle mode would be bad,
we raise TIMER_SOFTIRQ to kick us out of this mode.

Either way, we return to the idle loop.

o The idle loops until need_resched(). Upon exit from the
idle loop, we call tick_nohz_restart_sched_tick(), which
invokes rcu_exit_nohz(), which tells RCU to start paying
attention to this CPU once more.

OK, now for interrupts.

o The hardware interrupt handlers invoke irq_enter(), which in
turn invokes rcu_irq_enter(). This has no real effect (other
than incrementing a counter) if the interrupt did not come
from dyntick-idle mode.

Either way, RCU is now paying attention to RCU read-side
critical sections on this CPU.

o Upon return from interrupt, the hardware interrupt handlers
invoke irq_exit(), which in turn invokes rcu_irq_exit().
This has no real effect (other than decrementing a counter)
if the interrupt is not returning to dyntick-idle mode.

However, if the interrupt -is- returning to dyntick-idle
mode, then RCU will stop paying attention to RCU read-side
critical sections on this CPU.

So I do believe that rcu_enter_nohz() and rcu_exit_nohz() are in fact
invoked in pairs. One strange thing about this is that the idle loop
first invokes rcu_enter_nohz(), then invokes rcu_exit_nohz(), while
an interrupt handler first invokes rcu_irq_enter() and then invokes
rcu_irq_exit(). So the idle loop enters dyntick-idle mode and then
leaves it, while an interrupt handler might leave dyntick-idle mode and
then re-enter it.

Or am I still missing something here?

Thanx, Paul

 \ /
  Last update: 2009-10-14 03:19    [W:0.047 / U:5.512 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site