Messages in this thread | | | Date | Mon, 25 Jun 2012 15:39:40 -0700 | From | "Paul E. McKenney" <> | Subject | [GIT PULL rcu/urgent] Fix for RCU-related hang |
| |
Hello, Ingo,
This series has a single patch that fixes a system hang that can occur in perhaps unusual but very real circumstances. This hang occurs because of a very stupid bug of mine introduced in commit b1420f1c (Make rcu_barrier() less disruptive) that can cause CPUs to miscount RCU callbacks. The sequence of events leading to the hang is as follows:
1. A CPU miscounts its callbacks. 2. That CPU invokes all of its callbacks, so that its callback list is empty, but the callback count is nonzero. 3. That CPU goes offline. Because its callback list is empty, RCU's CPU-hotplug CPU_DEAD notifiers leave both the list and the count alone. (In contrast, had the list been non-empty, RCU's CPU_DEAD notifiers would have emptied the list and zeroed the count.) 4. One of the remaining CPUs executes one of the rcu_barrier() family of primitives. The rcu_barrier() primitive notes that the offline CPU has a non-zero count of callbacks, and therefore hangs waiting for this count to reach zero. The theory behind the indefinite wait is that the only reason that an offline CPU can have a non-zero number of RCU callbacks is that the CPU's CPU_DEAD notifiers have not yet executed. But they already have executed, so the offlined CPU's callback count will remain non-zero until it is brought back online, in other words, perhaps never.
However, this bug is likely to pass a combined rcutorture/CPU-hotplug stress test because offlined CPUs tend to be brought back online reasonably quickly. For the rcutorture tests to fail, the system must be in the state indicated by step #3 above at the time the "rmmod rcutorture" executes.
The fix is simply to prevent the miscounting.
This change is available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git rcu/urgent
Thanx, Paul
------------------------------------------------------------------------
Paul E. McKenney (1): rcu: Stop rcu_do_batch() from multiplexing the "count" variable
kernel/rcutree.c | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-)
| |