Messages in this thread Patch in this message | | | Date | Fri, 09 Oct 2020 06:35:20 -0000 | From | "tip-bot2 for Paul E. McKenney" <> | Subject | [tip: core/rcu] rcu: Attempt QS when CPU discovers GP for strict GPs |
| |
The following commit has been merged into the core/rcu branch of tip:
Commit-ID: 1a2f5d57a33f7b9189b6b3e997eb858301482d79 Gitweb: https://git.kernel.org/tip/1a2f5d57a33f7b9189b6b3e997eb858301482d79 Author: Paul E. McKenney <paulmck@kernel.org> AuthorDate: Thu, 06 Aug 2020 16:35:08 -07:00 Committer: Paul E. McKenney <paulmck@kernel.org> CommitterDate: Mon, 24 Aug 2020 18:40:26 -07:00
rcu: Attempt QS when CPU discovers GP for strict GPs
A given CPU normally notes a new grace period during one RCU_SOFTIRQ, but avoids reporting the corresponding quiescent state until some later RCU_SOFTIRQ. This leisurly approach improves efficiency by increasing the number of update requests served by each grace period, but is not what is needed for kernels built with CONFIG_RCU_STRICT_GRACE_PERIOD=y.
This commit therefore adds a new rcu_strict_gp_check_qs() function which, in CONFIG_RCU_STRICT_GRACE_PERIOD=y kernels, simply enters and immediately exist an RCU read-side critical section. If the CPU is in a quiescent state, the rcu_read_unlock() will attempt to report an immediate quiescent state. This rcu_strict_gp_check_qs() function is invoked from note_gp_changes(), so that a CPU just noticing a new grace period might immediately report a quiescent state for that grace period.
Reported-by Jann Horn <jannh@google.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org> --- kernel/rcu/tree.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 4436857..36a860c 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -1575,6 +1575,19 @@ static void __maybe_unused rcu_advance_cbs_nowake(struct rcu_node *rnp, } /* + * In CONFIG_RCU_STRICT_GRACE_PERIOD=y kernels, attempt to generate a + * quiescent state. This is intended to be invoked when the CPU notices + * a new grace period. + */ +static void rcu_strict_gp_check_qs(void) +{ + if (IS_ENABLED(CONFIG_RCU_STRICT_GRACE_PERIOD)) { + rcu_read_lock(); + rcu_read_unlock(); + } +} + +/* * Update CPU-local rcu_data state to record the beginnings and ends of * grace periods. The caller must hold the ->lock of the leaf rcu_node * structure corresponding to the current CPU, and must have irqs disabled. @@ -1644,6 +1657,7 @@ static void note_gp_changes(struct rcu_data *rdp) } needwake = __note_gp_changes(rnp, rdp); raw_spin_unlock_irqrestore_rcu_node(rnp, flags); + rcu_strict_gp_check_qs(); if (needwake) rcu_gp_kthread_wake(); }
| |