[lkml]   [2018]   [Jul]   [11]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
Patch in this message
SubjectRe: [RFC] Make need_resched() return true when rcu_urgent_qs requested
On Mon, 2018-07-09 at 15:08 -0700, Paul E. McKenney wrote:

> And the earlier patch was against my -rcu tree, which won't be all that
> helpful for v4.15.  Please see below for a lightly tested backport to v4.15.
> It should apply to all the releases of interest.  If other backports
> are needed, please remind me of my woodhouse.v4.15.2018.07.09a tag.
>                                                         Thanx, Paul
> ------------------------------------------------------------------------
> commit 6361b81827a8f93f582124da385258fc04a38a7f
> Author: Paul E. McKenney <>
> Date:   Mon Jul 9 13:47:30 2018 -0700
>     rcu: Make need_resched() respond to urgent RCU-QS needs
>     The per-CPU rcu_dynticks.rcu_urgent_qs variable communicates an urgent
>     need for an RCU quiescent state from the force-quiescent-state processing
>     within the grace-period kthread to context switches and to cond_resched().
>     Unfortunately, such urgent needs are not communicated to need_resched(),
>     which is sometimes used to decide when to invoke cond_resched(), for
>     but one example, within the KVM vcpu_run() function.  As of v4.15, this
>     can result in synchronize_sched() being delayed by up to ten seconds,
>     which can be problematic, to say nothing of annoying.
>     This commit therefore checks rcu_dynticks.rcu_urgent_qs from within
>     rcu_check_callbacks(), which is invoked from the scheduling-clock
>     interrupt handler.  If the current task is not an idle task and is
>     not executing in usermode, a context switch is forced, and either way,
>     the rcu_dynticks.rcu_urgent_qs variable is set to false.  If the current
>     task is an idle task, then RCU's dyntick-idle code will detect the
>     quiescent state, so no further action is required.  Similarly, if the
>     task is executing in usermode, other code in rcu_check_callbacks() and
>     its called functions will report the corresponding quiescent state.
>     Reported-by: David Woodhouse <>
>     Suggested-by: Peter Zijlstra <>
>     Signed-off-by: Paul E. McKenney <>
>     [ paulmck: Backported to v4.15.  Probably applies elsewhere. ]

Hm, this doesn't appear to work. I'm still seeing latencies of 4-5
seconds in my testing. In fact, even our old workaround of adding
rcu_all_qs() into vcpu_enter_guest() didn't properly fix it AFAICT.

I'm just creating a VM with lots of CPUs, then attaching new devices to
it to cause the VMM to open more file descriptors, until it hits a
power of two and invokes expand_fdtable().

expand_fdtable (512) sync took 10472394964 cycles (3500000 µs).
expand_fdtable (512) sync took 15298908072 cycles (5100000 µs).

--- a/fs/file.c
+++ b/fs/file.c
@@ -162,8 +162,16 @@ static int expand_fdtable(struct files_struct *files, unsigned int nr)
        /* make sure all __fd_install() have seen resize_in_progress
         * or have finished their rcu_read_lock_sched() section.
-       if (atomic_read(&files->count) > 1)
+       if (atomic_read(&files->count) > 1) {
+               unsigned long sync_start, sync_end;
+               unsigned long j_start, j_end;
+               j_start = jiffies;
+               sync_start = get_cycles();
+               sync_end = get_cycles();
+               j_end = jiffies;
+               printk("expand_fdtable (%d) sync took %ld cycles (%ld µs).\n", nr, sync_end - sync_start, jiffies_to_usecs(j_end - j_start));
+       }
        if (!new_fdt)[unhandled content-type:application/x-pkcs7-signature]
 \ /
  Last update: 2018-07-11 12:58    [W:0.124 / U:9.140 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site