lkml.org 
[lkml]   [2022]   [Apr]   [8]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH RFC] rcu/nocb: Provide default all-CPUs mask for RCU_NOCB_CPU=y
    On Fri, Apr 08, 2022 at 10:52:21AM -0400, Joel Fernandes wrote:
    > On Fri, Apr 8, 2022 at 10:22 AM Paul E. McKenney <paulmck@kernel.org> wrote:
    > >
    > > On Thu, Apr 07, 2022 at 09:07:33PM +0000, Joel Fernandes wrote:
    > > > On systems with CONFIG_RCU_NOCB_CPU=y, there is no default mask provided
    > > > which ends up not offloading any CPU. This patch removes yet another
    > > > dependency from the bootloader having to know about RCU, about how many
    > > > CPUs the system has, and about how to provide the mask. Basically, I
    > > > think we should stop pretending that the user knows what they are doing :).
    > > > In other words, if NO_CB_CPU is enabled, lets make use of it.
    > > >
    > > > My goal is to make RCU as zero-config as possible with sane defaults. If
    > > > user wants to provide rcu_nocbs= or nohz_full= options, then those will
    > > > take precedence and this patch will have no effect.
    > > >
    > > > I tested providing rcu_nocbs= option, ensuring that is preferred over this.
    > >
    > > Unless something has changed, this would change behavior relied upon
    > > the enterprise distros. Last I checked, they want to supply a single
    > > binary, as evidenced by the recent CONFIG_PREEMPT_DYNAMIC Kconfig option,
    > > and they also want the default to be non-offloaded. That is, given a
    > > kernel built with CONFIG_RCU_NOCB_CPU=y and without either a nohz_full
    > > or a nocbs_cpu boot parameter, all of the CPUs must be non-offloaded.
    >
    > Just curious, do you have information (like data, experiment results)
    > on why they want default non-offloaded? Or maybe they haven't tried
    > the recent work done in NOCB code?

    I most definitely do. When I first introduced callback offloading, I
    made it completely replace softirq callback invocation. There were some
    important throughput-oriented workloads that got hit with significant
    performance degradation due to this change. Enterprise Java workloads
    were the worst hit.

    Android does not run these workloads, and I am not aware of ChromeOS
    running them, either.

    > Another option I think is to make it enforce NOCB if NR_CPUS <= 32 if
    > that makes sense.

    That would avoid hurting RHEL and SLES users, so this would be better
    than making the change unconditionally. But there are a lot of distros
    out there.

    I have to ask... Isn't there already a way of specifying a set of kernel
    boot parameters that are required for ChromeOS? If so, add rcu_nocbs=0-N
    to that list and be happy.

    > > So for me to push this to mainline, I need an ack from someone from each
    > > of the enterprise distros, and each of those someones needs to understand
    > > the single-binary strategy used by the corresponding distro.
    >
    > Ok.
    >
    > > And is it really all -that- hard to specify an additional boot parameter
    > > across ChromeOS devices? Android seems to manage it. ;-)
    >
    > That's not the hard part I think. The hard part is to make sure a
    > future Linux user who is not an RCU expert does not forget to turn it
    > on. ChromeOS is not the only OS that I've seen someone forget to do it
    > ;-D. AFAIR, there were Android devices too in the past where I saw
    > this forgotten. I don't think we should rely on the users doing the
    > right thing (as much as possible).
    >
    > The single kernel binary point makes sense but in this case, I think
    > the bigger question that I'd have is what is the default behavior and
    > what do *most* users of RCU want. So we can keep sane defaults for the
    > majority and reduce human errors related to configuration.

    If both the ChromeOS and Android guys need it, I could reinstate the
    old RCU_NOCB_CPU_ALL Kconfig option. This was removed due to complaints
    about RCU Kconfig complexity, but I believe that Reviewed-by from ChromeOS
    and Android movers and shakers would overcome lingering objections.

    Would that help?

    Thanx, Paul

    \
     
     \ /
      Last update: 2022-04-08 17:50    [W:4.699 / U:0.032 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site