lkml.org 
[lkml]   [2021]   [Feb]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    SubjectRe: [PATCH 5/8] sched/fair: Make check_misfit_status() only compare dynamic capacities
    Date
    On 04/02/21 11:49, Dietmar Eggemann wrote:
    > On 03/02/2021 16:15, Qais Yousef wrote:
    >> On 01/28/21 18:31, Valentin Schneider wrote:
    >
    > [...]
    >
    >>> @@ -10238,7 +10236,7 @@ static void nohz_balancer_kick(struct rq *rq)
    >>> * When ASYM_CPUCAPACITY; see if there's a higher capacity CPU
    >>> * to run the misfit task on.
    >>> */
    >>> - if (check_misfit_status(rq, sd)) {
    >>> + if (check_misfit_status(rq)) {
    >
    > Since check_misfit_status() doesn't need sd anymore it looks like that
    > rcu_dereference(per_cpu(sd_asym_cpucapacity, cpu)) could be replaced by
    > static_branch_unlikely(&sched_asym_cpucapacity)) in nohz_balancer_kick().
    >
    > But as you mentioned in an earlier conversation we do need to check sd
    > because of asymmetric CPU capacity systems w/ exclusive cpusets which
    > could create symmetric islands (unique capacity_orig among CPUs).
    >
    > Maybe worth putting a comment here (similar to the one in sis()) so
    > people don't try to optimize?

    How about:

    --->8---
    diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
    index c2351b87824f..4b71f4d1d324 100644
    --- a/kernel/sched/fair.c
    +++ b/kernel/sched/fair.c
    @@ -6322,15 +6322,8 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target)
    * sd_asym_cpucapacity rather than sd_llc.
    */
    if (static_branch_unlikely(&sched_asym_cpucapacity)) {
    + /* See sd_has_asym_cpucapacity() */
    sd = rcu_dereference(per_cpu(sd_asym_cpucapacity, target));
    - /*
    - * On an asymmetric CPU capacity system where an exclusive
    - * cpuset defines a symmetric island (i.e. one unique
    - * capacity_orig value through the cpuset), the key will be set
    - * but the CPUs within that cpuset will not have a domain with
    - * SD_ASYM_CPUCAPACITY. These should follow the usual symmetric
    - * capacity path.
    - */
    if (sd) {
    i = select_idle_capacity(p, sd, target);
    return ((unsigned)i < nr_cpumask_bits) ? i : target;
    @@ -10274,6 +10267,10 @@ static void nohz_balancer_kick(struct rq *rq)
    }
    }

    + /*
    + * Below checks don't actually use the sd, but they still hinge on its
    + * presence. See sd_has_asym_cpucapacity().
    + */
    sd = rcu_dereference(per_cpu(sd_asym_cpucapacity, cpu));
    if (sd) {
    /*
    diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
    index 21bd71f58c06..ea7f0155e268 100644
    --- a/kernel/sched/sched.h
    +++ b/kernel/sched/sched.h
    @@ -1482,6 +1482,33 @@ DECLARE_PER_CPU(struct sched_domain __rcu *, sd_asym_packing);
    DECLARE_PER_CPU(struct sched_domain __rcu *, sd_asym_cpucapacity);
    extern struct static_key_false sched_asym_cpucapacity;

    +/*
    + * Note that the static key is system-wide, but the visibility of
    + * SD_ASYM_CPUCAPACITY isn't. Thus the static key being enabled does not
    + * imply all CPUs can see asymmetry.
    + *
    + * Consider an asymmetric CPU capacity system such as:
    + *
    + * MC [ ]
    + * 0 1 2 3 4 5
    + * L L L L B B
    + *
    + * w/ arch_scale_cpu_capacity(L) < arch_scale_cpu_capacity(B)
    + *
    + * By default, booting this system will enable the sched_asym_cpucapacity
    + * static key, and all CPUs will see SD_ASYM_CPUCAPACITY set at their MC
    + * sched_domain.
    + *
    + * Further consider exclusive cpusets creating a "symmetric island":
    + *
    + * MC [ ][ ]
    + * 0 1 2 3 4 5
    + * L L L L B B
    + *
    + * Again, booting this will enable the static key, but CPUs 0-1 will *not* have
    + * SD_ASYM_CPUCAPACITY set in any of their sched_domain. This is the intending
    + * behaviour, as CPUs 0-1 should be treated as a regular, isolated SMP system.
    + */
    static inline bool sd_has_asym_cpucapacity(struct sched_domain *sd)
    {
    return static_branch_unlikely(&sched_asym_cpucapacity) &&
    \
     
     \ /
      Last update: 2021-02-04 12:39    [W:3.615 / U:0.020 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site