Messages in this thread Patch in this message |  | | Date | Tue, 5 May 2026 14:48:30 +0200 | | Subject | Re: [PATCH 2/5] sched/fair: Attach sched_domain_shared to sd_asym_cpucapacity | | From | Dietmar Eggemann <> |
| |
On 28.04.26 16:41, Andrea Righi wrote: > From: K Prateek Nayak <kprateek.nayak@amd.com> > > On asymmetric CPU capacity systems, the wakeup path uses > select_idle_capacity(), which scans the span of sd_asym_cpucapacity > rather than sd_llc. > > The has_idle_cores hint however lives on sd_llc->shared, so the > wakeup-time read of has_idle_cores operates on an LLC-scoped blob while > the actual scan/decision spans the asym domain; nr_busy_cpus also lives > in the same shared sched_domain data, but it's never used in the asym > CPU capacity scenario. > > Therefore, move the sched_domain_shared object to sd_asym_cpucapacity > whenever the CPU has a SD_ASYM_CPUCAPACITY_FULL ancestor and that > ancestor is non-overlapping (i.e., not built from SD_NUMA). In that case > the scope of has_idle_cores matches the scope of the wakeup scan. > > Fall back to attaching the shared object to sd_llc in three cases: > > 1) plain symmetric systems (no SD_ASYM_CPUCAPACITY_FULL anywhere); > > 2) CPUs in an exclusive cpuset that carves out a symmetric capacity > island: has_asym is system-wide but those CPUs have no > SD_ASYM_CPUCAPACITY_FULL ancestor in their hierarchy and follow > the symmetric LLC path in select_idle_sibling(); > > 3) exotic topologies where SD_ASYM_CPUCAPACITY_FULL lands on an > SD_NUMA-built domain. init_sched_domain_shared() keys the shared > blob off cpumask_first(span), which on overlapping NUMA domains > would alias unrelated spans onto the same blob. Keep the shared > object on the LLC there; select_idle_capacity() gracefully skips > the has_idle_cores preference when sd->shared is NULL.
Tested it with a coule of real & exotic topolgies, seems to work nicely.
$ cat /sys/devices/system/cpu/cpu*/cpu_capacity 160 160 160 160 498 498 1024 1024
(1) grouping CPUs with same CPU capacities
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/name MC PKG
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/flags ... SD_SHARE_LLC ... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
PKG { 0-7 } MC {0-3} {4,5} {6,7}
(2) flat
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/name MC
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/flags ... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
MC { 0-7 }
(3) flat, exotic, since w/ SMT
$ cat /sys/kernel/debug/sched/domains/cpu[0-7]/domain*/name SMT MC
... SD_SHARE_CPUCAPACITY SD_SHARE_LLC ... ... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL SD_SHARE_LLC ...
MC { 0-7 } SMT {0-1} {2-3} {4-5} {6-7}
(4) exotic, since asymmetric and w/ SMT
$ cat /sys/kernel/debug/sched/domains/cpu[0-3]/domain*/name SMT MC PKG
$ cat /sys/kernel/debug/sched/domains/cpu[0-3]/domain*/flags
... SD_SHARE_CPUCAPACITY SD_SHARE_LLC ... ... SD_SHARE_LLC ... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
$ cat /sys/kernel/debug/sched/domains/cpu[4-7]/domain*/name SMT PKG
$ cat /sys/kernel/debug/sched/domains/cpu[4-7]/domain*/flags ... SD_SHARE_CPUCAPACITY SD_SHARE_LLC ... ... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
PKG { 0-7 } MC { 0-3 } SMT {0-1} {2-3} {4-5} {6-7}
(5) same as (4) but partial CPU capacity asymmetry in MC { 0-3 }
cat /sys/devices/system/cpu/cpu*/cpu_capacity 160 160 498 498 160 160 1024 1024
$ cat /sys/kernel/debug/sched/domains/cpu[0-3]/domain*/flags
... SD_SHARE_CPUCAPACITY SD_SHARE_LLC ... ... SD_ASYM_CPUCAPACITY SD_SHARE_LLC ... ^^^^^^^^^^^^^^^^^^^ ... SD_ASYM_CPUCAPACITY SD_ASYM_CPUCAPACITY_FULL ...
(6) (5) w/ exclusive cpusets with one symmetric island
cd /sys/fs/cgroup echo +cpuset > cgroup.subtree_control mkdir cs1 echo "threaded" > cs1/cgroup.type echo 0-1,4-5 > cs1/cpuset.cpus echo 0 > cs1/cpuset.mems echo root > cs1/cpuset.cpus.partition mkdir cs2 echo "threaded" > cs2/cgroup.type echo 0 > cs2/cpuset.mems echo 2-3,6-7 > cs2/cpuset.cpus echo root > cs2/cpuset.cpus.partition
[ 0.006866] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=0 [ 0.006868] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=1 [ 0.006869] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=2 [ 0.006869] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=3 [ 0.006869] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=4 [ 0.006869] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=5 [ 0.006870] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=6 [ 0.006870] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=7 ... [ 222.767275] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=2 [ 222.767324] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=3 [ 222.767710] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=6 [ 222.767789] claim_asym_sched_domain_shared() (2) sd_asym=PKG cpu=7 [ 222.781015] build_sched_domains() (3) sd=MC cpu=0 [ 222.781017] build_sched_domains() (3) sd=MC cpu=1 [ 222.781017] build_sched_domains() (3) sd=MC cpu=4 [ 222.781018] build_sched_domains() (3) sd=MC cpu=5
[...]
> @@ -2650,6 +2665,49 @@ static void adjust_numa_imbalance(struct sched_domain *sd_llc) > } > } > > +static void init_sched_domain_shared(struct s_data *d, struct sched_domain *sd) > +{ > + int sd_id = cpumask_first(sched_domain_span(sd)); > + > + sd->shared = *per_cpu_ptr(d->sds, sd_id); > + atomic_set(&sd->shared->nr_busy_cpus, sd->span_weight);
Will be used only for sd_llc->shared, not for sd_asym, right?
> + atomic_inc(&sd->shared->ref); > +} > + > +/* > + * For asymmetric CPU capacity, attach sched_domain_shared on the innermost > + * SD_ASYM_CPUCAPACITY_FULL ancestor of @cpu's base domain when that ancestor is > + * not an overlapping NUMA-built domain (then LLC should claim shared). > + * > + * A CPU may lack any FULL ancestor (e.g., exclusive cpuset symmetric island), > + * then LLC must claim shared instead. > + * > + * Note: SD_ASYM_CPUCAPACITY_FULL is only set when multiple distinct capacities
s/multiple/all ? We want to see all possible CPU capacity values in wakeup.
> + * exist in the domain span, so the asym domain we attach to cannot degenerate > + * into a single-capacity group. The relevant edge cases are instead covered by > + * the caveats above. [...]
|  |