[lkml]   [2019]   [Nov]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH v2] sched/topology, cpuset: Account for housekeeping CPUs to avoid empty cpumasks
On 15/11/2019 17:18, Michal Koutný wrote:
> Hello.
> On Thu, Nov 14, 2019 at 04:03:50PM +0000, Valentin Schneider <> wrote:
>> Michal, could I nag you for a reviewed-by? I'd feel a bit more confident
>> with any sort of approval from folks who actually do use cpusets.
> TL;DR I played with the v5.4-rc6 _without_ this fixup and I conclude it
> unnecessary (IOW my previous theoretical observation was wrong).

Thanks for going through the trouble of testing the thing.

> The original problem is non-issue with v2 cpuset controller, because
> effective_cpus are never empty. isolcpus doesn't take out cpuset CPUs,
> hotplug does. In the case, no online CPU remains in the cpuset, it
> inherits ancestor's non-empty cpuset.

But we still take out the isolcpus from the domain span before handing it
over to the scheduler:

cpumask_or(dp, dp, b->effective_cpus);
cpumask_and(dp, dp, housekeeping_cpumask(HK_FLAG_DOMAIN));


> I reproduced the problem with v1 (before your fix). However, in v1
> effective == allowed (we're destructive and overwrite allowed on
> hotunplug) and we already check the emptiness of
> cpumask_intersects(cp->cpus_allowed, housekeeping_cpumask(HK_FLAG_DOMAIN)
> few lines higher. I.e. the fixup adds redundant check against the empty
> sched domain production.

...You're right, I've been misreading that as a '!is_sched_load_balance()'
condition ever since. Duh. So this condition will always catch cpusets than
only span outside the housekeeping domain, and my previous fixup will
catch newly-empty cpusets (due to HP). Perhaps it would've been cleaner to
merge the two, but as things stand this patch isn't needed (as you say).

I tried this out to really be sure (8 CPU SMP aarch64 qemu target):

cd /sys/fs/cgroup/cpuset

mkdir cs1
echo 1 > cs1/cpuset.cpu_exclusive
echo 0 > cs1/cpuset.mems
echo 0-4 > cs1/cpuset.cpus

mkdir cs2
echo 1 > cs2/cpuset.cpu_exclusive
echo 0 > cs2/cpuset.mems
echo 5-7 > cs2/cpuset.cpus

echo 0 > cpuset.sched_load_balance

booted with


It seems that creating a cpuset with CPUs only outside the housekeeping
domain is forbidden, so I'm creating cs2 with *one* CPU in the domain. When
I hotplug it out, nothing dies horribly:

echo 0 > /sys/devices/system/cpu/cpu5/online
[ 24.688145] CPU5: shutdown
[ 24.689438] psci: CPU5 killed.
[ 24.714168] allowed=0-4 effective=0-4 housekeeping=0-5
[ 24.714642] allowed=6-7 effective=6-7 housekeeping=0-5
[ 24.715416] CPU5 attaching NULL sched-domain.

> Sorry for the noise and HTH,

Sure does, thanks!

> Michal

 \ /
  Last update: 2019-11-15 19:49    [W:0.080 / U:1.540 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site