Messages in this thread | | | Date | Fri, 5 Aug 2022 09:19:04 +0100 | From | Mel Gorman <> | Subject | Re: Fwd: Question about how to fix numabalancing to work with isolated cpus? |
| |
On Fri, Aug 05, 2022 at 02:42:18PM +0800, guanjing (D) wrote: > On 2022/6/25 14:33, guanjing (D) wrote: > > Hi, > > I notice a problem that you guys discussed in the link below: > > > > https://lore.kernel.org/all/1491464210.4718.82.camel@gmx.de/T/ > > "The isolated cpus are part of the cpus allowed list. In the above > > case, > > numabalancing ends up scheduling some of these tasks on isolated > > cpus. " > > > > I wonder if we're going to fix this eventually? > >
The last discussion did not reach a solid conclusion. Isolated CPUs are avoided by the general CPU scheduler but if a task explicitly uses sched_setaffinity() to include the isolated CPUs then by definition it's allowed to use those CPUs and NUMA balancing will use allowed CPUs to improve locality if possible. It's not clear that this is a bug at all other than it may be surprising in some cases. Furthermore, a task that is sensitive to latency probably should be using mempolicies to avoid NUMA balancing entirely as it's a source of interference.
There needs to be a use case that describes exactly what the sematics should be for a task using isolated CPUs and why. For example, if the task policy only allowed isolated CPUs then NUMA balancing may never be able to select a CPU with task_numa_find_cpu, is that expected? Should it be the case that NUMA balancing avoids isolated CPUs but will select one if no other local CPU is available? If so, why? If only isolated CPUs are available, is it expected and desired that data will be migrated instead? If a task is running on an isolated CPU, should NUMA balancing avoid moving the task off the CPU? If so, why? Should the behaviour be that if a task is allowed to run on isolated CPUs, should NUMA balancing simply be disabled? If so, why? Whatever the result, the sematics should be mentioned under the isocpus= parameter with an additional note under numa_balancing=.
If anything is to change here, it needs to be made clear why it's a bug that NUMA balancing selects an allowed but isolated CPU and a description of what the desired semantics should be.
-- Mel Gorman SUSE Labs
| |