Messages in this thread | | | From | "Zengtao (B)" <> | Subject | RE: [PATCH] cpu-topology: warn if NUMA configurations conflicts with lower layer | Date | Fri, 3 Jan 2020 04:24:04 +0000 |
| |
> -----Original Message----- > From: Valentin Schneider [mailto:valentin.schneider@arm.com] > Sent: Thursday, January 02, 2020 9:22 PM > To: Zengtao (B); Sudeep Holla > Cc: Linuxarm; Greg Kroah-Hartman; Rafael J. Wysocki; > linux-kernel@vger.kernel.org; Morten Rasmussen > Subject: Re: [PATCH] cpu-topology: warn if NUMA configurations conflicts > with lower layer > > On 02/01/2020 12:47, Zengtao (B) wrote: > >> > >> As I said, wrong configurations need to be detected when generating > >> DT/ACPI if possible. The above will print warning on systems with > NUMA > >> within package. > >> > >> NUMA: 0-7, 8-15 > >> core_siblings: 0-15 > >> > >> The above is the example where the die has 16 CPUs and 2 NUMA > nodes > >> within a package, your change throws error to the above config which is > >> wrong. > >> > > From your example, the core 7 and core 8 has got different LLC but the > same Low > > Level cache? > > AFAIA what matters here is memory controllers, less so LLCs. Cores within > a single die could have private LLCs and separate memory controllers, or > shared LLC and separate memory controllers. > > > From schedule view of point, lower level sched domain should be a subset > of higher > > Level sched domain. > > > > Right, and that is checked when you have sched_debug on the cmdline > (or write 1 to /sys/kernel/debug/sched_debug & regenerate the sched > domains) >
No, here I think you don't get my issue, please try to understand my example First:.
************************************* NUMA: 0-2, 3-7 core_siblings: 0-3, 4-7 ************************************* When we are building the sched domain, per the current code: (1) For core 3 MC sched domain fallbacks to 3~7 DIE sched domain is 3~7 (2) For core 4: MC sched domain is 4~7 DIE sched domain is 3~7
When we are build sched groups for the MC level: (1). core3's sched groups chain is built like as: 3->4->5->6->7->3 (2). core4's sched groups chain is built like as: 4->5->6->7->4 so after (2), core3's sched groups is overlapped, and it's not a chain any more. In the afterwards usecase of core3's sched groups, deadloop happens.
And it's difficult for the scheduler to find out such errors, that is why I think a warning is necessary here.
> Now, I don't know how this plays out for the numa-in-package topologies > like > the one suggested by Sudeep. x86 and AMD had to play some games to > get > numa-in-package topologies working, see e.g. > > cebf15eb09a2 ("x86, sched: Add new topology for multi-NUMA-node > CPUs") > > perhaps you need to "lie" here and ensure sub-NUMA sched domain levels > are > a subset of the NUMA levels, i.e. lie for core_siblings. I might go and > play with this to see how it behaves.
| |