Messages in this thread | | | Subject | Re: [PATCH] arm64: NUMA: Kconfig: Increase max number of nodes | From | Robin Murphy <> | Date | Thu, 22 Oct 2020 12:21:27 +0100 |
| |
On 2020-10-22 02:07, Vanshi Konda wrote: > On Thu, Oct 22, 2020 at 12:44:15AM +0100, Robin Murphy wrote: >> On 2020-10-21 12:02, Jonathan Cameron wrote: >>> On Wed, 21 Oct 2020 09:43:21 +0530 >>> Anshuman Khandual <anshuman.khandual@arm.com> wrote: >>> >>>> On 10/20/2020 11:39 PM, Valentin Schneider wrote: >>>>> >>>>> Hi, >>>>> >>>>> Nit on the subject: this only increases the default, the max is >>>>> still 2?????. >>>> >>>> Agreed. >>>> >>>>> >>>>> On 20/10/20 18:34, Vanshidhar Konda wrote: >>>>>> The current arm64 max NUMA nodes default to 4. Today's arm64 >>>>>> systems can >>>>>> reach or exceed 16. Increase the number to 64 (matching x86_64). >>>>>> >>>>>> Signed-off-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com> >>>>>> --- >>>>>> arch/arm64/Kconfig | 2 +- >>>>>> 1 file changed, 1 insertion(+), 1 deletion(-) >>>>>> >>>>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig >>>>>> index 893130ce1626..3e69d3c981be 100644 >>>>>> --- a/arch/arm64/Kconfig >>>>>> +++ b/arch/arm64/Kconfig >>>>>> @@ -980,7 +980,7 @@ config NUMA >>>>>> config NODES_SHIFT >>>>>> int "Maximum NUMA Nodes (as a power of 2)" >>>>>> range 1 10 >>>>>> - default "2" >>>>>> + default "6" >>>>> >>>>> This leads to more statically allocated memory for things like node >>>>> to CPU >>>>> maps (see uses of MAX_NUMNODES), but that shouldn't be too much of an >>>>> issue. >>>> >>>> The smaller systems should not be required to waste those memory in >>>> a default case, unless there is a real and available larger system >>>> with those increased nodes. >>>> >>>>> >>>>> AIUI this also directly correlates to how many more page->flags >>>>> bits are >>>>> required: are we sure the max 10 works on any aarch64 platform? I'm >>>> >>>> We will have to test that. Besides 256 (2 ^ 8) is the first threshold >>>> to be crossed here. >>>> >>>>> genuinely asking here, given that I'm mostly a stranger to the mm >>>>> world. The default should be something we're somewhat confident works >>>>> everywhere. >>>> >>>> Agreed. Do we really need to match X86 right now ? Do we really have >>>> systems that has 64 nodes ? We should not increase the default node >>>> value and then try to solve some new problems, when there might not >>>> be any system which could even use that. I would suggest increase >>>> NODES_SHIFT value upto as required by a real and available system. >>> >>> I'm not going to give precise numbers on near future systems but it >>> is public >>> that we ship 8 NUMA node ARM64 systems today. Things will get more >>> interesting as CXL and CCIX enter the market on ARM systems, >>> given chances are every CXL device will look like another NUMA >>> node (CXL spec says they should be presented as such) and you >>> may be able to rack up lots of them. >>> >>> So I'd argue minimum that makes sense today is 16 nodes, but looking >>> forward >>> even a little and 64 is not a great stretch. >>> I'd make the jump to 64 so we can forget about this again for a year >>> or two. >>> People will want to run today's distros on these new machines and we'd >>> rather not have to go around all the distros asking them to carry a >>> patch >>> increasing this count (I assume they are already carrying such a patch >>> due to those 8 node systems) > > To echo Jonathan's statement above we are looking at systems that will > need approximately 64 NUMA nodes over the next 5-6 years - the time for > which an LTS kernel would be maintained. Some of the reason's for > increasing NUMA nodes during this time period include CXL, CCIX and > NVDIMM (like Jonathan pointed out). > > The main argument against increasing the NODES_SHIFT seems to be a > concern that it negatively impacts other ARM64 systems. Could anyone > share what kind of systems we are talking about? For a system that has > NEED_MULTIPLE_NODES set, would the impact be noticeable?
Systems like the ESPRESSObin - sure, sane people aren't trying to run desktops or development environments in 1GB of RAM, but it's not uncommon for them to use a minimal headless install of their favourite generic arm64 distro rather than something more "embedded" like OpenWrt or Armbian. Increasing a generic kernel's memory footprint (and perhaps more importantly, cache footprint) more than necessary is going to have *some* impact.
Robin.
> > Vanshi > >> >> Nit: I doubt any sane distro is going to carry a patch to adjust the >> *default* value of a Kconfig option. They might tune the actual value >> in their config, but, well, isn't that the whole point of configs? ;) >> >> Robin. >> >>> >>> Jonathan >>> >>>> >>>>>> depends on NEED_MULTIPLE_NODES >>>>>> help >>>>>> Specify the maximum number of NUMA Nodes available on the >>>>>> target >>>> >>>> _______________________________________________ >>>> linux-arm-kernel mailing list >>>> linux-arm-kernel@lists.infradead.org >>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel >>> >>> >>> >>> _______________________________________________ >>> linux-arm-kernel mailing list >>> linux-arm-kernel@lists.infradead.org >>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel >>>
| |