Messages in this thread | | | Date | Wed, 8 Jan 2020 18:03:17 +0000 | From | Mel Gorman <> | Subject | Re: [PATCH] sched, fair: Allow a small degree of load imbalance between SD_NUMA domains v2 |
| |
On Wed, Jan 08, 2020 at 05:46:57PM +0100, Vincent Guittot wrote: > > Allowing just 1 extra task would work for netperf in some cases except when > > softirq is involved. It would partially work for IO on ext4 as it's only > > communicating with one journal thread but a bit more borderline for XFS > > due to workqueue usage. XFS is not a massive concern in this context as > > the workqueue is close to the IO issuer and short-lived so I don't think > > it would crop up much for load balancing unlike ext4 where jbd2 could be > > very active. > > > > If v4 of the patch fails to meet approval then I'll try a patch that > > My main concern with v4 was the mismatch between the computed value and the goal to not overload the LLCs >
Fair enough.
> > allows a hard-coded imbalance of 2 tasks (one communicating task and > > If there is no good way to compute the allowed imbalance, a hard coded > value of 2 is probably simple value to start with
Indeed.
> > > one kthread) regardless of NUMA domain span up to 50% of utilisation > > Are you sure that it's necessary ? This degree of imbalance already applies only if the group has spare capacity > > something like > > + /* Consider allowing a small imbalance between NUMA groups */ > + if (env->sd->flags & SD_NUMA) { > + > + /* > + * Until we found a good way to compute an acceptable > + * degree of imbalance linked to the system topology > + * and that will not impatc mem bandwith and latency, > + * let start with a fixed small value. > + */ > + imbalance_adj = 2; > + > + /* > + * Ignore small imbalances when the busiest group has > + * low utilisation. > + */ > + env->imbalance -= min(env->imbalance, imbalance_adj); > + } >
This is more or less what I had in mind with the exception that the "low utilisation" part of the comment would go away. The 50% utilisation may be unnecessary and was based simply on the idea that at that point memory bandwidth, HT considerations or both would also be dominating factors. I can leave out the check and add it in as a separate patch if proven to be necessary.
-- Mel Gorman SUSE Labs
| |