lkml.org 
[lkml]   [2019]   [Apr]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [RFC PATCH v2 00/17] Core scheduling v2
On Thu, Apr 25, 2019 at 08:53:43PM +0200 Ingo Molnar wrote:
> Interesting. This strongly suggests sub-optimal SMT-scheduling in the
> non-saturated HT case, i.e. a scheduler balancing bug.
>
> As long as loads are clearly below the physical cores count (which they
> are in the early phases of your table) the scheduler should spread tasks
> without overlapping two tasks on the same core.
>
> Clearly it doesn't.
>

That's especially true if there are cgroups with different numbers of
tasks in them involved.

Here's an example showing the average number of tasks on each of the 4 numa
nodes during a test run. 20 cpus per node. There are 78 threads total, 76
for lu and 2 stress cpu hogs. So fewer than the 80 CPUs on the box. The GROUP
test has the two stresses and lu in distinct cgroups. The NORMAL test has them
all in one. This is from 5.0-rc3+, but the version doesn't matter. It's
reproducible on any kernel. SMT is on, but that also doesn't matter here.

The first two lines show where the stress jobs ran and the second show where
the 76 threads of lu ran.

GROUP_1.stress.ps.numa.hist Average 1.00 1.00
NORMAL_1.stress.ps.numa.hist Average 0.00 1.10 0.90

lu.C.x_76_GROUP_1.ps.numa.hist Average 10.97 11.78 26.28 26.97
lu.C.x_76_NORMAL_1.ps.numa.hist Average 19.70 18.70 17.80 19.80

The NORMAL test is evenly balanced across the 20 cpus per numa node. There
is between a 4x and 10x performance hit to the lu benchmark between group
and normal in any of these test runs. In this particular case it was 10x:

============76_GROUP========Mop/s===================================
min q1 median q3 max
3776.51 3776.51 3776.51 3776.51 3776.51
============76_GROUP========time====================================
min q1 median q3 max
539.92 539.92 539.92 539.92 539.92
============76_NORMAL========Mop/s===================================
min q1 median q3 max
39386 39386 39386 39386 39386
============76_NORMAL========time====================================
min q1 median q3 max
51.77 51.77 51.77 51.77 51.77


This a bit off topic, but since balancing bugs was mentioned and I've been
trying to track this down for a while (and learning the scheduler code in
the process) I figured I'd just throw it out there :)


Cheers,
Phil

--

\
 
 \ /
  Last update: 2019-04-26 16:15    [W:0.131 / U:2.216 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site