Messages in this thread | | | From | Srikar Dronamraju <> | Subject | [PATCH v2 0/4] Improve numa load balancing | Date | Tue, 16 Jun 2015 17:25:58 +0530 |
| |
Here is a patchset that improves numa load balancing. It prefers numa over cache during regular balancing. It also takes imbalance_pct into consideration in numa_has_capacity() and provides few fixes to task_numa_migrate().
Here are results from 5 runs of autonuma-benchmark (https://github.com/pholasek/autonuma-benchmark)
KernelVersion: 4.1.0-rc7-tip Testcase: Min Max Avg StdDev elapsed_numa01: 858.85 949.18 915.64 33.06 elapsed_numa02: 23.09 29.89 26.43 2.18 Testcase: Min Max Avg StdDev system_numa01: 1516.72 1855.08 1686.24 113.95 system_numa02: 63.69 79.06 70.35 5.87 Testcase: Min Max Avg StdDev user_numa01: 73284.76 80818.21 78060.88 2773.60 user_numa02: 1690.18 2071.07 1821.64 140.25 Testcase: Min Max Avg StdDev total_numa01: 74801.50 82572.60 79747.12 2875.61 total_numa02: 1753.87 2142.77 1891.99 143.59
KernelVersion: 4.1.0-rc7-tip + patchset
Testcase: Min Max Avg StdDev %Change elapsed_numa01: 718.47 916.32 867.63 75.47 5.24% elapsed_numa02: 19.12 28.79 24.78 3.46 5.73% Testcase: Min Max Avg StdDev %Change system_numa01: 870.48 1493.04 1208.64 202.34 31.99% system_numa02: 41.25 64.56 55.12 7.96 23.59% Testcase: Min Max Avg StdDev %Change user_numa01: 57736.89 74336.14 70092.39 6213.45 10.72% user_numa02: 1478.86 2122.50 1768.01 221.06 2.53% Testcase: Min Max Avg StdDev %Change total_numa01: 58607.40 75628.90 71301.04 6380.18 11.17% total_numa02: 1520.11 2183.44 1823.13 227.16 3.15%
Performance counter stats for 'system wide':
numa01 KernelVersion: 4.1.0-rc7-tip 7,58,804 cs [100.00%] 1,29,961 migrations [100.00%] 7,08,643 faults 3,97,97,92,613 cache-misses
949.188997751 seconds time elapsed
KernelVersion: 4.1.0-rc7-tip + patchset 11,38,993 cs [100.00%] 2,05,490 migrations [100.00%] 8,72,140 faults 3,97,61,99,522 cache-misses
907.583509759 seconds time elapsed
numa02 KernelVersion: 4.1.0-rc7-tip 29,573 cs [100.00%] 4,671 migrations [100.00%] 33,236 faults 12,68,88,021 cache-misses
29.897733416 seconds time elapsed
KernelVersion: 4.1.0-rc7-tip + patchset 23,422 cs [100.00%] 4,306 migrations [100.00%] 23,858 faults 9,98,87,326 cache-misses
22.776749102 seconds time elapsed
# numactl -H available: 4 nodes (0-3) node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 node 0 size: 32425 MB node 0 free: 31352 MB node 1 cpus: 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 node 1 size: 31711 MB node 1 free: 30147 MB node 2 cpus: 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 node 2 size: 30431 MB node 2 free: 29636 MB node 3 cpus: 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 node 3 size: 32219 MB node 3 free: 31053 MB node distances: node 0 1 2 3 0: 10 20 40 40 1: 20 10 40 40 2: 40 40 10 20 3: 40 40 20 10
Srikar Dronamraju (4): sched/tip:Prefer numa hotness over cache hotness sched: Consider imbalance_pct when comparing loads in numa_has_capacity sched: Fix task_numa_migrate to always update preferred node sched: Use correct nid while evaluating task weights
kernel/sched/fair.c | 109 ++++++++++++++---------------------------------- kernel/sched/features.h | 18 +++----- 2 files changed, 37 insertions(+), 90 deletions(-)
-- 1.8.3.1
| |