Messages in this thread | | | Date | Fri, 8 Jun 2018 10:24:51 +0100 | From | Mel Gorman <> | Subject | Re: [4.17 regression] Performance drop on kernel-4.17 visible on Stream, Linpack and NAS parallel benchmarks |
| |
On Fri, Jun 08, 2018 at 10:49:03AM +0200, Jirka Hladky wrote: > Hi Mel, > > automatic NUMA balancing doesn't run long enough to migrate all the > > memory. That would definitely be the case for STREAM. > > This could explain the behavior we observe. stream is running ~20 seconds > at the moment. I can easily change the runtime by changing the number of > iterations. What is the time period when you expect the memory to be fully > migrated? >
Unknown and unknowable. It depends entirely on the reference pattern of the different threads. If they are fully parallelised with private buffers that are page-aligned then I expect it to be quick (to pass the 2-reference filter). If threads are sharing data on a 4K (base page case) or 2M boundary (THP enabled) then it may take longer as two or more threads will disagree on what the appropriate placement for a page is.
> I have now checked numastat logs and after 15 seconds I see roughly 80MiB > out of 200MiB of the allocated memory migrated for each of 10 processes > which have changed the NUMA CPU node after started. This is on 2 > socket Gold 6126 CPU @ 2.60GHz server with DDR4 2666 MHz. That's 800 MiB of > memory migrated in 15 seconds which is results in the average migration > rate of 50MiB/s - is this an expected value? >
I expect that to be far short of the capabilities of the machine. Again, migrations can be delayed indefinitely if threads have buffers that are not page-aligned (4K or 2M depending).
-- Mel Gorman SUSE Labs
| |