Messages in this thread | | | Subject | Re: hackbench regression due to commit 9dfc6e68bfe6e | From | "Zhang, Yanmin" <> | Date | Thu, 08 Apr 2010 15:54:50 +0800 |
| |
On Thu, 2010-04-08 at 09:00 +0200, Eric Dumazet wrote: > Le jeudi 08 avril 2010 à 07:39 +0200, Eric Dumazet a écrit : > > I suspect NUMA is completely out of order on current kernel, or my > > Nehalem machine NUMA support is a joke > > > > # numactl --hardware > > available: 2 nodes (0-1) > > node 0 size: 3071 MB > > node 0 free: 2637 MB > > node 1 size: 3062 MB > > node 1 free: 2909 MB > > > > > > # cat try.sh > > hackbench 50 process 5000 > > numactl --cpubind=0 --membind=0 hackbench 25 process 5000 >RES0 & > > numactl --cpubind=1 --membind=1 hackbench 25 process 5000 >RES1 & > > wait > > echo node0 results > > cat RES0 > > echo node1 results > > cat RES1 > > > > numactl --cpubind=0 --membind=1 hackbench 25 process 5000 >RES0_1 & > > numactl --cpubind=1 --membind=0 hackbench 25 process 5000 >RES1_0 & > > wait > > echo node0 on mem1 results > > cat RES0_1 > > echo node1 on mem0 results > > cat RES1_0 > > > > # ./try.sh > > Running with 50*40 (== 2000) tasks. > > Time: 16.865 > > node0 results > > Running with 25*40 (== 1000) tasks. > > Time: 16.767 > > node1 results > > Running with 25*40 (== 1000) tasks. > > Time: 16.564 > > node0 on mem1 results > > Running with 25*40 (== 1000) tasks. > > Time: 16.814 > > node1 on mem0 results > > Running with 25*40 (== 1000) tasks. > > Time: 16.896 > > If run individually, the tests results are more what we would expect > (slow), but if machine runs the two set of process concurrently, each > group runs much faster... If there are 2 nodes in the machine, processes on node 0 will contact MCH of node 1 to access memory of node 1. I suspect the MCH of node 1 might enter a power-saving mode when all the cpus of node 1 are free. So the transactions from MCH 1 to MCH 0 has a larger latency.
> > > # numactl --cpubind=0 --membind=1 hackbench 25 process 5000 > Running with 25*40 (== 1000) tasks. > Time: 21.810 > > # numactl --cpubind=1 --membind=0 hackbench 25 process 5000 > Running with 25*40 (== 1000) tasks. > Time: 20.679 > > # numactl --cpubind=0 --membind=1 hackbench 25 process 5000 >RES0_1 & > [1] 9177 > # numactl --cpubind=1 --membind=0 hackbench 25 process 5000 >RES1_0 & > [2] 9196 > # wait > [1]- Done numactl --cpubind=0 --membind=1 hackbench > 25 process 5000 >RES0_1 > [2]+ Done numactl --cpubind=1 --membind=0 hackbench > 25 process 5000 >RES1_0 > # echo node0 on mem1 results > node0 on mem1 results > # cat RES0_1 > Running with 25*40 (== 1000) tasks. > Time: 13.818 > # echo node1 on mem0 results > node1 on mem0 results > # cat RES1_0 > Running with 25*40 (== 1000) tasks. > Time: 11.633 > > Oh well... > >
-- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |