lkml.org 
[lkml]   [2011]   [Mar]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [GIT PULL tip:x86/mm]
    On Thu, 24 Feb 2011, Yinghai Lu wrote:

    > DavidR reported that x86/mm broke his numa emulation with 128M etc.
    >
    > So wonder if that would hold you to push whole tip/x86/mm to Linus for .39
    > or need to rebase it while taking the tip/x86/numa-emulation-unify out.
    >

    Ok, so 1f565a896ee1 (x86-64, NUMA: Fix size of numa_distance array) fixes
    the boot failure when using numa=fake, but there's still another issue
    that was introduced with regard to emulated distances between fake nodes
    sitting hardware using a SLIT.

    This is important because we want to ensure that the physical topoloy of
    the machine is still represented in an emulated environment to
    appropriately describe the expected latencies between the nodes. It also
    allows users who are using numa=fake purely as a debugging tool to test
    more interesting configurations and benchmark memory accesses between
    emulated nodes as though they were real.

    For example, on my four-node system with a custom SLIT, this is the
    distance when booting without numa=fake:

    $ cat /sys/devices/system/node/node*/distance
    10 20 20 30
    20 10 20 20
    20 20 10 20
    30 20 20 10

    These physical nodes are all symmetric in size.

    With numa=fake=16, we expect to see the fake nodes interleaved (as the
    default) over the set of physical nodes. This would suggest distance
    files for these nodes to be:

    10 20 20 30 10 20 20 30 10 20 20 30 10 20 20 30
    20 20 10 20 20 20 10 20 20 20 10 20 20 20 10 20
    30 20 20 10 30 20 20 10 30 20 20 10 30 20 20 10
    10 20 20 30 10 20 20 30 10 20 20 30 10 20 20 30
    20 10 20 20 20 10 20 20 20 10 20 20 20 10 20 20
    20 20 10 20 20 20 10 20 20 20 10 20 20 20 10 20
    30 20 20 10 30 20 20 10 30 20 20 10 30 20 20 10
    20 10 20 20 20 10 20 20 20 10 20 20 20 10 20 20
    20 20 10 20 20 20 10 20 20 20 10 20 20 20 10 20
    30 20 20 10 30 20 20 10 30 20 20 10 30 20 20 10
    10 20 20 30 10 20 20 30 10 20 20 30 10 20 20 30
    20 10 20 20 20 10 20 20 20 10 20 20 20 10 20 20
    20 20 10 20 20 20 10 20 20 20 10 20 20 20 10 20
    30 20 20 10 30 20 20 10 30 20 20 10 30 20 20 10
    10 20 20 30 10 20 20 30 10 20 20 30 10 20 20 30
    20 10 20 20 20 10 20 20 20 10 20 20 20 10 20 20

    (And that is what we see with 2.6.37.)

    However, x86/mm describes these distances differently:

    node0/distance:10 20 20 20 10 20 20 20 10 20 20 20 10 20 20 20
    node1/distance:10 10 20 20 10 20 20 20 10 20 20 20 10 20 20 20
    node2/distance:10 20 10 20 10 20 20 20 10 20 20 20 10 20 20 20
    node3/distance:10 20 20 10 10 20 20 20 10 20 20 20 10 20 20 20
    node4/distance:10 20 20 20 10 20 20 20 10 20 20 20 10 20 20 20
    node5/distance:10 20 20 20 10 10 20 20 10 20 20 20 10 20 20 20
    node6/distance:10 20 20 20 10 20 10 20 10 20 20 20 10 20 20 20
    node7/distance:10 20 20 20 10 20 20 10 10 20 20 20 10 20 20 20
    node8/distance:10 20 20 20 10 20 20 20 10 20 20 20 10 20 20 20
    node9/distance:10 20 20 20 10 20 20 20 10 10 20 20 10 20 20 20
    node10/distance:10 20 20 20 10 20 20 20 10 20 10 20 10 20 20 20
    node11/distance:10 20 20 20 10 20 20 20 10 20 20 10 10 20 20 20
    node12/distance:10 20 20 20 10 20 20 20 10 20 20 20 10 20 20 20
    node13/distance:10 20 20 20 10 20 20 20 10 20 20 20 10 10 20 20
    node14/distance:10 20 20 20 10 20 20 20 10 20 20 20 10 20 10 20
    node15/distance:10 20 20 20 10 20 20 20 10 20 20 20 10 20 20 10

    It looks as though the emulation changes sitting in x86/mm have dropped
    the SLIT and are merely describing the emulated nodes as either having
    physical affinity or not.


    \
     
     \ /
      Last update: 2011-03-01 18:21    [W:0.113 / U:28.868 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site