lkml.org 
[lkml]   [2013]   [Oct]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH part2 v2 0/8] Arrange hotpluggable memory as ZONE_MOVABLE
    Hello, Yinghai.

    On Mon, Oct 14, 2013 at 12:34:49PM -0700, Yinghai Lu wrote:
    > The points for parsing SRAT early instead of Yanfei/Tang v7:
    >
    > 1. We just reached one unified path to setup page tables for 32bit,
    > 64bit and xen or non xen after several years. We should not have add
    > another path for system
    > that support hotplug.

    The separate code path we're talking about is tiny. It's just an
    extra function for page table allocation and another for memblock
    allocation which is symmetric to the existing one. Sure, there are
    benefits to not diverging code paths but these are fairly trivial in
    terms of maintenance overhead and test coverage.

    > 2. also we should avoid adding "movable_nodes" command line.

    Can we? What about the pgdat? We're allocating them off-node with
    movable_nodes which can't be the default behavior.

    > 3. debug mapping 4k, and it is working all the way, why breaking it even for
    > memory hotplug path?

    If it comes for free, sure, no reason to break it. On the other hand,
    if maintaining it fully with a niche feature costs overhead, it's
    somethinig to be traded-off. It's not like using 4k page mapping with
    bottom-up allocation will be immediately broken either. It might
    affect devices which can't DMA to higher addresses on gigantic
    machines under debug configs. It's quite a corner case.

    > 4. numa_meminfo now is static structure.
    > we have no reason that we can not parse SRAT etc to fill that struct.

    Sure, there's no reason we can't. The whole point is that the
    benefits arent' strong enough. We don't do things just because we
    can.

    > 5. for device tree, i assume that we could do same like srat parsing to find out
    > numa to fill the numa_meminfo early. or with help of BRK.

    Digesting device tree involves a lot more complexity. The whole
    reason why things like SRAT are broken into tables in the first place.
    We'll be basically pulling in huge chunk of ACPICA into early boot.
    Again, justfications. The *only* thing which may benefit from that
    are debug setups. We'll have to pull in a lot of complexity before
    page table setup and modify page table allocation to be
    memory-device-specific just for debug configs, which is not a good
    trade-off. Benefit / cost ratio doesn't make any sense.

    > 6. in the long run, We should rework our NUMA booting:
    > a. boot system with boot numa nodes early only.
    > b. in later init stage or user space, init other nodes
    > RAM/CPU/PCI...in parallel.
    > that will reduce boot time for 8 sockets/32 sockets dramatically.
    >
    > We will need to parse srat table early so could avoid init memory for
    > non-boot nodes.

    Among the six you listed, this one sounds somewhat valid but still
    assuming huge page, what difference does it make? We're just talking
    about page table alloc / init and ACPI init. If you wanna speed up
    huge NUMA machine booting and chop down memory init per-NUMA, sure,
    move those pieces to later stages. You can init the amount necessary
    during early boot and then bring up the rest later on. I don't see
    why that'd require parsing SRAT. In fact, I think there'll be more
    cases where you want to actively ignore NUMA mapping during early
    boot. What if the system maps low memory to a non-boot numa node?

    Optimizing NUMA boot just requires moving the heavy lifting to
    appropriate NUMA nodes. It doesn't require that early boot phase
    should strictly follow NUMA node boundaries.

    Thanks.

    --
    tejun


    \
     
     \ /
      Last update: 2013-10-15 03:21    [W:3.958 / U:0.012 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site