lkml.org 
[lkml]   [2017]   [Nov]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: [PATCH 0/6] Cache coherent device memory (CDM) with HMM v5
    >> You may think it as a CCIX device or CAPI device.
    >> The requirement is eliminate any extra copy.
    >> A typical usecase/requirement is malloc() and madvise() allocate from
    >> device memory, then CPU write data to device memory directly and
    >> trigger device to read the data/do calculation.
    >
    > I suggest you rely on the device driver userspace API to do a migration after malloc
    > then. Something like:
    > ptr = malloc(size);
    > my_device_migrate(ptr, size);
    >
    > Which would call an ioctl of the device driver which itself would migrate memory or
    > allocate device memory for the range if pointer return by malloc is not yet back by
    > any pages.
    >

    So for CCIX, I don't think there is going to be an inline device
    driver that would allocate any memory for you. The expansion memory
    will become part of the system memory as part of the boot process. So,
    if the host DDR is 256GB and the CCIX expansion memory is 4GB, the
    total system mem will be 260GB.

    Assume that the 'mm' is taught to mark/anoint the ZONE_DEVICE(or
    ZONE_XXX) range from 256 to 260 GB. Then, for kmalloc it(mm) won't use
    the ZONE_DEV range. But for a malloc, it will/can use that range.


    > There has been several discussions already about madvise/mbind/set_mempolicy/
    > move_pages and at this time i don't think we want to add or change any of them to
    > understand device memory. My personal opinion is that we first need to have enough

    We will visit these APIs when we are more closer to building exotic
    CCIX devices. And the plan is to present/express the CCIX proximity
    attributes just like a NUMA node-proximity attribute today. That way
    there would be minimal disruptions to the existing OS ecosystem.



    Chetan

    \
     
     \ /
      Last update: 2017-11-16 03:10    [W:4.445 / U:0.060 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site