lkml.org 
[lkml]   [2022]   [May]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRe: RFC: Memory Tiering Kernel Interfaces (v2)
    Date

    Wei Xu <weixugc@google.com> writes:

    > On Tue, May 24, 2022 at 6:27 AM Aneesh Kumar K.V
    > <aneesh.kumar@linux.ibm.com> wrote:
    >>
    >> Wei Xu <weixugc@google.com> writes:
    >>
    >> > On Wed, May 18, 2022 at 5:00 AM Jonathan Cameron
    >> > <Jonathan.Cameron@huawei.com> wrote:
    >> >>
    >> >> On Wed, 18 May 2022 00:09:48 -0700
    >> >> Wei Xu <weixugc@google.com> wrote:
    >>
    >> ...
    >>
    >> > Nice :)
    >> >>
    >> >> Initially I thought this was over complicated when compared to just leaving space, but
    >> >> after a chat with Hesham just now you have us both convinced that this is an elegant solution.
    >> >>
    >> >> Few corners probably need fleshing out:
    >> >> * Use of an allocator for new tiers. Flat number at startup, or new one on write of unique
    >> >> value to set_memtier perhaps? Also whether to allow drivers to allocate (I think
    >> >> we should).
    >> >> * Multiple tiers with same rank. My assumption is from demotion path point of view you
    >> >> fuse them (treat them as if they were a single tier), but keep them expressed
    >> >> separately in the sysfs interface so that the rank can be changed independently.
    >> >> * Some guidance on what values make sense for given rank default that might be set by
    >> >> a driver. If we have multiple GPU vendors, and someone mixes them in a system we
    >> >> probably don't want the default values they use to result in demotion between them.
    >> >> This might well be a guidance DOC or appropriate set of #define
    >> >
    >> > All of these are good ideas, though I am afraid that these can make
    >> > tier management too complex for what it's worth.
    >> >
    >> > How about an alternative tier numbering scheme that uses major.minor
    >> > device IDs? For simplicity, we can just start with 3 major tiers.
    >> > New tiers can be inserted in-between using minor tier IDs.
    >>
    >>
    >> What drives the creation of a new memory tier here? Jonathan was
    >> suggesting we could do something similar to writing to set_memtier for
    >> creating a new memory tier.
    >>
    >> $ echo "memtier128" > sys/devices/system/node/node1/set_memtier
    >>
    >> But I am wondering whether we should implement that now. If we keep
    >> "rank" concept and detach tier index (memtier0 is the memory tier with
    >> index 0) separate from rank, I assume we have enough flexibility for a
    >> future extension that will allow us to create a memory tier from userspace
    >> and assigning it a rank value that helps the device to be placed before or
    >> after DRAM in demotion order.
    >>
    >> ie, For now we will only have memtier0, memtier1, memtier2. We won't add
    >> dynamic creation of memory tiers and the above memory tiers will have
    >> rank value 0, 1, 2 according with demotion order 0 -> 1 -> 2.
    >
    > Great. So the consensus is to go with the "rank" approach. The above
    > sounds good to me as a starting point.

    The rank approach seems good to me too.

    - Alistair

    >> -aneesh

    \
     
     \ /
      Last update: 2022-05-25 09:51    [W:3.338 / U:0.420 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site