lkml.org 
[lkml]   [2023]   [May]   [23]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH v3 00/19] x86/resctrl: monitored closid+rmid together, separate arch/fs locking
    Hi all,

    Looking at the changes already applied, and those planned to support
    new architectures, new features, and quirks in specific implementations,
    it is clear to me that the original resctrl file system implementation
    did not provide enough flexibility for all the additions that are
    needed.

    So I've begun musing with 20-20 hindsight on how resctrl could have
    provided better abstract building blocks.

    The concept of a "resource" structure with a list of domains for
    specific instances of that structure on a platform still seems like
    a good building block.

    But sharing those structures across increasingly different implementations
    of the underlying resource is resulting in extra gymnastic efforts to
    make all the new uses co-exist with the old. E.g. the domain structure
    has elements for every type of resource even though each instance is
    linked to just one resource type.

    I had begun this journey with a plan to just allow new features to
    hook into the existing resctrl filesystem with a "driver" registration
    mechanism:

    https://lore.kernel.org/all/20230420220636.53527-1-tony.luck@intel.com/

    But feedback from Reinette that this would be cleaner if drivers created
    new resources, rather than adding a patchwork of callback functions with
    special case "if (type == DRIVER)" sprinkled around made me look into
    a more radical redesign instead of joining in the trend of making the
    smallest set of changes to meet my goals.


    Goals:
    1) User interfaces for existing resource control features should be
    unchanged.

    2) Admin interface should have the same capabilities, but interfaces
    may change. E.g. kernel command line and mount options may be replaced
    by choosing which resource modules to load.

    3) Should be easy to create new modules to handle big differences
    between implementations, or to handle model specific features that
    may not exist in the same form across multiple CPU generations.

    Initial notes:

    Core resctrl filesystem functionality will just be:

    1) Mount/unmount of filesystem. Architecture hook to allocate monitor
    and control IDs for the default group.

    2) Creation/removal/rename of control and monitor directories (with
    call to architecture specific code to allocate/free the control and monitor
    IDs to attach to the directory.

    3) Maintaining the "tasks" file with architecture code to update the
    control and monitor IDs in the task structure.

    4) Maintaining the "cpus" file - similar to "tasks"

    5) Context switch code to update h/w with control/monitor IDs.

    6) CPU hotplug interface to build and maintain domain list for each
    registered resource.

    7) Framework for "schemata" file. Calls to resource specific functions
    to maintain each line in the file.

    8) Resource registration interface for modules to add new resources
    to the list (and remove them on module unload). Modules may add files
    to the info/ hierarchy, and also to each mon_data/ directory and/or
    to each control/control_mon directory.

    9) Note that the core code starts with an empty list of resources.
    System admins must load modules to add support for each resource they
    want to use.


    We'd need a bunch of modules to cover existing x86 functionality. E.g.
    an "L3" one for standard L3 cache allocation, an "L3CDP" one to be used
    instead of the plain "L3" one for code/data priority mode by creating
    a separate resource for each of code & data.

    Logically separate mbm_local, mbm_total, and llc_cache_occupancy modules
    (though could combine the mbm ones because they both need a periodic
    counter read to avoid wraparound). "MB" for memory bandwidth allocation.

    The "mba_MBps" mount option would be replaced with a module that does
    both memory bandwidth allocation and monitoring, with a s/w feedback loop.

    Peter's workaround for the quirks of AMD monitoring could become a
    separate AMD specific module. But minor differences (e.g. contiguous
    cache bitmask Intel requirements) could be handled within a module
    if desired.

    Pseudo-locking would be another case to load a different module to
    set up pseudo-locking and enforce the cache bitmask rules between resctrl
    groups instead of the basic cache allocation one.

    Core resctrl code could handle overlaps between modules that want to
    control the same resource with a "first to load reserves that feature"
    policy.

    Are there additional ARM specific architectural requirements that this
    approach isn't addressing? Could the core functionality be extended to
    make life easier for ARM?

    -Tony

    \
     
     \ /
      Last update: 2023-05-23 19:16    [W:4.410 / U:0.116 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site