lkml.org 
[lkml]   [2023]   [Oct]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH v8 8/8] x86/resctrl: Update documentation with Sub-NUMA cluster changes
    Date
    With Sub-NUMA Cluster mode enabled the scope of monitoring resources is
    per-NODE instead of per-L3 cache. Suffixes of directories with "L3" in
    their name refer to Sub-NUMA nodes instead of L3 cache ids.

    Users should be aware that SNC mode also affects the amount of L3 cache
    available for allocation within each SNC node.

    Signed-off-by: Tony Luck <tony.luck@intel.com>

    ---

    Changes since v5:

    Added addtional details about challenges tracking tasks when SNC
    mode is enabled.

    ---
    diff --git a/Documentation/arch/x86/resctrl.rst b/Documentation/arch/x86/resctrl.rst
    index cb05d90111b4..222c507089a5 100644
    --- a/Documentation/arch/x86/resctrl.rst
    +++ b/Documentation/arch/x86/resctrl.rst
    @@ -345,9 +345,9 @@ When control is enabled all CTRL_MON groups will also contain:
    When monitoring is enabled all MON groups will also contain:

    "mon_data":
    - This contains a set of files organized by L3 domain and by
    - RDT event. E.g. on a system with two L3 domains there will
    - be subdirectories "mon_L3_00" and "mon_L3_01". Each of these
    + This contains a set of files organized by L3 domain or by NUMA
    + node (depending on whether Sub-NUMA Cluster (SNC) mode is disabled
    + or enabled respectively) and by RDT event. Each of these
    directories have one file per event (e.g. "llc_occupancy",
    "mbm_total_bytes", and "mbm_local_bytes"). In a MON group these
    files provide a read out of the current value of the event for
    @@ -452,6 +452,23 @@ and 0xA are not. On a system with a 20-bit mask each bit represents 5%
    of the capacity of the cache. You could partition the cache into four
    equal parts with masks: 0x1f, 0x3e0, 0x7c00, 0xf8000.

    +Notes on Sub-NUMA Cluster mode
    +==============================
    +When SNC mode is enabled Linux may load balance tasks between Sub-NUMA
    +nodes much more readily than between regular NUMA nodes since the CPUs
    +on Sub-NUMA nodes share the same L3 cache and the system may report
    +the NUMA distance between Sub-NUMA nodes with a lower value than used
    +for regular NUMA nodes. Users who do not bind tasks to the CPUs of a
    +specific Sub-NUMA node must read the "llc_occupancy", "mbm_total_bytes",
    +and "mbm_local_bytes" for all Sub-NUMA nodes where the tasks may execute
    +to get the full view of traffic for which the tasks were the source.
    +
    +The cache allocation feature still provides the same number of
    +bits in a mask to control allocation into the L3 cache. But each
    +of those ways has its capacity reduced because the cache is divided
    +between the SNC nodes. The values reported in the resctrl
    +"size" files are adjusted accordingly.
    +
    Memory bandwidth Allocation and monitoring
    ==========================================

    \
     
     \ /
      Last update: 2023-10-03 23:32    [W:4.391 / U:0.044 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site