lkml.org 
[lkml]   [2020]   [Mar]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    Subject[patch 2/3] isolcpus: affine kernel threads to specified cpumask
    This is a kernel enhancement to configure the cpu affinity of kernel threads via kernel boot option isolcpus=no_kthreads,<isolcpus_params>,<cpulist>

    When this option is specified, the cpumask is immediately applied upon
    thread launch. This does not affect kernel threads that specify cpu
    and node.

    This allows CPU isolation (that is not allowing certain threads
    to execute on certain CPUs) without using the isolcpus=domain parameter,
    making it possible to enable load balancing on such CPUs
    during runtime (see kernel-parameters.txt).

    Note-1: this is based off on Wind River's patch at
    https://github.com/starlingx-staging/stx-integ/blob/master/kernel/kernel-std/centos/patches/affine-compute-kernel-threads.patch

    Difference being that this patch is limited to modifying
    kernel thread cpumask: Behaviour of other threads can
    be controlled via cgroups or sched_setaffinity.

    Note-2: Wind River's patch was based off Christoph Lameter's patch at
    https://lwn.net/Articles/565932/ with the only difference being
    the kernel parameter changed from kthread to kthread_cpus.

    Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>

    ---
    Documentation/admin-guide/kernel-parameters.txt | 8 ++++++++
    include/linux/sched/isolation.h | 1 +
    kernel/kthread.c | 6 ++++--
    kernel/sched/isolation.c | 6 ++++++
    4 files changed, 19 insertions(+), 2 deletions(-)

    Index: linux-2.6/Documentation/admin-guide/kernel-parameters.txt
    ===================================================================
    --- linux-2.6.orig/Documentation/admin-guide/kernel-parameters.txt
    +++ linux-2.6/Documentation/admin-guide/kernel-parameters.txt
    @@ -1959,6 +1959,14 @@
    the CPU affinity syscalls or cpuset.
    <cpu number> begins at 0 and the maximum value is
    "number of CPUs in system - 1".
    + When using cpusets, use the isolcpus option "kthread"
    + to avoid creation of kernel threads on isolated CPUs.
    +
    + kthread
    + Adjust the CPU affinity mask of unbound kernel threads to
    + not contain CPUs on the isolated list. This complements
    + the isolation provided by the cpusets mechanism described
    + above and by managed_irq option.

    managed_irq

    Index: linux-2.6/include/linux/sched/isolation.h
    ===================================================================
    --- linux-2.6.orig/include/linux/sched/isolation.h
    +++ linux-2.6/include/linux/sched/isolation.h
    @@ -14,6 +14,7 @@ enum hk_flags {
    HK_FLAG_DOMAIN = (1 << 5),
    HK_FLAG_WQ = (1 << 6),
    HK_FLAG_MANAGED_IRQ = (1 << 7),
    + HK_FLAG_KTHREAD = (1 << 8),
    };

    #ifdef CONFIG_CPU_ISOLATION
    Index: linux-2.6/kernel/kthread.c
    ===================================================================
    --- linux-2.6.orig/kernel/kthread.c
    +++ linux-2.6/kernel/kthread.c
    @@ -23,6 +23,7 @@
    #include <linux/ptrace.h>
    #include <linux/uaccess.h>
    #include <linux/numa.h>
    +#include <linux/sched/isolation.h>
    #include <trace/events/sched.h>

    static DEFINE_SPINLOCK(kthread_create_lock);
    @@ -347,7 +348,8 @@ struct task_struct *__kthread_create_on_
    * The kernel thread should not inherit these properties.
    */
    sched_setscheduler_nocheck(task, SCHED_NORMAL, &param);
    - set_cpus_allowed_ptr(task, cpu_possible_mask);
    + set_cpus_allowed_ptr(task,
    + housekeeping_cpumask(HK_FLAG_KTHREAD));
    }
    kfree(create);
    return task;
    @@ -572,7 +574,7 @@ int kthreadd(void *unused)
    /* Setup a clean context for our children to inherit. */
    set_task_comm(tsk, "kthreadd");
    ignore_signals(tsk);
    - set_cpus_allowed_ptr(tsk, cpu_possible_mask);
    + set_cpus_allowed_ptr(tsk, housekeeping_cpumask(HK_FLAG_KTHREAD));
    set_mems_allowed(node_states[N_MEMORY]);

    current->flags |= PF_NOFREEZE;
    Index: linux-2.6/kernel/sched/isolation.c
    ===================================================================
    --- linux-2.6.orig/kernel/sched/isolation.c
    +++ linux-2.6/kernel/sched/isolation.c
    @@ -169,6 +169,12 @@ static int __init housekeeping_isolcpus_
    continue;
    }

    + if (!strncmp(str, "kthread,", 8)) {
    + str += 8;
    + flags |= HK_FLAG_KTHREAD;
    + continue;
    + }
    +
    pr_warn("isolcpus: Error, unknown flag\n");
    return 0;
    }

    \
     
     \ /
      Last update: 2020-03-28 16:28    [W:4.122 / U:0.628 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site