lkml.org 
[lkml]   [2009]   [Feb]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH] cpuset: various documentation fixes and updates
    Li Zefan wrote:
    > I noticed the old commit 8f5aa26c75b7722e80c0c5c5bb833d41865d7019
    > ("cpusets: update_cpumask documentation fix") is not a complete fix,
    > resulting in inconsistent paragraphs. This patch fixes it and does
    > other fixes and updates:
    >
    > - s/migrate_all_tasks()/migrate_live_tasks()/
    > - describe more cpuset control files
    > - s/cpumask_t/struct cpumask/
    > - document cpu hotplug and change of 'sched_relax_domain_level' may cause
    > domain rebuild
    > - document various ways to query or modify cpusets
    > - the equivalent of "mount -t cpuset" is "mount -t cgroup -o cpuset,noprefix"
    > - fix typos
    >
    > Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
    > ---
    > Documentation/cgroups/cpusets.txt | 65 +++++++++++++++++++++----------------
    > 1 files changed, 37 insertions(+), 28 deletions(-)
    >
    > diff --git a/Documentation/cgroups/cpusets.txt b/Documentation/cgroups/cpusets.txt
    > index 5c86c25..3543706 100644
    > --- a/Documentation/cgroups/cpusets.txt
    > +++ b/Documentation/cgroups/cpusets.txt
    > @@ -142,7 +142,7 @@ into the rest of the kernel, none in performance critical paths:
    > - in fork and exit, to attach and detach a task from its cpuset.
    > - in sched_setaffinity, to mask the requested CPUs by what's
    > allowed in that tasks cpuset.
    > - - in sched.c migrate_all_tasks(), to keep migrating tasks within
    > + - in sched.c migrate_live_tasks(), to keep migrating tasks within
    > the CPUs allowed by their cpuset, if possible.
    > - in the mbind and set_mempolicy system calls, to mask the requested
    > Memory Nodes by what's allowed in that tasks cpuset.
    > @@ -175,6 +175,10 @@ files describing that cpuset:
    > - mem_exclusive flag: is memory placement exclusive?
    > - mem_hardwall flag: is memory allocation hardwalled
    > - memory_pressure: measure of how much paging pressure in cpuset
    > + - memory_spread_page flag: if set, spread page cache evenly on allowed nodes
    > + - memory_spread_slab flag: if set, spread slab cache evenly on allowed nodes
    > + - sched_load_balance flag: if set, load balance within CPUs on that cpuset
    > + - sched_relax_domain_level: the searching range when migrating tasks
    >
    > In addition, the root cpuset only has the following file:
    > - memory_pressure_enabled flag: compute memory_pressure?
    > @@ -252,7 +256,7 @@ is causing.
    >
    > This is useful both on tightly managed systems running a wide mix of
    > submitted jobs, which may choose to terminate or re-prioritize jobs that
    > -are trying to use more memory than allowed on the nodes assigned them,
    > +are trying to use more memory than allowed on the nodes assigned to them,
    > and with tightly coupled, long running, massively parallel scientific
    > computing jobs that will dramatically fail to meet required performance
    > goals if they start to use more memory than allowed to them.
    > @@ -378,7 +382,7 @@ as cpusets and sched_setaffinity.
    > The algorithmic cost of load balancing and its impact on key shared
    > kernel data structures such as the task list increases more than
    > linearly with the number of CPUs being balanced. So the scheduler
    > -has support to partition the systems CPUs into a number of sched
    > +has support to partition the systems CPUs into a number of sched
    > domains such that it only load balances within each sched domain.
    > Each sched domain covers some subset of the CPUs in the system;
    > no two sched domains overlap; some CPUs might not be in any sched
    > @@ -485,17 +489,22 @@ of CPUs allowed to a cpuset having 'sched_load_balance' enabled.
    > The internal kernel cpuset to scheduler interface passes from the
    > cpuset code to the scheduler code a partition of the load balanced
    > CPUs in the system. This partition is a set of subsets (represented
    > -as an array of cpumask_t) of CPUs, pairwise disjoint, that cover all
    > -the CPUs that must be load balanced.
    > -
    > -Whenever the 'sched_load_balance' flag changes, or CPUs come or go
    > -from a cpuset with this flag enabled, or a cpuset with this flag
    > -enabled is removed, the cpuset code builds a new such partition and
    > -passes it to the scheduler sched domain setup code, to have the sched
    > -domains rebuilt as necessary.
    > +as an array of struct cpumask) of CPUs, pairwise disjoint, that cover
    > +all the CPUs that must be load balanced.
    > +
    > +The cpuset code builds a new such partition and passes it to the
    > +scheduler sched domain setup code, to have the sched domains rebuilt
    > +as necessary, whenever:
    > + - the 'sched_load_balance' flag of a cpuset with non-empty CPUs changes,
    > + - or CPUs come or go from a cpuset with this flag enabled,
    > + - or 'sched_relax_domain_level' value of a cpuset with non-empty CPUs
    > + and with this flag enabled changes,
    > + - or a cpuset with non-empty CPUs and with this flag enabled is removed,
    > + - or a cpu is offlined/onlined.
    >
    > This partition exactly defines what sched domains the scheduler should
    > -setup - one sched domain for each element (cpumask_t) in the partition.
    > +setup - one sched domain for each element (struct cpumask) in the
    > +partition.
    >
    > The scheduler remembers the currently active sched domain partitions.
    > When the scheduler routine partition_sched_domains() is invoked from
    > @@ -559,7 +568,7 @@ domain, the largest value among those is used. Be careful, if one
    > requests 0 and others are -1 then 0 is used.
    >
    > Note that modifying this file will have both good and bad effects,
    > -and whether it is acceptable or not will be depend on your situation.
    > +and whether it is acceptable or not depends on your situation.
    > Don't modify this file if you are not sure.
    >
    > If your situation is:
    > @@ -600,19 +609,15 @@ to allocate a page of memory for that task.
    >
    > If a cpuset has its 'cpus' modified, then each task in that cpuset
    > will have its allowed CPU placement changed immediately. Similarly,
    > -if a tasks pid is written to a cpusets 'tasks' file, in either its
    > -current cpuset or another cpuset, then its allowed CPU placement is
    > -changed immediately. If such a task had been bound to some subset
    > -of its cpuset using the sched_setaffinity() call, the task will be
    > -allowed to run on any CPU allowed in its new cpuset, negating the
    > -affect of the prior sched_setaffinity() call.
    > +if a tasks pid is written to another cpusets 'tasks' file, then its

    task's pid cpuset's

    > +allowed CPU placement is changed immediately. If such a task had been
    > +bound to some subset of its cpuset using the sched_setaffinity() call,
    > +the task will be allowed to run on any CPU allowed in its new cpuset,
    > +negating the affect of the prior sched_setaffinity() call.

    effect

    >
    > In summary, the memory placement of a task whose cpuset is changed is
    > updated by the kernel, on the next allocation of a page for that task,
    > -but the processor placement is not updated, until that tasks pid is
    > -rewritten to the 'tasks' file of its cpuset. This is done to avoid
    > -impacting the scheduler code in the kernel with a check for changes
    > -in a tasks processor placement.
    > +and the processor placement is updated immediately.
    >
    > Normally, once a page is allocated (given a physical page
    > of main memory) then that page stays on whatever node it
    > @@ -681,10 +686,14 @@ and then start a subshell 'sh' in that cpuset:
    > # The next line should display '/Charlie'
    > cat /proc/self/cpuset
    >
    > -In the future, a C library interface to cpusets will likely be
    > -available. For now, the only way to query or modify cpusets is
    > -via the cpuset file system, using the various cd, mkdir, echo, cat,
    > -rmdir commands from the shell, or their equivalent from C.
    > +There are ways to query or modify cpusets:
    > + - via the cpuset file system directly, using the various cd, mkdir, echo,
    > + cat, rmdir commands from the shell, or there equivalent from C.

    their

    > + - via the C library libcpuset.
    > + - via the C library libcgroup.
    > + (http://sourceforge.net/proects/libcg/)
    > + - via the python application cset.
    > + (http://developer.novell.com/wiki/index.php/Cpuset)
    >
    > The sched_setaffinity calls can also be done at the shell prompt using
    > SGI's runon or Robert Love's taskset. The mbind and set_mempolicy
    > @@ -756,7 +765,7 @@ mount -t cpuset X /dev/cpuset
    >
    > is equivalent to
    >
    > -mount -t cgroup -ocpuset X /dev/cpuset
    > +mount -t cgroup -ocpuset,noprefix X /dev/cpuset

    I'm used to "-o options_list"... I guess either is OK.

    > echo "/sbin/cpuset_release_agent" > /dev/cpuset/release_agent
    >
    > 2.2 Adding/removing cpus


    --
    ~Randy


    \
     
     \ /
      Last update: 2009-02-19 06:29    [W:2.845 / U:0.140 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site