lkml.org 
[lkml]   [2007]   [Oct]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [patch 3/3] cpusets: add memory_spread_user option
On Thu, 25 Oct 2007, Paul Jackson wrote:

> I'm figuring that when someone looks at the cpuset flag:
>
> memory_spread_user
>
> they will expect that turning it on will cause user space memory to be
> spread over the nodes of the cpuset. Sure makes sense that it would
> mean that.
>
> But, for the most part, it doesn't. Only tasks that have
> previously called set_mempolicy(MPOL_INTERLEAVE), and only
> after the 'mems' of the cpuset are subsequently changed,
> will have their user memory forced to be spread across the
> cpuset as a result of this flag setting.
>

And also tasks which are attached to a cpuset that has memory_spread_user
enabled and had preexisting MPOL_INTERLEAVE memory policies.
cpuset_attach() rebinds memory policies; if memory_spread_user is set, the
the interleaved nodemask for the task's mempolicy will be set to the
cpuset's mems_allowed.

> Any chance, David, that you could have this flag mean:
>
> Spread user memory allocations over the cpuset,
> period, anytime it is set, regardless of what
> mempolicy calls the task has made and regardless
> of whether or not or when the cpusets 'mems' were
> last changed.
>

This would override the custom mempolicies of all tasks attached to the
cpuset. All tasks would have MPOL_INTERLEAVE memory policies with a
nodemask of the cpuset's mems_allowed and could not be changed.

With my current proposal, tasks receive the full interleaved nodemask of
the cpuset's mems_allowed when they have preexisting MPOL_INTERLEAVE
memory policies and:

- the 'mems' change in a cpuset enabled with memory_spread_user,

- it is attached to a cpuset enabled with memory_spread_user, or

- memory_spread_page is enabled for its cpuset.

We respect the changes that tasks make to their mempolicies with
MPOL_INTERLEAVE after any of those three scenarios.

Your change would force all tasks in a memory_spread_user cpuset to have
MPOL_INTERLEAVE with a nodemask of mems_allowed. That's very easy to
define but is going to require additional cpusets to be created with
duplicate settings (excluding memory_spread_user) if you want different
behavior for its tasks. I won't argue against that.

> Most power, or excessive confusion? Straight forward consistency and
> simple predictability are far more important in almost all cases. The
> usual exception is when you have a serious use case requiring
> something that can only be done in a more obscure fashion.
>

I don't think cpuset files such as memory_spread_{page,slab,user} or
memory_pressure, etc, are completely and accurately descriptive of what
they do anyway. That's why anybody who is going to use cpusets is going
to refer to Documentation/cpusets.txt where the semantics are explicitly
written.

> There is always a price paid for supporting such complexities in an API
> however, the price being increased confusion, frustration, errors and
> bugs on the part of most users of the API.
>

We can certainly code it the way you suggested: memory_spread_user
requires all tasks to be MPOL_INTERLEAVE with a static nodemask of
mems_allowed. If this use-case is so narrow that any sane implementation
would never want several different tasks with different contextualized
interleaved memory policies in a dynamically changing cpuset, that's the
perfect way to code it.

> ... Now most likely you will claim you have such a use case, and when
> I ask for it, I will be frustrated at the lack of compelling detail of
> what is going on in user space - what sorts of users, apps and systems
> involved. Ok, no biggie. If this goes down that path, then perhaps
> at least I need to reconsider the name:
>
> memory_spread_user
>

I don't actually have any use-cases where I want two different
MPOL_INTERLEAVE tasks sharing the same cpuset and only one of them
adjusted on a mems change and the other to remain static unless I
explicitly call set_mempolicy().

It all comes down to the decision of whether we want to permit
set_mempolicy() calls for tasks and respect the nodemask passed in a
memory_spread_user cpuset. If so, we must do it my way where the
set_mempolicy() occurs after attachment or the setting of the flag.
There's just no other time where you can allow them to differ from the
memory_spread_user behavior that the cpuset is configured with.

So if you don't have any issue with a hard and fast rule of requiring
tasks in memory_spread_user cpusets to have MPOL_INTERLEAVE policies with
the nodemask of mems_allowed and not giving them the option of changing
it, I have no objection. I simply coded it so that you could work around
the cpuset flag through the mempolicy interface. I don't have any express
need for it.

David
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2007-10-26 11:27    [W:0.149 / U:0.060 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site