lkml.org 
[lkml]   [2012]   [Feb]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [RFD] cgroup: about multiple hierarchies
On Tue, Feb 21, 2012 at 01:19:38PM -0800, Tejun Heo wrote:
> Hello, guys.
>
> I've been thinking about multiple hierarchy support in cgroup for a
> while, especially after Frederic's pending task counter patchset.
> This is a write up of what I've been thinking. I don't know what to
> do yet and simply continuing the current situation definitely is an
> option, so please read on and throw in your 20 Won (or whatever amount
> in whatever currency you want).
>
> * The problems.
>
> The support for multiple process hierarchies always struck me as
> rather strange. If you forget about the current cgroup controllers
> and their implementations, the *only* reason to support multiple
> hierarchies is if you want to apply resource limits based on different
> orthogonal categorizations.
>
> Documentation/cgroups.txt seems to be written with this consideration
> on mind. It's giving an example of applying limits accoring to two
> orthogonal categorizations - user groups (profressors, students...)
> and applications (WWW, NFS...). While it may sound like a valid use
> case, I'm very skeptical how useful or common mixing such orthogonal
> categorizations in a single setup would be.
>
> If support for multiple hierarchies comes for free, at least in terms
> of features, maybe it can be better but of course it isn't so. Any
> given cgroup subsystem (or controller) can only be applied to a single
> hierarchy, which makes sense for a lot of things - what would two
> different limits on the same resource from different hierarchies mean?
> But, there also are things which can be used and useful in all
> hierarchies - e.g. cgroup freezer and task counter.
>
> While the current cgroup implementation and conventions can probably
> allow admins and engineers to tailor cgroup configuration for a
> specific setup, it is very difficult to use in generic and automated
> way. I mean, who owns the freezer or task counter? If they're
> mounted on their own hierarchies, how should they be structured?
> Should the different hierarchies be structured such that they are
> projections of one unified hierarchy so that those generic mechanisms
> can be applied uniformly? If so, why do we need multiple hierarchies
> at all?
>
> A related limitation is that as different subsystems don't know which
> hierarchies they'll end up on, they can't cooperate. Wouldn't it make
> more sense if task counter is a separate thing watching the resources
> and triggers different actions as conifgured - be it failing forks or
> freezing?

For this particular example, I think we'd better have a file in which
a task can poll and get woken up when the task limit has been reached.
Then that task can decide to freeze or whatever.

>
> And yet another oddity is how cgroup handles nested cgroups - some
> care about nesting but others just treat both internal and leaf nodes
> equally. They don't care about the topology at all. This, too, can
> be fine if you approach things subsys by subsys and use them in
> different ways but if you try to combine them in generic way you get
> sucked into the lala land of whatevers.
>
> The following is a "best practices" document on using cgroups.
>
> http://www.freedesktop.org/wiki/Software/systemd/PaxControlGroups
>
> To me, it seems to demonstrate the rather ugly situation that the
> current cgroup is providing. Everyone should tip-toe around cgroup
> hierarchies and nobody has full knowledge or control over them.
> e.g. base system management (e.g. systemd) can't use freezer or task
> counter as someone else might want to use it for different hierarchy
> layout.
>
> It seems to me that cgroup interface is too complicated and inflexible
> at the same time to be useful in generic manner. Sure, it can be
> useful for setups individually crafted by engineers and admins to
> match specific sites or applications but as soon as you try to do
> something automatic and generic with it, there just are too many
> different scenarios and limitations to consider.
>
>
> * So, what to do?
>
> Heh, I don't know. IIRC, last year at LinuxCon Japan, I heard
> Christoph saying that the biggest problem w/ cgroup was that it was
> building completely separate hierarchies out of the traditional
> process hierarchies. After thinking about this stuff for a while, I
> fully agree with him. I think this whole thing should have been a
> layer over the process tree like sessions or program groups.
>
> Unfortunately, that ship sailed long ago and we gotta make do with
> what we have on our collective hands. Here are some paths that we can
> take.
>
> 1. We're screwed anyway. Just don't worry about it and continue down
> on this path. Can't get much worse, right?
>
> This approach has the apparent advantage of not having to do
> anything and is probably most likely to be taken. This isn't ideal
> but hey nothing is. :P

Thing is we have an ABI now and it has been there for a while now. Aren't
we stuck with it? I'm no big fan of that multiple hierarchies thing either
but now I fear we have to support it.

>
> 2. Make it more flexible (and likely more complex, unfortunately).
> Allow the utility type subsystems to be used in multiple
> hierarchies. The easiest and probably dirtiest way to achieve that
> would be embedding them into cgroup core.
>
> Thinking about doing this depresses me and it's not like I have a
> cheerful personality to begin with. :(

Another solution is to support a class of multi-bindable subsystems as in
this old patch from Paul:

https://lkml.org/lkml/2009/7/1/578

It sounds to me more healthy to iterate only over subsystems in fork/exit.
We probably don't want to add a new iteration over cgroups themselves
on these fast path.


\
 
 \ /
  Last update: 2012-02-22 16:47    [W:0.171 / U:10.692 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site