[lkml]   [2012]   [Feb]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: [RFD] cgroup: about multiple hierarchies
    Sorry, forgot to cc hch.  Cc'ing him and quoting whole message.

    On Tue, Feb 21, 2012 at 01:19:38PM -0800, Tejun Heo wrote:
    > Hello, guys.
    > I've been thinking about multiple hierarchy support in cgroup for a
    > while, especially after Frederic's pending task counter patchset.
    > This is a write up of what I've been thinking. I don't know what to
    > do yet and simply continuing the current situation definitely is an
    > option, so please read on and throw in your 20 Won (or whatever amount
    > in whatever currency you want).
    > * The problems.
    > The support for multiple process hierarchies always struck me as
    > rather strange. If you forget about the current cgroup controllers
    > and their implementations, the *only* reason to support multiple
    > hierarchies is if you want to apply resource limits based on different
    > orthogonal categorizations.
    > Documentation/cgroups.txt seems to be written with this consideration
    > on mind. It's giving an example of applying limits accoring to two
    > orthogonal categorizations - user groups (profressors, students...)
    > and applications (WWW, NFS...). While it may sound like a valid use
    > case, I'm very skeptical how useful or common mixing such orthogonal
    > categorizations in a single setup would be.
    > If support for multiple hierarchies comes for free, at least in terms
    > of features, maybe it can be better but of course it isn't so. Any
    > given cgroup subsystem (or controller) can only be applied to a single
    > hierarchy, which makes sense for a lot of things - what would two
    > different limits on the same resource from different hierarchies mean?
    > But, there also are things which can be used and useful in all
    > hierarchies - e.g. cgroup freezer and task counter.
    > While the current cgroup implementation and conventions can probably
    > allow admins and engineers to tailor cgroup configuration for a
    > specific setup, it is very difficult to use in generic and automated
    > way. I mean, who owns the freezer or task counter? If they're
    > mounted on their own hierarchies, how should they be structured?
    > Should the different hierarchies be structured such that they are
    > projections of one unified hierarchy so that those generic mechanisms
    > can be applied uniformly? If so, why do we need multiple hierarchies
    > at all?
    > A related limitation is that as different subsystems don't know which
    > hierarchies they'll end up on, they can't cooperate. Wouldn't it make
    > more sense if task counter is a separate thing watching the resources
    > and triggers different actions as conifgured - be it failing forks or
    > freezing?
    > And yet another oddity is how cgroup handles nested cgroups - some
    > care about nesting but others just treat both internal and leaf nodes
    > equally. They don't care about the topology at all. This, too, can
    > be fine if you approach things subsys by subsys and use them in
    > different ways but if you try to combine them in generic way you get
    > sucked into the lala land of whatevers.
    > The following is a "best practices" document on using cgroups.
    > To me, it seems to demonstrate the rather ugly situation that the
    > current cgroup is providing. Everyone should tip-toe around cgroup
    > hierarchies and nobody has full knowledge or control over them.
    > e.g. base system management (e.g. systemd) can't use freezer or task
    > counter as someone else might want to use it for different hierarchy
    > layout.
    > It seems to me that cgroup interface is too complicated and inflexible
    > at the same time to be useful in generic manner. Sure, it can be
    > useful for setups individually crafted by engineers and admins to
    > match specific sites or applications but as soon as you try to do
    > something automatic and generic with it, there just are too many
    > different scenarios and limitations to consider.
    > * So, what to do?
    > Heh, I don't know. IIRC, last year at LinuxCon Japan, I heard
    > Christoph saying that the biggest problem w/ cgroup was that it was
    > building completely separate hierarchies out of the traditional
    > process hierarchies. After thinking about this stuff for a while, I
    > fully agree with him. I think this whole thing should have been a
    > layer over the process tree like sessions or program groups.
    > Unfortunately, that ship sailed long ago and we gotta make do with
    > what we have on our collective hands. Here are some paths that we can
    > take.
    > 1. We're screwed anyway. Just don't worry about it and continue down
    > on this path. Can't get much worse, right?
    > This approach has the apparent advantage of not having to do
    > anything and is probably most likely to be taken. This isn't ideal
    > but hey nothing is. :P
    > 2. Make it more flexible (and likely more complex, unfortunately).
    > Allow the utility type subsystems to be used in multiple
    > hierarchies. The easiest and probably dirtiest way to achieve that
    > would be embedding them into cgroup core.
    > Thinking about doing this depresses me and it's not like I have a
    > cheerful personality to begin with. :(
    > 3. Head towards single hierarchy with the pie-in-the-sky goal of
    > merging things into process hierarchy in some distant future.
    > The first step would be herding people to use a unified hierarchy
    > (ie. all subsystems mounted on a single cgroup tree) which is
    > controlled by single entity in userland (be it systemd or cgroupd,
    > cgroup-kit or whatever); however, even if we exclude supporting
    > orthogonal categorizations, there are good number of non-trivial
    > hurdles to clear before this can be realized.
    > Most importantly, we would need to clean up how nesting is handled
    > across different subsystems. Handling internal and leaf nodes as
    > equals simply can't work. Membership should be recursive, and for
    > subsystems which can't support proper nesting, the right thing to
    > do would be somehow ensuring that only single node in the path from
    > root to leaf is active for the controller. We may even have to
    > introduce an alternative of operation to support this (yuck).
    > This path would require the most amount of work and we would be
    > excluding a feature - support for multiple orthogonal
    > categorizations - which has been available till now, probably
    > through deprecation process spanning years; however, this at least
    > gives us hope that we may reach sanity in the end, how distant that
    > end may be. Oh, hope. :)
    > So, I mean, I don't know. What do other people think? Is this a
    > unnecessary worry? Are people generally happy with the way things
    > are? Lennart, Kay, what do you guys think?
    > Thanks.
    > --
    > tejun


     \ /
      Last update: 2012-02-21 22:23    [W:0.031 / U:4.156 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site