[lkml]   [2008]   [Jul]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    Subject[RFC] How to handle the rules engine for cgroups

    While development is going on for cgroup and various controllers, we also
    need a facility so that an admin/user can specify the group creation and
    also specify the rules based on which tasks should be placed in respective
    groups. Group creation part will be handled by libcg which is already
    under development. We still need to tackle the issue of how to specify
    the rules and how these rules are enforced (rules engine).

    I have gathered few views, with regards to how rule engine can possibly be
    implemented, I am listing these down.

    Proposal 1
    Let user space daemon hanle all that. Daemon will open a netlink socket
    and receive the notifications for various kernel events. Daemon will
    also parse appropriate admin specified rules config file and place the
    processes in right cgroup based on rules as and when events happen.

    I have written a prototype user space program which does that. Program
    can be found here. Currently it is in very crude shape.

    Various people have raised two main issues with this approach.

    - netlink is not a reliable protocol.
    - Messages can be dropped and one can loose message. That means a
    newly forked process might never go into right group as meant.

    - How to handle delays in rule exectuion?
    - For example, if an "exec" happens and by the time process is moved to
    right group, it might have forked off few more processes or might
    have done quite some amount of memory allocation which will be
    charged to the wring group. Or, newly exec process might get
    killed in existing cgroup because of lack of memory (despite the
    fact that destination cgroup has sufficient memory).

    Proposal 2
    Implement one or more kernel modules which will implement the rule engine.
    User space program can parse the config files and pass it to module.
    Kernel will be patched only on select points to look for the rules (as
    provided by modules). Very minimal code running inside the kernel if there
    are no rules loaded.

    - Rules can become complex and we don't want to handle that complexity in

    - Reliable and precise movement of tasks in right cgroup based on rules.

    Proposal 3
    How about if additional parameters can be passed to system calls and one
    can pass destination cgroup as additional parameter. Probably something
    like sys_indirect proposal. Maybe glibc can act as a wrapper to pass
    additional parameter so that applications don't need any modifications.

    - Looks like sys_indirect interface for passing extra flags was rejected.
    - Requires extra work in glibc which can also involve parsing of rule
    files. :-(

    Proposal 4
    Some vauge thoughts are there regarding how about kind of freezing the
    process or thread upon fork, exec and unfreeze it once the thread has been
    placed in right cgroup.

    - Requires reliable netlink protocol otherwise there is a possibility that
    a task never gets unfrozen.
    - On what basis does one freeze a thread. There might not be any rules to
    process for that thread we will unnecessarily delay it.

    Please provide your inputs regarding what's the best way to handle the
    rules engine.

    To me, letting the rules live in separate module/modules seems to be a
    reasonable way to move forward which will provide reliable and timely
    execution of rules and by making it modular, we can remove most of the
    complexity from core kernel code.


     \ /
      Last update: 2008-07-01 21:15    [W:0.043 / U:8.356 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site