lkml.org 
[lkml]   [2010]   [Oct]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: [RFC tg_shares_up improvements - v1 00/12] [RFC tg_shares_up - v1 00/12] Reducing cost of tg->shares distribution
    On Wed, Oct 20, 2010 at 11:36 PM, Paul Turner <pjt@google.com> wrote:
    > On Sat, Oct 16, 2010 at 12:46 PM, Peter Zijlstra <peterz@infradead.org> wrote:
    >> On Fri, 2010-10-15 at 21:43 -0700, pjt@google.com wrote:
    >>> Hi all,
    >>>
    >>> Peter previously posted a patchset that attempted to improve the problem of
    >>> task_group share distribution.  This is something that has been a long-time
    >>> pain point for group scheduling.  The existing algorithm considers
    >>> distributions on a per-cpu-per-domain basis and carries a fairly high update
    >>> overhead, especially on larger machines.
    >>>
    >>> I was previously looking at improving this using Fenwick trees to allow a
    >>> single sum without the exorbitant cost but then Peter's idea above was better :).
    >>>
    >>> The kernel is that by monitoring the average contribution to load on a
    >>> per-cpu-per-taskgroup basis we can distribute the weight for which we are
    >>> expected to consume.
    >>>
    >>> This set extends the original posting with a focus on increased fairness and
    >>> reduced convergence (to true average) time.  In particular the case of large
    >>> over-commit in the case of a distributed wake-up is a concern which is now
    >>> fairly well addressed.
    >>>
    >>> Obviously everything's experimental but it should be stable/fair.
    >>
    >> I like what you've done with it, my only worry is 10/12 where you allow
    >> for extra updates to the global state -- I think they should be fairly
    >> limited in number, and I can see the need for the update if we get too
    >> far out of whack, but it is something to look at while testing this
    >> stuff.
    >>
    >
    > So my original answer here was to only update when there was load and
    > it was > n% delta which stops 1 thread waking up and sleeping from
    > thrashing it, but the 2 thread case is just as obviously broken for
    > any n.  It needs a rate limit but I'm sort of loathe to introduce
    > _another_ set of timestamps.  I don't suppose there's much harm in
    > doing so though and I don't think it's going to be clean to overload
    > one of the existing ones so perhaps another counter is the answer.
    >
    > I'll make sure this is addressed in v2.

    Ahh -- I remember my original reasoning here now:

    These global updates are rate limited since they only occur when we
    fold the averaging period thus are limited to occur at a rate of
    period/2 (e.g. each time we fold).

    One concern is we could get a local "storm" since several cgroups
    could trip this limit at the same time.. but it's also obviously not
    going to be any worse than an update_shares_cpu().

    While there may be a concern on *giant* cpu systems, I think we
    already have that problem in both the original and (to a lesser
    extent) current approaches.

    >
    >>> TODO:
    >>> - Validate any RT interaction
    >>
    >> I don't think there's anything to worry about there, the only
    >> interaction which there is between this and the rt scheduling classes is
    >> the initial sharing of the load-avg window, but you 'cure' that in 7/12.
    >>
    >> (I think that sysctl wants a _us postfix someplace and we thus want some
    >> NSEC_PER_USEC multiplication in there).
    >>
    >
    > Yes -- updated, thanks.
    >
    >>> - Continue collecting/analyzing performance and fairness data
    >>
    >> Yes please ;-), I'll try and run this on some machines as well.
    >>
    >>> - Should the shares period just be the sched_latency?
    >>
    >> Interesting idea.. lets keep it a separate sysctl for now for easy
    >> tuning, if things settle down and we're still good in that range we can
    >> consider merging them.
    >>
    >>
    >>
    >
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2010-10-22 02:17    [W:0.030 / U:30.188 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site