lkml.org 
[lkml]   [2020]   [Jul]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [RFC PROPOSAL] memcg: per-memcg user space reclaim interface
    On Fri, Jul 03, 2020 at 09:27:19AM -0700, Shakeel Butt wrote:
    > On Fri, Jul 3, 2020 at 8:50 AM Roman Gushchin <guro@fb.com> wrote:
    > >
    > > On Fri, Jul 03, 2020 at 07:23:14AM -0700, Shakeel Butt wrote:
    > > > On Thu, Jul 2, 2020 at 11:35 PM Michal Hocko <mhocko@kernel.org> wrote:
    > > > >
    > > > > On Thu 02-07-20 08:22:22, Shakeel Butt wrote:
    > > > > [...]
    > > > > > Interface options:
    > > > > > ------------------
    > > > > >
    > > > > > 1) memcg interface e.g. 'echo 10M > memory.reclaim'
    > > > > >
    > > > > > + simple
    > > > > > + can be extended to target specific type of memory (anon, file, kmem).
    > > > > > - most probably restricted to cgroup v2.
    > > > > >
    > > > > > 2) fadvise(PAGEOUT) on cgroup_dir_fd
    > > > > >
    > > > > > + more general and applicable to other FSes (actually we are using
    > > > > > something similar for tmpfs).
    > > > > > + can be extended in future to just age the LRUs instead of reclaim or
    > > > > > some new use cases.
    > > > >
    > > > > Could you explain why memory.high as an interface to trigger pro-active
    > > > > memory reclaim is not sufficient. Also memory.low limit to protect
    > > > > latency sensitve workloads?
    > >
    > > I initially liked the proposal, but after some thoughts I've realized
    > > that I don't know a good use case where memory.high is less useful.
    > > Shakeel, what's the typical use case you thinking of?
    > > Who and how will use the new interface?
    > >
    > > >
    > > > Yes, we can use memory.high to trigger [proactive] reclaim in a memcg
    > > > but note that it can also introduce stalls in the application running
    > > > in that memcg. Let's suppose the memory.current of a memcg is 100MiB
    > > > and we want to reclaim 20MiB from it, we can set the memory.high to
    > > > 80MiB but any allocation attempt from the application running in that
    > > > memcg can get stalled/throttled. I want the functionality of the
    > > > reclaim without potential stalls.
    > >
    > > But reclaiming some pagecache/swapping out anon pages can always
    > > generate some stalls caused by pagefaults, no?
    > >
    >
    > Thanks for looking into the proposal. Let me answer both of your
    > questions together. I have added the two use-cases but let me explain
    > the proactive reclaim a bit more as we actually use that in our
    > production.
    >
    > We have defined tolerable refault rates for the applications based on
    > their type (latency sensitive or not). Proactive reclaim is triggered
    > in the application based on their current refault rates and usage. If
    > the current refault rate exceeds the tolerable refault rate then
    > stop/slowdown the proactive reclaim.
    >
    > For the second question, yes, each individual refault can induce the
    > stall as well but we have more control on that stall as compared to
    > stalls due to reclaim. For us almost all the reclaimable memory is
    > anon and we use compression based swap, so, the cost of each refault
    > is fixed and a couple of microseconds.
    >
    > I think the next question is what about the refaults from disk or
    > source with highly variable cost. Usually the latency sensitive
    > applications remove such uncertainty by mlocking the pages backed by
    > such backends (e.g. mlocking the executable) or at least that is the
    > case for us.

    Got it.

    It feels like you're suggesting something similar to memory.high with
    something similar to a different gfp flags. In other words, the
    difference is only which pages can be reclaimed and which not. I don't
    have a definitive answer here, but I wonder if we can somehow
    generalize the existing interface? E.g. if the problem is with artificially
    induced delays, we can have a config option/sysctl/sysfs knob/something else
    which would disable it. Otherwise we risk ending up with many different kinds
    of soft memory limits.

    Thanks!

    \
     
     \ /
      Last update: 2020-07-06 23:40    [W:3.549 / U:0.020 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site