lkml.org 
[lkml]   [2012]   [Feb]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [RFC] [PATCH v5 0/3] fadvise: support POSIX_FADV_NOREUSE
    On Tue, Feb 14, 2012 at 03:22:20PM -0800, Andrew Morton wrote:
    > On Tue, 14 Feb 2012 23:59:22 +0100
    > Andrea Righi <andrea@betterlinux.com> wrote:
    >
    > > On Tue, Feb 14, 2012 at 01:33:37PM -0800, Andrew Morton wrote:
    > > > On Sun, 12 Feb 2012 01:21:35 +0100
    > > > Andrea Righi <andrea@betterlinux.com> wrote:
    > > >
    > > > > The new proposal is to implement POSIX_FADV_NOREUSE as a way to perform a real
    > > > > drop-behind policy where applications can mark certain intervals of a file as
    > > > > FADV_NOREUSE before accessing the data.
    > > >
    > > > I think you and John need to talk to each other, please. The amount of
    > > > duplication here is extraordinary.
    > >
    > > Yes, definitely. I'm currently reviewing and testing the John's patch
    > > set. I was even considering to apply my patch set on top of the John's
    > > patch, or at least propose my tree-based approach to manage the list of
    > > the POSIX_FADV_VOLATILE ranges.
    >
    > Cool.
    >
    > > >
    > > > Both patchsets add fields to the address_space (and hence inode), which
    > > > is significant - we should convince ourselves that we're getting really
    > > > good returns from a feature which does this.
    > > >
    > > >
    > > >
    > > > Regarding the use of fadvise(): I suppose it's a reasonable thing to do
    > > > in the long term - if the feature works well, popular data streaming
    > > > applications will eventually switch over. But I do think we should
    > > > explore interfaces which don't require modification of userspace source
    > > > code. Because there will always be unconverted applications, and the
    > > > feature becomes available immediately.
    > > >
    > > > One such interface would be to toss the offending application into a
    > > > container which has a modified drop-behind policy. And here we need to
    > > > drag out the crystal ball: what *is* the best way of tuning application
    > > > pagecache behaviour? Will we gravitate towards containerization, or
    > > > will we gravitate towards finer-tuned fadvise/sync_page_range/etc
    > > > behaviour? Thus far it has been the latter, and I don't think that has
    > > > been a great success.
    > > >
    > > > Finally, are the problems which prompted these patchsets already
    > > > solved? What happens if you take the offending streaming application
    > > > and toss it into a 16MB memcg? That *should* avoid perturbing other
    > > > things running on that machine.
    > >
    > > Moving the streaming application into a 16MB memcg can be dangerous in
    > > some cases... the application might start to do "bad" things, like
    > > swapping (if the memcg can swap) or just fail due to OOMs.
    >
    > Well OK, maybe there are problems with the current implementation. But
    > are they unfixable problems? Is the right approach to give up on ever
    > making containers useful for this application and to instead go off and
    > implement a new and separate feature?
    >
    > > > And yes, a container-based approach is pretty crude, and one can
    > > > envision applications which only want modified reclaim policy for one
    > > > particualr file. But I suspect an application-wide reclaim policy
    > > > solves 90% of the problems.
    > >
    > > I really like the container-based approach. But for this we need a
    > > better file cache control in the memory cgroup; now we have the
    > > accounting of file pages, but there's no way to limit them.
    >
    > Again, if/whem memcg becomes sufficiently useful for this application
    > we're left maintaining the obsolete POSIX_FADVISE_NOREUSE for ever.

    Yes, totally agree. For the future a memcg-based solution is probably
    the best way to go.

    This reminds me to the old per-memcg dirty memory discussion
    (http://thread.gmane.org/gmane.linux.kernel.mm/67114), cc'ing Greg.

    Maybe the generic feature to provide that could solve both problems is
    a better file cache isolation in memcg.

    Thanks,
    -Andrea


    \
     
     \ /
      Last update: 2012-02-15 02:39    [W:4.199 / U:0.008 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site