lkml.org 
[lkml]   [2007]   [Mar]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: userspace pagecache management tool
On Sat, 03 Mar 2007 20:23:07 -0500 Rik van Riel <riel@redhat.com> wrote:

> Andrew Morton wrote:
> > On Sat, 03 Mar 2007 19:01:01 -0500 Rik van Riel <riel@redhat.com> wrote:
>
> >> The use-once policy we have in the kernel should work
> >> perfectly fine for backups. All we need to do is
> >> actually honor the accessed bit on active page cache
> >> pages, instead of flushing them onto the inactive
> >> list.
> >>
> >> What am I overlooking?
> >
> > That'll improve backups but will break other things.
> >
> > To do this effectively we'd need to change the policy so that new pagecache
> > allocations cause no scanning of used-twice pages at all. So that even
> > after many gigs of backing up, the working set is still there.
> >
> > Problem is, (for example) what about the person who has 80% of memory in
> > used-twice state and who then reads a file or files which are 20% or more of
> > the size of memory, two or more times. It'll be 100% cache misses, every time.
> > This will happen quite a lot. IOW, once those pages are in used-twice state,
> > how does further pagecache activity ever get them _out_ of that state? Only
> > by joining the used-twice page set, and that can't happen if the used-once-so-far
> > pages got reclaimed.
> >
> > Doing a refault thing would help a bit, but stops working at a certain point.
>
> At what point does it stop working?

We need to store that this-page-got-reclaimed info somewhere. I don't know
how space-efficient that is. Did anyone ever do an implementation?

Of course, the pages need to be re-read again so there's a potential 100%
hit there, which is in fact not a huge amount in this context. Depends how
often it occurs (all the time when refault is being useful?) versus what we
gain from it.

> I am not asking this to be difficult, I just want to get Linux
> a VM that does not need to be kludged up every time a distro
> ships it to its customers.

We have a communication problem here. Please please please work harder to
get these problems communicated to the MM developers. The only vendor MM
kludge of which I'm aware is a thing which Andrea is working on to address
a large-shm-segment versus bulk-IO problem (yup, database).

If you have enough of an understanding of a problem to be able to develop
and productise a fix then share that info madly, asap.

otoh, rhel-on-the-desktop-or-smaller probably isn't a huge priority, which
can be taken advantage of.

> I believe one starting point would be a concept that people
> cannot shoot holes in any more. That is no guarantee, but
> as long as the concept has known holes coding it up is likely
> to be a waste of time since the code will need kludges to
> deal with the problems later on and we'd be back to square
> one.

You mean design it and review the design before coding it? You'll find few
objections there.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2007-03-04 02:53    [W:0.194 / U:0.848 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site