lkml.org 
[lkml]   [1998]   [Nov]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [2.1.130-3] Page cache DEFINATELY too persistant... feature?


On Thu, 26 Nov 1998, Stephen C. Tweedie wrote:
>
> Right. Linus, it looks like this is a result of a long-standing
> property of the buffer cache, namely that we only ever throttle writes
> when we run out of memory. If we have large quantities of dirty buffers
> due to an install routine, then the shrink_mmap becomes less and less
> able to free memory. The recent vm changes make it a lot easier to
> swap things out in this situation.

Hmm.. I know we _used_ to do a sync() in do_free_pages() when we started
to need to page things out exactly to avoid this issue. Considering how
much has changed I wouldn't be surprised if that has been disabled or
deleted by mistake, because it tends to happen only under specific
circumstances that are _not_ the normal things you tend to test when you
test low-memory situations.

> The 130-pre3 changes seem to have fixed the swap aggression in the
> normal case when we are loading the page cache primarily with reads, but
> under high write load we still drop rapidly into swap as soon as the
> buffer cache is saturated.

We should really be a lot more aggressive about getting rid of dirty
buffer cache entries. They are almost never worth having around.

> There are basically two ways I think we can address this. We can either
> throttle the writes, or we can extend what we have done recently in
> try_to_free_pages to include dirty buffers.

I'd _much_ rather do the second thing. We should really consider a dirty
buffer to be the same thing as a dirty user page, and write it out the
same way we write out user pages. In fact, this whole problem wouldn't
exist if we just wrote to the page cache directly and had the page cache
do the writeout, but sadly I never sat down and really worked that out.

> This looks like a natural way to take advantage of existing behaviour to
> fix a very long standing write performance problem. Comments? Would
> you prefer a more natural way of throttling writes at source?

I think throwttling them at the source is just wrong, and think your
suggestion makes sense. I don't know whether it makes much sense to do
this with bdflush, though - I'd be more inclined to just do it directly.

Think of it as the analogous case to kswapd - we count on kswapd to
generally keep "enough" memory free and to balance the peak usage. But
when we really need more memory for a process, the process will try to
free stuff up itself.

Now, the same should be true of "bdflush" - we could on bdflush to
generally keep the number of dirty pages down, and to balance the peak
usage so that we don't get bouts of extremely heavy disk activity (like we
did with the old "update" process). But when we really need to write stuff
out, the page-out process should really try to free stuff up itself.

Would you agree with that analogy?

Linus


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:45    [from the cache]
©2003-2011 Jasper Spaans