Messages in this thread | | | Date | Thu, 26 Nov 1998 09:49:47 -0800 (PST) | From | Linus Torvalds <> | Subject | Re: [2.1.130-3] Page cache DEFINATELY too persistant... feature? |
| |
On Thu, 26 Nov 1998, Stephen C. Tweedie wrote: > > Right. Linus, it looks like this is a result of a long-standing > property of the buffer cache, namely that we only ever throttle writes > when we run out of memory. If we have large quantities of dirty buffers > due to an install routine, then the shrink_mmap becomes less and less > able to free memory. The recent vm changes make it a lot easier to > swap things out in this situation.
Hmm.. I know we _used_ to do a sync() in do_free_pages() when we started to need to page things out exactly to avoid this issue. Considering how much has changed I wouldn't be surprised if that has been disabled or deleted by mistake, because it tends to happen only under specific circumstances that are _not_ the normal things you tend to test when you test low-memory situations.
> The 130-pre3 changes seem to have fixed the swap aggression in the > normal case when we are loading the page cache primarily with reads, but > under high write load we still drop rapidly into swap as soon as the > buffer cache is saturated.
We should really be a lot more aggressive about getting rid of dirty buffer cache entries. They are almost never worth having around.
> There are basically two ways I think we can address this. We can either > throttle the writes, or we can extend what we have done recently in > try_to_free_pages to include dirty buffers.
I'd _much_ rather do the second thing. We should really consider a dirty buffer to be the same thing as a dirty user page, and write it out the same way we write out user pages. In fact, this whole problem wouldn't exist if we just wrote to the page cache directly and had the page cache do the writeout, but sadly I never sat down and really worked that out.
> This looks like a natural way to take advantage of existing behaviour to > fix a very long standing write performance problem. Comments? Would > you prefer a more natural way of throttling writes at source?
I think throwttling them at the source is just wrong, and think your suggestion makes sense. I don't know whether it makes much sense to do this with bdflush, though - I'd be more inclined to just do it directly.
Think of it as the analogous case to kswapd - we count on kswapd to generally keep "enough" memory free and to balance the peak usage. But when we really need more memory for a process, the process will try to free stuff up itself.
Now, the same should be true of "bdflush" - we could on bdflush to generally keep the number of dirty pages down, and to balance the peak usage so that we don't get bouts of extremely heavy disk activity (like we did with the old "update" process). But when we really need to write stuff out, the page-out process should really try to free stuff up itself.
Would you agree with that analogy?
Linus
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |