Messages in this thread |  | | | From | "Joseph Malicki" <> | | Subject | Re: cache limit | | Date | Wed, 27 Aug 2003 12:03:36 -0400 |
| |
I was premature about the test case, but still, a process (or several) that mmap several GB's of files and dont unmap what they don't need has caused issues in the past.
-joe
----- Original Message ----- From: "Joseph Malicki" <jmalicki@starbak.net> To: "William Lee Irwin III" <wli@holomorphy.com>; "Takao Indoh" <indou.takao@soft.fujitsu.com> Cc: "Mike Fedyk" <mfedyk@matchmail.com>; <linux-kernel@vger.kernel.org> Sent: Wednesday, August 27, 2003 12:01 PM Subject: Re: cache limit
> I've had experience with unneeded, *mapped* pages that would be ideally > flushed oppressing needed mapped and unmapped pages. > Test case: grep --mmap SOME_STRING_I_WONT_FIND some_multi-GB-file > > Sure, it's bad programming etc, but in that case, once those pages are > mapped, they can't be forcibly unmapped even though > in a utopian VM they would be discarded as unneeded. > > This could very well be the problem? > > -joe > > ----- Original Message ----- > From: "William Lee Irwin III" <wli@holomorphy.com> > To: "Takao Indoh" <indou.takao@soft.fujitsu.com> > Cc: "Mike Fedyk" <mfedyk@matchmail.com>; <linux-kernel@vger.kernel.org> > Sent: Wednesday, August 27, 2003 5:45 AM > Subject: Re: cache limit > > > > On Wed, Aug 27, 2003 at 06:36:10PM +0900, Takao Indoh wrote: > > > This problem happened a few month ago and the detailed data does not > > > remain. Therefore it is difficult to know what is essential cause for > > > this problem, but, I guessed that pagecache used as I/O cache grew > > > gradually during system running, and finally it oppressed memory. > > > > But this doesn't make any sense; the only memory you could "oppress" > > is pagecache. > > > > > > On Wed, Aug 27, 2003 at 06:36:10PM +0900, Takao Indoh wrote: > > > Besides this problem, there are many cases where increase of pagecache > > > causes trouble, I think. > > > For example, DBMS. > > > DBMS caches index of DB in their process space. > > > This index cache conflicts with the pagecache used by other > applications, > > > and index cache may be paged out. It cause uneven response of DBMS. > > > In this case, limiting pagecache is effective. > > > > Why is it effective? You're describing pagecache vs. pagecache > > competition and the DBMS outcompeting the cooperating applications for > > memory to the detriment of the workload; this is a very different > > scenario from what "limiting pagecache" sounds like. > > > > How do you know it would be effective? Have you written a patch to > > limit it in some way and tried running it? > > > > > > On Tue, 26 Aug 2003 02:46:34 -0700, William Lee Irwin III wrote: > > >> One thing I thought of after the post was whether they actually had in > > >> mind tunable hard limits on _unmapped_ pagecache, which is, in fact, > > >> useful. OTOH that's largely speculation and we really need them to > > >> articulate the true nature of their problem. > > > > On Wed, Aug 27, 2003 at 06:36:10PM +0900, Takao Indoh wrote: > > > I also think that is effective. Empirically, in the case where pagecache > > > causes memory shortage, most of pagecache is unmapped page. Of course > > > real problem may not be pagecashe, as you or Mike said. > > > > How do you know most of it is unmapped? > > > > At any rate, the above assigns a meaningful definition to the words you > > used; it does not necessarily have anything to do with the issue you're > > trying to describe. If you could start from the very basics, reproduce > > the problem, instrument the workload with top(1) and vmstat(1), and find > > some way to describe how the performance is inadequate (e.g. performance > > metrics for your running DBMS/whatever in MB/s or transactions/s etc.), > > it would be much more helpful than proposing a solution up front. > > Without any evidence, we can't know it is a solution at all, or that > > it's the right solution. > > > > > > -- wli > > - > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > Please read the FAQ at http://www.tux.org/lkml/ > > >
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
|  |