lkml.org 
[lkml]   [2005]   [Oct]   [19]   [last100]   RSS Feed
Views: [more markup]   [less markup]   [headers]   [forward]  
 
Messages in this thread
/
DateWed, 19 Oct 2005 09:23:45 +0200 (CEST)
FromBodo Eggert <>
SubjectRe: large files unnecessary trashing filesystem cache?
On Tue, 18 Oct 2005, Badari Pulavarty wrote:

> On Tue, 2005-10-18 at 23:58 +0200, Bodo Eggert wrote:
> > Badari Pulavarty <pbadari@gmail.com> wrote:
> > > On Tue, 2005-10-18 at 22:01 +0200, Guido Fiala wrote:
> > 
> > [large files trash cache]
> > 
> > > Is there a reason why those applications couldn't use O_DIRECT ?
> > 
> > The cache trashing will affect all programs handling large files:
> > 
> > mkisofs * > iso
> > dd < /dev/hdx42 | gzip > imagefile
> > perl -pe's/filenamea/filenameb/' < iso | cdrecord - # <- never tried
> > 
> 
> Are these examples which demonstrate the thrashing problem.

You can alyo cat a big file into /dev/null. I made those examples in order 
to demonstrate the problem with using O_DIRECT.

OTOH, I don't realtime stuff on my computer, so I'm not really affected, 
but I'll try to show it anyway.

> > Changing a few programs will only partly cover the problems.
> > 
> > I guess the solution would be using random cache eviction rather than
> > a FIFO. I never took a look the cache mechanism, so I may very well be
> > wrong here.
> 
> Read-only pages should be re-cycled really easily & quickly. I can't
> belive read-only pages are causing you all the trouble.

Just a q&d test:

$ time ls -l $DIR > /dev/null
real    0m0.442s
user    0m0.008s
sys     0m0.024s
$ time ls -l $DIR > /dev/null
real    0m0.077s
user    0m0.008s
sys     0m0.008s
cat $BIGFILES_1.5GB > /Dev/null
$ time ls -l $DIR > /dev/null
real    0m0.270s
user    0m0.008s
sys     0m0.008s
$ time ls -l $DIR > /dev/null
real    0m0.078s
user    0m0.004s
sys     0m0.004s


BTW:
I suggested the random eviction because it will evict pages from large 
files more likely than pages from small files, but I now think it will 
cause the evicted pages to be non-continuous, too, and thereby cause 
rereading them to be slower. I don't know which effect would be worse.

-- 
Is reading in the bathroom considered Multitasking? 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-10-19 05:30    [W:0.336 / U:0.420 seconds]
©2003-2008 Jasper Spaans