lkml.org 
[lkml]   [2005]   [Oct]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: large files unnecessary trashing filesystem cache?
On Tue, 18 Oct 2005, Badari Pulavarty wrote:

> On Tue, 2005-10-18 at 23:58 +0200, Bodo Eggert wrote:
> > Badari Pulavarty <pbadari@gmail.com> wrote:
> > > On Tue, 2005-10-18 at 22:01 +0200, Guido Fiala wrote:
> >
> > [large files trash cache]
> >
> > > Is there a reason why those applications couldn't use O_DIRECT ?
> >
> > The cache trashing will affect all programs handling large files:
> >
> > mkisofs * > iso
> > dd < /dev/hdx42 | gzip > imagefile
> > perl -pe's/filenamea/filenameb/' < iso | cdrecord - # <- never tried
> >
>
> Are these examples which demonstrate the thrashing problem.

You can alyo cat a big file into /dev/null. I made those examples in order
to demonstrate the problem with using O_DIRECT.

OTOH, I don't realtime stuff on my computer, so I'm not really affected,
but I'll try to show it anyway.

> > Changing a few programs will only partly cover the problems.
> >
> > I guess the solution would be using random cache eviction rather than
> > a FIFO. I never took a look the cache mechanism, so I may very well be
> > wrong here.
>
> Read-only pages should be re-cycled really easily & quickly. I can't
> belive read-only pages are causing you all the trouble.

Just a q&d test:

$ time ls -l $DIR > /dev/null
real 0m0.442s
user 0m0.008s
sys 0m0.024s

$ time ls -l $DIR > /dev/null
real 0m0.077s
user 0m0.008s
sys 0m0.008s

cat $BIGFILES_1.5GB > /Dev/null

$ time ls -l $DIR > /dev/null
real 0m0.270s
user 0m0.008s
sys 0m0.008s

$ time ls -l $DIR > /dev/null
real 0m0.078s
user 0m0.004s
sys 0m0.004s



BTW:
I suggested the random eviction because it will evict pages from large
files more likely than pages from small files, but I now think it will
cause the evicted pages to be non-continuous, too, and thereby cause
rereading them to be slower. I don't know which effect would be worse.

--
Is reading in the bathroom considered Multitasking?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-10-19 09:30    [W:0.064 / U:2.080 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site