lkml.org 
[lkml]   [2004]   [Feb]   [5]   [last100]   RSS Feed
Views: [more markup]  [less markup]  [headers]  [forward] 
 
Messages in this thread
/
DateThu, 5 Feb 2004 19:09:04 -0800
FromAndrew Morton <>
SubjectRe: Limit hash table size
Andi Kleen <ak@suse.de> wrote:
>
> Andrew Morton <akpm@osdl.org> writes:
> 
> > Ken, I remain unhappy with this patch.  If a big box has 500 million
> > dentries or inodes in cache (is possible), those hash chains will be more
> > than 200 entries long on average.  It will be very slow.
> 
> How about limiting the global size of the dcache in this case ? 

But to what size?

The thing is, any workload which touches a huge number of dentries/inodes
will, if it touches them again, touch them again in exactly the same order.
This triggers the worst-case LRU behaviour.

So if you limit dcache to 100MB and you happen to have a workload which
touches 101MB's worth, you get a 100% miss rate.  You suffer a 100000%
slowdown on the second pass, which is unhappy.  It doesn't seem worth
crippling such workloads just because of the updatedb thing.
> I cannot imagine a workload where it would make sense to ever cache 
> 500 million dentries. It just risks to keep the whole file system
> after an updatedb in memory on a big box, which is not necessarily
> good use of the memory.

A decent approach to the updatedb problem is an application hint which says
"reclaim i/dcache harder".  Just turn it on during the updatedb run -
crude, but it's a start.

But I've been telling poeple for a year that they should set
/proc/sys/vm/swappiness to zero during the updatedb run and afaik nobody has
bothered to try it...

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:00    [from the cache]
©2003-2008