Messages in this thread | | | Date | Thu, 5 Feb 2004 19:09:04 -0800 | From | Andrew Morton <> | Subject | Re: Limit hash table size |
| |
Andi Kleen <ak@suse.de> wrote: > > Andrew Morton <akpm@osdl.org> writes: > > > Ken, I remain unhappy with this patch. If a big box has 500 million > > dentries or inodes in cache (is possible), those hash chains will be more > > than 200 entries long on average. It will be very slow. > > How about limiting the global size of the dcache in this case ?
But to what size?
The thing is, any workload which touches a huge number of dentries/inodes will, if it touches them again, touch them again in exactly the same order. This triggers the worst-case LRU behaviour.
So if you limit dcache to 100MB and you happen to have a workload which touches 101MB's worth, you get a 100% miss rate. You suffer a 100000% slowdown on the second pass, which is unhappy. It doesn't seem worth crippling such workloads just because of the updatedb thing.
> I cannot imagine a workload where it would make sense to ever cache > 500 million dentries. It just risks to keep the whole file system > after an updatedb in memory on a big box, which is not necessarily > good use of the memory.
A decent approach to the updatedb problem is an application hint which says "reclaim i/dcache harder". Just turn it on during the updatedb run - crude, but it's a start.
But I've been telling poeple for a year that they should set /proc/sys/vm/swappiness to zero during the updatedb run and afaik nobody has bothered to try it...
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |