lkml.org 
[lkml]   [2008]   [Mar]   [5]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
Subject[PATCH][BUGFIX][RFC] fix soft lock up at NFS mount by making limitation of dentry_unused
[Summary]
Make a limitation of dentry_unused to avoid soft lock up at NFS mounts
and remounting any filesystem.

[Descriptions]
- background
dentry_unused is a list of dentries which is not in use. This works
as a cache against not-exisiting files. dentry_unused grows up when
directories or files are removed. This list can be *verrry* long
if there is no memory pressure, because there is no limit.

- what's problem
When prune_dcache() is called, it scans *all* dentry_unused linearly
under spin_lock(). This scan costs very much if there are many entries.
For example, prune_dcache() is called at mounting NFS.
In our test, when there are 100,000,000 of unused dentries, mounting
NFS took 1 minutes and almost all user programs hang during it.

100,000,000 is possible number on large systems.

This problem already happend on our system.
Therefore, we need a limitation of dentry_unused.

- How to fix
Limit number of unused dentries to suitable value.

Threshold is as follows:
dentry_unused_ratio: default value is 10000(%). If the amount of
dentry_unused reaches to 10000% of the amount of dentry_in_use,
5% of them are freed.

I feel we need more tests to determine resonable value to any system.
So, please test.

This patch is based on linux-2.6.25-rc4.

-Test Results

Result on 24GB boxes with excessive unused dentries.
Without patch:
# cat /proc/sys/fs/dentry-state
103327453 103313783 45 0 0
# time mount -t nfs 192.168.0.2:/export /mnt
real 1m4.698s
user 0m0.000s
sys 1m4.672s

With this patch:
# cat /proc/sys/fs/dentry-state
118681 117225 45 0 0 0
# time mount -t nfs 192.168.0.2:/export /mnt
real 0m0.103s
user 0m0.004s
sys 0m0.076s

Tested on Intel Itanium 2 9050 (dualcore) x12 MEM 24GB , kernel-2.6.25-rc4
I found no peformance regression in my tests.


Best Regards,
Kentaro Makita

Signed-off-by: Kentaro Makita <k-makita@np.css.fujitsu.com>
---
fs/dcache.c | 7 +++++++
1 files changed, 7 insertions(+)
diff -rupN -X linux-2.6.25-rc4/Documentation/dontdiff linux-2.6.25-rc4/fs/dcache.c linux-2.6.25-rc4mod/fs/dcache.c
--- linux-2.6.25-rc4/fs/dcache.c 2008-03-05 13:33:54.000000000 +0900
+++ linux-2.6.25-rc4mod/fs/dcache.c 2008-03-05 16:47:18.000000000 +0900
@@ -42,6 +42,8 @@ __cacheline_aligned_in_smp DEFINE_SEQLOC

EXPORT_SYMBOL(dcache_lock);

+/* threshold to limit dentry_unused */
+unsigned int dentry_unused_ratio = 10000;
static struct kmem_cache *dentry_cache __read_mostly;

#define DNAME_INLINE_LEN (sizeof(struct dentry)-offsetof(struct dentry,d_iname))
@@ -61,6 +63,7 @@ static unsigned int d_hash_mask __read_m
static unsigned int d_hash_shift __read_mostly;
static struct hlist_head *dentry_hashtable __read_mostly;
static LIST_HEAD(dentry_unused);
+static void prune_dcache(int count, struct super_block *sb);

/* Statistics gathering. */
struct dentry_stat_t dentry_stat = {
@@ -214,6 +217,10 @@ repeat:
}
spin_unlock(&dentry->d_lock);
spin_unlock(&dcache_lock);
+ /* Prune unused dentry over threshold level */
+ int nr_in_use = (dentry_stat.nr_dentry - dentry_stat.nr_unused);
+ if (dentry_stat.nr_dentry > nr_in_use * dentry_unused_ratio / 100)
+ prune_dcache(dentry_stat.nr_unused * 5 / 100 , NULL);
return;

unhash_it:

\
 
 \ /
  Last update: 2008-03-06 05:43    [W:0.060 / U:1.036 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site