lkml.org 
[lkml]   [2008]   [Mar]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
SubjectRe: [PATCH][BUGFIX][RFC] fix soft lock up at NFS mount by making limitation of dentry_unused
On Thu, Mar 06, 2008 at 01:41:29PM +0900, Kentaro Makita wrote:
> [Summary]
> Make a limitation of dentry_unused to avoid soft lock up at NFS mounts
> and remounting any filesystem.
>
> [Descriptions]
> - background
> dentry_unused is a list of dentries which is not in use. This works
> as a cache against not-exisiting files. dentry_unused grows up when
> directories or files are removed. This list can be *verrry* long
> if there is no memory pressure, because there is no limit.
>
> - what's problem
> When prune_dcache() is called, it scans *all* dentry_unused linearly
> under spin_lock(). This scan costs very much if there are many entries.
> For example, prune_dcache() is called at mounting NFS.
> In our test, when there are 100,000,000 of unused dentries, mounting
> NFS took 1 minutes and almost all user programs hang during it.
>
> 100,000,000 is possible number on large systems.
>
> This problem already happend on our system.
> Therefore, we need a limitation of dentry_unused.

No, we need a smarter free list structure. There have been several attempts
at this in the past. Two that I can recall off the top of my head:

- per node unused LRUs
- per superblock unusued LRUs

I guess we need to revisit this again, because limiting the size of
the cache like this is not an option.

> I feel we need more tests to determine resonable value to any system.
> So, please test.
.....
> Tested on Intel Itanium 2 9050 (dualcore) x12 MEM 24GB , kernel-2.6.25-rc4
> I found no peformance regression in my tests.

Try something that relies on leaving the working set on the unused
list, like NFS server benchmarks that have a working set of tens of
million of files....

> Signed-off-by: Kentaro Makita <k-makita@np.css.fujitsu.com>
> ---
> fs/dcache.c | 7 +++++++
> 1 files changed, 7 insertions(+)
> diff -rupN -X linux-2.6.25-rc4/Documentation/dontdiff linux-2.6.25-rc4/fs/dcache.c linux-2.6.25-rc4mod/fs/dcache.c
> --- linux-2.6.25-rc4/fs/dcache.c 2008-03-05 13:33:54.000000000 +0900
> +++ linux-2.6.25-rc4mod/fs/dcache.c 2008-03-05 16:47:18.000000000 +0900
> @@ -42,6 +42,8 @@ __cacheline_aligned_in_smp DEFINE_SEQLOC
>
> EXPORT_SYMBOL(dcache_lock);
>
> +/* threshold to limit dentry_unused */
> +unsigned int dentry_unused_ratio = 10000;
> static struct kmem_cache *dentry_cache __read_mostly;
>
> #define DNAME_INLINE_LEN (sizeof(struct dentry)-offsetof(struct dentry,d_iname))
> @@ -61,6 +63,7 @@ static unsigned int d_hash_mask __read_m
> static unsigned int d_hash_shift __read_mostly;
> static struct hlist_head *dentry_hashtable __read_mostly;
> static LIST_HEAD(dentry_unused);
> +static void prune_dcache(int count, struct super_block *sb);
>
> /* Statistics gathering. */
> struct dentry_stat_t dentry_stat = {
> @@ -214,6 +217,10 @@ repeat:
> }
> spin_unlock(&dentry->d_lock);
> spin_unlock(&dcache_lock);
> + /* Prune unused dentry over threshold level */
> + int nr_in_use = (dentry_stat.nr_dentry - dentry_stat.nr_unused);
> + if (dentry_stat.nr_dentry > nr_in_use * dentry_unused_ratio / 100)
> + prune_dcache(dentry_stat.nr_unused * 5 / 100 , NULL);

nr_in_use is going to overflow 32 bits with this calculation.

Cheers,

Dave.
--
Dave Chinner
Principal Engineer
SGI Australian Software Group


\
 
 \ /
  Last update: 2008-03-06 06:57    [W:0.174 / U:11.744 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site