lkml.org 
[lkml]   [2005]   [Mar]   [19]   [last100]   RSS Feed
Views: [more markup]  [less markup]  [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: huge filesystems
From(Eric W. Biederman)
Date19 Mar 2005 04:06:17 -0700
Andreas Dilger <adilger@shaw.ca> writes:

> On Mar 14, 2005  21:37 -0700, jmerkey wrote:
> > 1. Scaling issues with readdir() with huge numbers of files (not even 
> > huge really. 87000 files in a dir takes a while
> > for readdir() to return results). I average 2-3 million files per 
> > directory on 2.6.9. It can take a up to a minute for
> > readdir() to return from initial reading from on of these directories 
> > with readdir() through the VFS.
> 
> Actually, unless I'm mistaken the problem is that "ls" (even when you
> ask it not to sort entries) is doing readdir on the whole directory
> before returning any results.  We see this with Lustre and very large
> directories.  Run strace on "ls" and it is doing masses of readdirs, but
> no output to stdout.  Lustre readdir works OK on directories up to 10M
> files, but ls sucks.

The classic test is does 'echo *' which does the readdir but not the
stat come back quickly?  

Anyway most of the readdir work is in the filesystem so I don't see
how the VFS would be involved....
Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:11    [W:0.225 / U:0.000 seconds]
©2003-2008 Jasper Spaans