lkml.org 
[lkml]   [2005]   [Mar]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: huge filesystems
From
Date
Andreas Dilger <adilger@shaw.ca> writes:

> On Mar 14, 2005 21:37 -0700, jmerkey wrote:
> > 1. Scaling issues with readdir() with huge numbers of files (not even
> > huge really. 87000 files in a dir takes a while
> > for readdir() to return results). I average 2-3 million files per
> > directory on 2.6.9. It can take a up to a minute for
> > readdir() to return from initial reading from on of these directories
> > with readdir() through the VFS.
>
> Actually, unless I'm mistaken the problem is that "ls" (even when you
> ask it not to sort entries) is doing readdir on the whole directory
> before returning any results. We see this with Lustre and very large
> directories. Run strace on "ls" and it is doing masses of readdirs, but
> no output to stdout. Lustre readdir works OK on directories up to 10M
> files, but ls sucks.

The classic test is does 'echo *' which does the readdir but not the
stat come back quickly?

Anyway most of the readdir work is in the filesystem so I don't see
how the VFS would be involved....

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 14:11    [W:0.080 / U:0.028 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site