Messages in this thread | | | Date | Wed, 21 Apr 2010 22:56:57 +0400 | From | Evgeniy Polyakov <> | Subject | Re: readahead on directories |
| |
On Wed, Apr 21, 2010 at 07:51:24PM +0100, Jamie Lokier (jamie@shareable.org) wrote: > Fwiw, I found sorting directories by inode and reading them in that > order help to reduce seeks, some 10 years ago. I implemented > something like 'find' which works like that, keeping a queue of > directories to read and things to open/stat, ordered by inode number > seen in d_ino before open/stat and st_ino after. However it did not > try to readahead the blocks inside a directory, or sort operations by > block number. It reduced some 'find'-like operations to about a > quarter of the time on cold cache. I still use that program sometimes > before "git status" ;-) Google "treescan" and "lokier" if you're > interested in trying it (though I use 0.7 which isn't published).
As you might expect it is not really a directory readahead :) Nad I'm not really sure ext234 can implement it in kernel more optimally without breaking backward compatibility though.
> > it is not about readdir(). Plain read() is synchronous too. But > > filesystem can respond to readahead calls and read next block to current > > one, while it won't do this for next direntry. > > I'm surprised it makes much difference, as directories are usually not > very large anyway.
Well, having several tens of millions of files in 64k dirs takes from tens of seconds to minutes to read just because of that.
> But if it does, go on, try FIEMAP and blockdev reading, you know you > want to :-)
Well, it requires substantial underlying fs knowledge and is not simple and, well, appropriate to do in some cases.
-- Evgeniy Polyakov
| |