[lkml]   [2002]   [Jun]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: 3x slower file reading oddity
    On Mon, 17 Jun 2002, Andreas Dilger wrote:

    > On Jun 17, 2002 17:36 -0700, Andrew Morton wrote:
    > > You can probably lessen the seek-rate by accessing the files in the correct
    > > order. Read all the files from a directory before descending into any of
    > > its subdirectories. Can find(1) do that? You should be able to pretty
    > > much achieve disk bandwidth this way - it depends on how bad the inter-
    > > and intra-file fragmentation has become.
    > Just FYI - "find -depth" will do that, from find(1):
    > -depth Process each directory's contents before the directory itself.

    not quite... even with this, find processes a directory's contents in the
    order that the filenames appear on disk ... and this will recurse into
    subdirs before finishing with all the files.

    what i think andrew was suggesting is to process all non-directory entries
    before handling the directory entries.

    i'm working with something like this:

    find -type f -print \
    | perl -ne '@c = split(/\//); print $#c . " $_";' \
    | sort -k 1,1nr -k 2,2 \
    | perl -pe 's#^\d+ ##;' \
    > filelist

    (deliberately not using perl's sort because i want the thing to scale of
    course ;)

    that reverse sorts by number of path components first, pathname second.

    it seems to be worth another 10% or so improvement on the single reader

    i figured handling the leaves first was best... hard to tell if there's
    any difference if i remove the "r". (this is on a live system with other
    loads perturbing the results unfortunately.)


    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 13:26    [W:0.023 / U:1.476 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site