[lkml]   [2008]   [Aug]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: Recursive directory accounting for size, ctime, etc.
    On Tue 2008-07-15 11:28:22, Sage Weil wrote:
    > All-
    > Ceph is a new distributed file system for Linux designed for scalability
    > (terabytes to exabytes, tens to thousands of storage nodes), reliability,
    > and performance. The latest release (v0.3), aside from xattr support and
    > the usual slew of bugfixes, includes a unique (?) recursive accounting
    > infrastructure that allows statistics about all metadata nested beneath a
    > point in the directory hierarchy to be efficiently propagated up the tree.
    > Currently this includes a file and directory count, total bytes (summation
    > over file sizes), and most recent inode ctime. For example, for a
    > directory like /home, Ceph can efficiently report the total number of
    > files, directories, and bytes contained by that entire subtree of the
    > directory hierarchy.
    > The file size summation is the most interesting, as it effectively gives
    > you directory-based quota space accounting with fine granularity. In many
    > deployments, the quota _accounting_ is more important than actual
    > enforcement. Anybody who has had to figure out what has filled/is filling
    > up a large volume will appreciate how cumbersome and inefficient 'du' can
    > be for that purpose--especially when you're in a hurry.
    > There are currently two ways to access the recursive stats via a standard
    > shell. The first simply sets the directory st_size value to the
    > _recursive_ bytes ('rbytes') value (when the client is mounted with -o
    > rbytes). For example (watch the directory sizes),

    > Naturally, there are a few caveats:
    > - There is some built-in delay before statistics fully propagate up
    > toward the root of the hierarchy. Changes are propagated
    > opportunistically when lock/lease state allows, with an upper bound of (by
    > default) ~30 seconds for each level of directory nesting.

    Having instant rctime would be very nice -- for stuff like locate and
    speeding up kde startup.

    > I'm extremely interested in what people think of overloading the file
    > system interface in this way. Handy? Crufty? Dangerous? Does anybody

    Too ugly to live.

    What about new rstat() syscall?

    (cesky, pictures)

     \ /
      Last update: 2008-08-07 22:13    [W:0.032 / U:9.404 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site