[lkml]   [2008]   [Dec]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRE: Pull request for FS-Cache, including NFS patches
    Local disk cache was great for AFS back around 1992.  Typical networks
    were 10 or 100Mbps (slower than disk access at the time), and memories
    were small (typical 16MB). FS-Cache appears to help only with read
    traffic--one reason why the web loves caching--and only for reads that
    would miss the buffer/page cache (memories are now "large"). Solaris
    has had CacheFS since ~1995, HPUX had a port of it since ~1997. I'd be
    interested in evidence of even a small fraction of Solaris and/or HPUX
    shops using CacheFS. I am aware of customers who thought it sounded
    like a good idea, but ended up ditching it for various reasons (e.g.,
    CacheFS just adds overhead if you almost always hit your local mem

    One argument in favor that I don't see here is that local disk cache is
    persistent (I'm assuming it is in your implementation).

    Addressing 1 and 2 in your list, I'd be curious how often a miss in core
    is a hit on disk.
    Number 3 scares me. How does this play with the expected semantics of
    Number 5 is hard, if not provably requiring human intervention to do
    syncs when writes are involved (see Disconnected AFS work by
    UM/CITI/Huston, and work at CMU).
    Add persistence as number 6. This may be the best reason to have it,


    -----Original Message-----
    From: David Howells []
    Sent: Thursday, December 18, 2008 6:27 PM
    To: Andrew Morton
    Subject: Re: Pull request for FS-Cache, including NFS patches

    Andrew Morton <> wrote:

    > I don't believe that it has yet been convincingly demonstrated that we

    > want to merge it at all.
    > It's a huuuuuuuuge lump of new code, so it really needs to provide
    > decent value. Can we revisit this? Yet again? What do we get from
    > all this?

    I should tell you to go and reread LKML at this point.

    But... What can FS-Cache do for you? Well, in your specific case,
    probably nothing. If all your computers are local to your normal
    desktop box and are connected by sufficiently fast network and you have
    sufficiently few of them, or you don't use any of NFS, AFS, CIFS,
    Lustre, CRFS, CD-ROMs then it is likely that won't gain you anything.

    Even if you do use some of those "netfs's", it won't get you anything
    yet because I haven't included patches to support anything other than
    NFS and the in-kernel AFS client yet.

    However, if you do use NFS (or my AFS client), and you are accessing
    computers via slow networks, or you have lots of machines spamming your
    NFS server, then it might avail you.

    It's a compromise: a trade-off between the loading and latencies of your
    network vs the loading and latencies of your disk; you sacrifice disk
    space to make up for the deficiencies of your network. The worst bit is
    that the latencies are additive under some circumstances (when doing a
    page read you may have to check disk and then go to the network).

    So, FS-Cache can do the following for you:

    (1) Allow you to reduce network loading by diverting repeat reads to

    (2) Allow you to reduce the latency of slow network links by diverting
    reads to local storage.

    (3) Allow you to reduce the effect of uncontactable servers by serving
    out of local storage.

    (4) Allows you to reduce the latency of slow rotating media (such as
    and CD-changers).

    (5) Allow you to implement disconnected operation, partly by (3), but
    also by
    caching changes for later syncing.

    Now, (1) and (2) are readily demonstrable. I have posted benchmarks to
    do this. (3) to (5) are not yet implemented; these have to be mostly
    implemented in the filesystems that use FS-Cache rather than FS-Cache
    itself. FS-Cache currently has sufficient functionality to do (3) and
    (4), but needs some extra bits to do (5).

    I've tried to implement just the minimal useful functionality for
    persistent caching. There is more to be added, but it's not immediately

    FS-Cache tries to make its facilities as simple and as general as
    possible so that _any_ filesystem or blockdev can use it. I have
    patches here to make NFS and AFS use it. I know someone is working on
    getting Lustre to use it, and other filesystem maintainers have
    expressed interest, subject to the code getting upstream.

    Furthermore, FS-Cache hides the implementation of the cache from the
    Not only that, it hides I/O errors in the cache from the netfs. Why
    should the netfs have to deal with such things?

    Another way to look at things is to look at other cases of cached
    OpenAFS for example. It has a local cache of its own. Solaris has
    local NFS caching. Windows has local caching for NFS and CIFS, I think.
    Even web browsers have local caching.

    > 303 files changed, 21049 insertions(+), 3726 deletions(-)

    A big chunk of that, particularly the deletions, is the creds patches.
    Excluding the stuff pulled from the security and NFS trees, the combined
    FS-Cache, CacheFiles and AFS+ and NFS+FS-Cache patches are, in fact:

    86 files changed, 15385 insertions(+), 413 deletions(-)

    and over 19% of the insertions is documentation. Most of the deletions
    (373) are in AFS.

    > Are any distros pushing for this? Or shipping it? If so, are they
    > able to weigh in and help us with this quite difficult decision?

    We (Red Hat) have shipped it in RHEL-5 and some Fedora releases. Doing
    so is quite an effort, though, precisely because the code is not yet
    upstream. We have customers using it and are gaining more customers who
    want it. There even appear to be CentOS users using it (or at least
    complaining when it breaks).

    I don't know what will convince you. I've given you theoretical reasons
    why caching ought to be useful; I've backed up the ones I've implemented
    with benchmarks; I've given you examples of what our customers are doing
    with it or want to do with it. Please help me understand what else you

    Do you perhaps want the netfs maintainers (such as Trond) to say that
    it's necessary?

    NFSv4 mailing list

     \ /
      Last update: 2008-12-19 04:49    [W:0.030 / U:18.360 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site