[lkml]   [2008]   [Dec]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: Pull request for FS-Cache, including NFS patches
    On Thu, Dec 18, 2008 at 07:45:57PM -0800, Muntz, Daniel wrote:
    > Local disk cache was great for AFS back around 1992. Typical networks
    > were 10 or 100Mbps (slower than disk access at the time),

    Would a disk cache on SSD make any sense? Seems like it'd change the
    latency tradeoff.

    > and memories
    > were small (typical 16MB). FS-Cache appears to help only with read
    > traffic--one reason why the web loves caching--and only for reads that
    > would miss the buffer/page cache (memories are now "large"). Solaris
    > has had CacheFS since ~1995, HPUX had a port of it since ~1997. I'd be
    > interested in evidence of even a small fraction of Solaris and/or HPUX
    > shops using CacheFS. I am aware of customers who thought it sounded
    > like a good idea, but ended up ditching it for various reasons (e.g.,
    > CacheFS just adds overhead if you almost always hit your local mem
    > cache).

    More details on the experiences of RHEL/Fedora users might be
    interesting. My (vague, mostly uniformed) impression is that the group
    of people who think they need it is indeed a lot larger than the group
    who really do need it--but that the latter group still exists.


    > One argument in favor that I don't see here is that local disk cache is
    > persistent (I'm assuming it is in your implementation).
    > Addressing 1 and 2 in your list, I'd be curious how often a miss in core
    > is a hit on disk.
    > Number 3 scares me. How does this play with the expected semantics of
    > NFS?
    > Number 5 is hard, if not provably requiring human intervention to do
    > syncs when writes are involved (see Disconnected AFS work by
    > UM/CITI/Huston, and work at CMU).
    > Add persistence as number 6. This may be the best reason to have it,
    > imho.
    > -Dan
    > -----Original Message-----
    > From: David Howells []
    > Sent: Thursday, December 18, 2008 6:27 PM
    > To: Andrew Morton
    > Cc:;;
    > Subject: Re: Pull request for FS-Cache, including NFS patches
    > Andrew Morton <> wrote:
    > > I don't believe that it has yet been convincingly demonstrated that we
    > > want to merge it at all.
    > >
    > > It's a huuuuuuuuge lump of new code, so it really needs to provide
    > > decent value. Can we revisit this? Yet again? What do we get from
    > > all this?
    > I should tell you to go and reread LKML at this point.
    > But... What can FS-Cache do for you? Well, in your specific case,
    > probably nothing. If all your computers are local to your normal
    > desktop box and are connected by sufficiently fast network and you have
    > sufficiently few of them, or you don't use any of NFS, AFS, CIFS,
    > Lustre, CRFS, CD-ROMs then it is likely that won't gain you anything.
    > Even if you do use some of those "netfs's", it won't get you anything
    > yet because I haven't included patches to support anything other than
    > NFS and the in-kernel AFS client yet.
    > However, if you do use NFS (or my AFS client), and you are accessing
    > computers via slow networks, or you have lots of machines spamming your
    > NFS server, then it might avail you.
    > It's a compromise: a trade-off between the loading and latencies of your
    > network vs the loading and latencies of your disk; you sacrifice disk
    > space to make up for the deficiencies of your network. The worst bit is
    > that the latencies are additive under some circumstances (when doing a
    > page read you may have to check disk and then go to the network).
    > So, FS-Cache can do the following for you:
    > (1) Allow you to reduce network loading by diverting repeat reads to
    > local
    > storage.
    > (2) Allow you to reduce the latency of slow network links by diverting
    > repeat
    > reads to local storage.
    > (3) Allow you to reduce the effect of uncontactable servers by serving
    > data
    > out of local storage.
    > (4) Allows you to reduce the latency of slow rotating media (such as
    > CDROM
    > and CD-changers).
    > (5) Allow you to implement disconnected operation, partly by (3), but
    > also by
    > caching changes for later syncing.
    > Now, (1) and (2) are readily demonstrable. I have posted benchmarks to
    > do this. (3) to (5) are not yet implemented; these have to be mostly
    > implemented in the filesystems that use FS-Cache rather than FS-Cache
    > itself. FS-Cache currently has sufficient functionality to do (3) and
    > (4), but needs some extra bits to do (5).
    > I've tried to implement just the minimal useful functionality for
    > persistent caching. There is more to be added, but it's not immediately
    > necessary.
    > FS-Cache tries to make its facilities as simple and as general as
    > possible so that _any_ filesystem or blockdev can use it. I have
    > patches here to make NFS and AFS use it. I know someone is working on
    > getting Lustre to use it, and other filesystem maintainers have
    > expressed interest, subject to the code getting upstream.
    > Furthermore, FS-Cache hides the implementation of the cache from the
    > netfs.
    > Not only that, it hides I/O errors in the cache from the netfs. Why
    > should the netfs have to deal with such things?
    > Another way to look at things is to look at other cases of cached
    > netfs's.
    > OpenAFS for example. It has a local cache of its own. Solaris has
    > local NFS caching. Windows has local caching for NFS and CIFS, I think.
    > Even web browsers have local caching.
    > > 303 files changed, 21049 insertions(+), 3726 deletions(-)
    > A big chunk of that, particularly the deletions, is the creds patches.
    > Excluding the stuff pulled from the security and NFS trees, the combined
    > FS-Cache, CacheFiles and AFS+ and NFS+FS-Cache patches are, in fact:
    > 86 files changed, 15385 insertions(+), 413 deletions(-)
    > and over 19% of the insertions is documentation. Most of the deletions
    > (373) are in AFS.
    > > Are any distros pushing for this? Or shipping it? If so, are they
    > > able to weigh in and help us with this quite difficult decision?
    > We (Red Hat) have shipped it in RHEL-5 and some Fedora releases. Doing
    > so is quite an effort, though, precisely because the code is not yet
    > upstream. We have customers using it and are gaining more customers who
    > want it. There even appear to be CentOS users using it (or at least
    > complaining when it breaks).
    > I don't know what will convince you. I've given you theoretical reasons
    > why caching ought to be useful; I've backed up the ones I've implemented
    > with benchmarks; I've given you examples of what our customers are doing
    > with it or want to do with it. Please help me understand what else you
    > want.
    > Do you perhaps want the netfs maintainers (such as Trond) to say that
    > it's necessary?
    > David
    > _______________________________________________
    > NFSv4 mailing list
    > _______________________________________________
    > NFSv4 mailing list

     \ /
      Last update: 2008-12-19 05:13    [W:0.032 / U:4.584 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site