lkml.org 
[lkml]   [1999]   [Oct]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectLinux Buffer Cache Does Not Support Mirroring

    Linus,

    The Netware file system for linux is running mirroring now (up to 8
    mirrors per logical partition), but unfortunately, we had to write our
    own buffer cache and integrate it in a modular fashion with all the
    Linux versions -- not as simple an undertaking as one might think given
    all the versions of Linux. We are still calling getblk() underneath so
    both buffer caches are in sync, but are now looking at completely
    by-passing your cache and going to disk with make_request(). We are
    close on balancing memory consumption between the caches, but sure could
    use some help.

    All of this work delayed us by about 4 months with the project and was
    something we did not anticipate. The problem is that the buffer cache
    is based on physical disk LBA offsets rather than supporting a tiered
    semantic for logical disk objects. Ideally, if the buffer cache
    supported a logical object manager (partition based) that would
    translate logical partition blocks to LBA's, with a configurable "list"
    of devices to write to, we would be there.

    Here's the problem with what's there in a nutshell. If we use your
    buffer cache to perform the mirroring, we end up double, triple, and
    quadruple caching data. Since you store blocks based on LBA, each time
    we write a mirror, the data gets buffered once for each device.
    Ideally, since we are dealing with "mirrors" we only need one cached
    copy of the disk block and the ability to force it to write to several
    physical devices at once. There are also two other cases that are
    biting us in the ass. We load balance reads by striping them across the
    mirrors and reading them round robin -- again we get double caching for
    each mirror present.

    There is also another case where we perform an artificial hotfix to
    mirrors on read HOTFIX errors (if the sectors were not recovered on the
    primary, and could not be located on a valid mirror, then mirrors must
    be artificially HOTFIXED so users will get a disk read error on any
    files that could not be recovered.

    This will require a semantic similiar to what's in NT's VM Cache today
    (they already ran across this with their FTDISK.SYS stuff, and learned
    that this semantic is needed -- a cache object handle that is relative
    to each partition on a device that the buffer cache maps underneath to
    LBA). For now, we've got our own buffer cache that is customized for
    what we need, and I don't expect you to backrev older Linux versions,
    but we do need this moving forward, otherwise, Linux will be limited in
    terms of fault tolerance available.

    We've got all the tools for creating Netware volumes, mirrors, disk
    repair, etc. finished and will be putting them up soon. Steve Spicer is
    enabling NWFS for SPARC and ALPHA machines. We want to get EVERYTHING
    done at once and put up at one time.

    Please advise on these cache issues.

    Jeff Merkey

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.rutgers.edu
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-03-22 13:54    [W:6.339 / U:0.068 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site