lkml.org 
[lkml]   [2008]   [Mar]   [10]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRe: [ANNOUNCE] Ramback: faster than a speeding bullet
    Date
    Hi Alan,

    Nice to see so many redhatters taking an avid interest in storage :-)

    On Monday 10 March 2008 02:22, Alan Cox wrote:
    > > So now you can ask some hard questions: what if the power goes out
    > > completely or the host crashes or something else goes wrong while
    > > critical data is still in the ramdisk? Easy: use reliable components.
    >
    > Nice fiction - stuff crashes eventually - not that this isn't useful. For
    > a long time simply loading a 2-3GB Ramdisk off hard disk has been a good
    > way to build things like compile engines where loss of state is not bad.

    Right, and now with ramback you will be able to preserve that state and
    have the performance too. It is a wonderful world.

    > > If UPS power runs out while ramback still holds unflushed dirty data
    > > then things get ugly. Hopefully a fsck -f will be able to pull
    > > something useful out of the mess. (This is where you might want to be
    > > running Ext3.) The name of the game is to install sufficient UPS power
    > > to get your dirty ramdisk data onto stable storage this time, every
    > > time.
    >
    > Ext3 is only going to help you if the ramdisk writeback respects barriers
    > and ordering rules ?

    I was alluding to to e2fsck's amazing repair ability, not ext3's journal.

    > > * Previously saved data must be reloaded into the ramdisk on startup.
    >
    > /bin/cp from initrd

    But that does not satisfy the requirement you snipped:

    * Applications need to be able to read and write ramback data during
    initial loading.

    > > * Cannot transfer directly between ramdisk and backing store, so must
    > > first transfer into memory then relaunch to destination.
    >
    > Why not - providing you clear the dirty bit before the write and you
    > check it again after ? And on the disk size as you are going to have to

    More accurately: in general, cannot transfer directly. The ramdisk may
    be external and not present a memory interface. Even an external
    ramdisk with a memory interface (the Violin box has this) would require
    extra programming to maintain cache consistency. Then there is the
    issue of ramdisks on the way that exceed the 40 bit physical addressing
    of current generation processors.

    Even for the simple case where the ramdisk is just part of the kernel
    unified cache, I would rather not go delving into that code when these
    transfers are on the slow path anyway. Application IO does its normal
    single copy_to/from_user thing. If somebody wants to fiddle with vm,
    the place to attack is right there. The copy_to/from_user can be
    eliminated (provided alignment requirements are met) using stupid page
    table tricks. In spite of Linus claiming there is no performance win
    to be had, I would like to see that put to the test.

    > suck all the content back in presumably a log structure is not a big
    > concern ?

    Sorry, I failed to parse that.

    > > * Per chunk locking is not feasible for a terabyte scale ramdisk.
    >
    > And we care 8) ?

    "640K should be enough for anyone"

    http://www.violin-memory.com/products/violin1010.html <- 504 GB ramdisk

    > > * Handle chunk size other than PAGE_SIZE.
    >
    > If you are prepared to go bigger than the fs chunk size so lose the
    > ordering guarantees your chunk size really ought to be *big* IMHO

    The finer the granularity the faster the ramdisk syncs to backing
    store. The only attraction of coarse granularity I know of is
    shrinking the bitmap, which is currently not so big that it presents
    a problem.

    Your comment re fs chunk size reveals that I have failed to
    communicate the most basic principle of the ramback design: the
    backing store is not expected to represent a consistent filesystem
    state during normal operation. Only the ramdisk needs to maintain a
    consistent state, which I have taken care to ensure. You just need
    to believe in your battery, Linux and the hardware it runs on. Which
    of these do you mistrust?

    Regards,

    Daniel


    \
     
     \ /
      Last update: 2008-03-11 04:53    [W:4.062 / U:0.056 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site