lkml.org 
[lkml]   [2006]   [Jul]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: Solaris ZFS on Linux [Was: Re: the " 'official' point of view" expressed by kernelnewbies.org regarding reiser4 inclusion]
    Theodore Tso wrote:
    > On Mon, Jul 31, 2006 at 08:31:32PM -0500, David Masover wrote:
    >> So you use a repacker. Nice thing about a repacker is, everyone has
    >> downtime. Better to plan to be a little sluggish when you'll have
    >> 1/10th or 1/50th of the users than be MUCH slower all the time.
    >
    > Actually, that's a problem with log-structured filesystems in general.
    > There are quite a few real-life workloads where you *don't* have
    > downtime. The thing is, in a global economy, you move from the
    > London/European stock exchanges, to the New York/US exchanges, to the
    > Asian exchanges, with little to no downtime available.

    Such systems must have redundancy, however. And if you have 2-3 servers
    hot in case one of them goes down, I can see switching between which is
    more active, and which is repacking.

    This repacker is online, hence a filesystem being repacked would have to
    be less active, not necessarily down. So repack the backup server, then
    make the backup server the active one and repack the main server. If
    the main server goes down while the backup is repacking, kill the repack
    process.

    I actually have a problem imagining a system where you don't have enough
    spare capacity (disk, CPU, spare servers) to run a repacker every now
    and then, but which also must have 100% uptime. What happens when a
    disk goes bad? Or when power to half the country goes out? Or... You
    get the idea.

    > In addition,
    > people have been getting more sophisticated with workload
    > consolidation tricks so that you use your "downtime" for other
    > applications (either to service other parts of the world, or to do
    > daily summaries, 3-d frame rendering at animation companies, etc.) So
    > the assumption that there will always be time to run the repacker is a
    > dangerous one.

    3D frame rendering in particular doesn't require much disk use, does it?
    Daily summaries, I guess, depends on what kind of summaries they are.
    And anyway, those applications are making the same dangerous assumption.

    And anyway, I suspect the repacker will work best once a week or so, but
    no one knows yet, as they haven't written it yet.

    > The problem is that many benchmarks (such as taring and untaring the
    > kernel sources in reiser4 sort order) are overly simplistic, in that
    > they don't really reflect how people use the filesystem in real life.

    That's true. I'd also like to see lots more benchmarks.

    > If the benchmark doesn't take into account the need for
    > repacker, or if the repacker is disabled or fails to run during the
    > benchmark, the filesystem are in effect "cheating" on the benchmark
    > because there is critical work which is necessary for the long-term
    > health of the filesystem which is getting deferred until after the
    > benchmark has finished measuring the performance of the system under
    > test.

    In this case, the only fair test would be to run the benchmark 24/7 for
    a week, and run the repacker on a weekend. Or however you're planning
    to do it. It wouldn't be fair to run a 10-minute or 1-hour benchmark
    and then immediately run the repacker.

    But I'd also like to see more here, especially about fragmentation. If
    the repacker will cost money, the system should be reasonably good at
    avoiding fragmentation. I'm wondering if I should run a benchmark on my
    systems -- they're over a year old, and while they aren't under
    particularly heavy load, they should be getting somewhat fragmented by now.
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2006-08-01 05:41    [W:5.069 / U:0.012 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site