lkml.org 
[lkml]   [2016]   [Mar]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: [PATCH RFC 09/22] block, cfq: replace CFQ with the BFQ-v0 I/O scheduler
    From
    Hi Tejun,

    I'm doing a summary of this discussion as a part of presenting
    Linaro's involvement in Paolo's work. So I try to understand things.

    What I'm happy to see is a convergence in the discussion on
    timeslicing vs budgeting+time slicing, AFAICT there is consensus
    that whatever tool does the job best should be employed, despite
    some remaining disagreement on the scheduling unit (time or
    bandwidth).

    It also seems that BFQ delivers. It remains to be seen if it always
    delivers.

    The big question is this:

    On Wed, Mar 2, 2016 at 1:46 AM, Tejun Heo <tj@kernel.org> wrote:

    > For use cases where the differences in seekiness across workloads are
    > accidental - e.g. all are trying to stream different files but some
    > files are more fragmented by accident - using bandwidth as the
    > resource unit would be helpful in mitigating the random gaps that the
    > user shouldn't be bothered by, but that'd be focusing on a pretty
    > narrow set of use cases.
    >
    > Workloads are varied and underlying device performs wildly differently
    > depending on the specific IO pattern.

    What strikes me from the overall discussion is the reference to use
    cases, which ones are interesting for the I/O scheduler developers
    and which are interesting for users?

    I guess the current assumption is that CFQ address the needs of
    the average user. The special deadline scheduler is for a special
    class of users, not average. The noop scheduler is for testing.

    So Paolo was asked to develop BFQ as a replacement plug-in for
    CFQ on the assumption that it would better meet the needs of
    the average user. He also already prepared a version that is a
    selectable class like the deadline, but that was discouraged.

    The notion you bring up, that BFQ may only address a narrow
    set of use cases would merit it to be merged as a scheduler class
    for special use cases alongside the deadline scheduler. (The only
    objection I could see would be that it's a big piece of code to
    maintain.)

    But if it indeed meets the expectations of the "average user" the
    current approach would be correct.

    To me it seems it's very hard to proceed to conclude whether to
    replace CFQ with BFQ or make it a special-case class without
    first agreeing on what the average user use case(s) really is.
    Does anyone really know? All references in this communication
    seems rather handwavy.

    What I know is that Paolo & his colleagues made a test suite for
    BFQ that they are using, and I replicated their results with that
    suite:
    https://github.com/Algodev-github/S

    As you can see it uses FIO to generate some (specified)
    background noise and then tests some use cases such as:
    - File copy
    - Kernel development (git, compiles)
    - Video playback
    - Video streaming
    - Startup of applications

    Apart from the obvious that kernel devs may naively take themselves
    as reference: git+compiles, that must be the most important
    use case, and yes I noted my git checkouts are smackier with
    BFQ but that's a subjective measure. This probably means the
    average command line user issuing stuff to the left and right will
    be happy with BFQ, especially since it has this boost for newly
    forked processes.

    There are geeksy usecases: for example video playback
    on a set of random readers+writers trivially simulates someone
    playing back 1080p video while downloading another one using
    bittorrent in the background. That's geeky, but is it relevant for the
    average user?

    A NAS user for example (these boxes all run Linux more or less):
    sequential reading and writing of large files is what they mostly do.
    If the user downloads torrents onto the NAS, it's the former geeky
    usecase.

    But what is the usecase for a Google server, a NYSE transaction
    system or an Oracle database? Can the people running these loads
    tell us how to synthesize their typical workload using FIO and play
    back something that looks like what they are doing? Is there a place
    with these workload recipes that I'm obviously missing because of
    being new to the block layer?

    What would be helpful would be if people tested their favourite
    workloads before and after this patch set and reported results.

    My colleagues tried some tests on Android. It appears Android
    starts up applications quicker (with cold cache) when there is
    workload of dd-type sequential readers/writers in the background.
    This corresponds to a smartphone user starting Candy Crush while
    updating an application in the background, a usecase where Android
    is known to be somewhat unresponsive. There are millions of
    Android users, so this matters to them, we can add these seconds
    up to years of people's lives across the userbase. (This claim may
    need to be verified, but AFAICT it is genuine.)

    So: can we figure out a set of FIO-based use cases that will
    correspond to "average user"? That would help us determine
    whether BFQ is addressing that or something fringe, so it needs
    to be a special-case scheduler.

    Yours,
    Linus Walleij

    \
     
     \ /
      Last update: 2016-03-04 19:01    [W:4.119 / U:0.448 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site