[lkml]   [2009]   [Nov]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: [RFC] Block IO Controller V2 - some results
    Hi Vivek,
    the performance drop reported by Alan was my main concern about your
    approach. Probably you should mention/document somewhere that when the
    number of groups is too large, there is large decrease in random read

    However, we can check few things:
    * is this kernel built with HZ < 1000? The smallest idle CFQ will do
    is given by 2/HZ, so running with a small HZ will increase the impact
    of idling.

    On Tue, Nov 17, 2009 at 3:14 PM, Vivek Goyal <> wrote:
    > Regarding the reduced throughput for random IO case, ideally we should not
    > idle on sync-noidle group on this hardware as this seems to be a fast NCQ
    > supporting hardware. But I guess we might not be detecting the queue depth
    > properly which leads to idling on per group sync-noidle workload and
    > forces the queue depth to be 1.

    * This can be ruled out testing my NCQ detection fix patch

    However, my feeling is that the real problem is having multiple
    separate sync-noidle trees.
    Inter group idle is marginal, since each sync-noidle tree already has
    its end-of-tree idle enabled for rotational devices (The difference in
    the table is in fact small).
    > ---- ---- - ----------- ----------- ----------- -----------
    > Mode RdWr N base ioc off ioc no idle ioc idle
    > ---- ---- - ----------- ----------- ----------- -----------
    > rnd rd 2 17.3 17.1 9.4 9.1
    > rnd rd 4 27.1 27.1 8.1 8.2
    > rnd rd 8 37.1 37.1 6.8 7.1

    2 random readers without groups have bw = 17.3 ; this means that a
    single random reader will have bw > 8.6 (since the two readers go
    usually in parallel when no groups are involved, unless two random
    reads are actually queued to the same disk).

    When the random readers are in separate groups, we give the full disk
    to only one at a time, so the max aggregate bw achievable is the bw of
    a single random reader less the overhead proportional to number of
    groups. This is compatible with the numbers.

    So, an other thing to mention in the docs is that having one process
    per group is not a good idea (cfq already has I/O priorities to deal
    with single processes). Groups are coarse grain entities, and they
    should really be used when you need to get fairness between groups of

    * An other thing to do is to try setting rotational = 0, since even
    with NCQ correctly detected, if the device is rotational, we still
    introduce some idle delays (that are good in the root group, but not
    when you have multiple groups).

    > I am also trying to setup a higher end system here and will do some
    > experiments.
    > Thanks
    > Vivek


     \ /
      Last update: 2009-11-17 17:21    [W:0.021 / U:217.516 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site