lkml.org 
[lkml]   [2009]   [Sep]   [15]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    Subjectioband: Writer starves reader even without competitors (Re: Regarding dm-ioband tests)
    On Tue, Sep 08, 2009 at 01:54:00PM -0400, Vivek Goyal wrote:

    [..]
    > I ran a test to show how readers can be starved in certain cases. I launched
    > one reader and three writers. I ran this test twice. First without dm-ioband
    > and then with dm-ioband.
    >
    > Following are few lines from the script to launch readers and writers.
    >
    > **************************************************************
    > sync
    > echo 3 > /proc/sys/vm/drop_caches
    >
    > # Launch writers on sdd2
    > dd if=/dev/zero of=/mnt/sdd2/writezerofile1 bs=4K count=262144 &
    >
    > # Launch writers on sdd1
    > dd if=/dev/zero of=/mnt/sdd1/writezerofile1 bs=4K count=262144 &
    > dd if=/dev/zero of=/mnt/sdd1/writezerofile2 bs=4K count=262144 &
    >
    > echo "sleeping for 5 seconds"
    > sleep 5
    >
    > # launch reader on sdd1
    > time dd if=/mnt/sdd1/testzerofile1 of=/dev/zero &
    > echo "launched reader $!"
    > *********************************************************************
    >
    > Without dm-ioband, reader finished in roughly 5 seconds.
    >
    > 289533952 bytes (290 MB) copied, 5.16765 s, 56.0 MB/s
    > real 0m5.300s
    > user 0m0.098s
    > sys 0m0.492s
    >
    > With dm-ioband, reader took, more than 2 minutes to finish.
    >
    > 289533952 bytes (290 MB) copied, 122.386 s, 2.4 MB/s
    >
    > real 2m2.569s
    > user 0m0.107s
    > sys 0m0.548s
    >
    > I had created ioband1 on /dev/sdd1 and ioband2 on /dev/sdd2 with weights
    > 200 and 100 respectively.

    Hi Ryo,

    I notice that with-in a single ioband device, a single writer starves the
    reader even without any competitor groups being present.

    I ran following two tests with and without dm-ioband devices

    Test1
    ====
    Try to use fio for a sequential reader job. First fio will lay out the
    file and do write operation. While those writes are going on, try to do
    ls on that partition and observe latency of ls operation.

    with dm-ioband (ls test)
    ------------------------
    # cd /mnt/sdd2
    # time ls

    real 0m9.483s
    user 0m0.000s
    sys 0m0.002s

    without dm-ioband (ls test)
    ---------------------------
    # cd /mnt/sdd2
    # time ls

    256M-file1 256M-file5 2G-file1 2G-file5 writefile1 writezerofile
    256M-file2 256M-file6 2G-file2 files writefile2
    256M-file3 256M-file7 2G-file3 fio writefile3
    256M-file4 256M-file8 2G-file4 lost+found writefile4

    real 0m0.067s
    user 0m0.000s
    sys 0m0.002s

    Notice the time simle "ls" operation took in two cases.


    Test2
    =====
    Same case where fio is laying out a file and then try to read some small
    files on that partition at the interval of 1 second.

    small file read with dm-ioband
    ------------------------------
    [root@chilli fairness-tests]# ./small-file-read.sh
    file # 0, plain reading it took: 0.24 seconds
    file # 1, plain reading it took: 13.40 seconds
    file # 2, plain reading it took: 6.27 seconds
    file # 3, plain reading it took: 13.84 seconds
    file # 4, plain reading it took: 5.63 seconds


    small file read with-out dm-ioband
    ==================================
    [root@chilli fairness-tests]# ./small-file-read.sh
    file # 0, plain reading it took: 0.04 seconds
    file # 1, plain reading it took: 0.03 seconds
    file # 2, plain reading it took: 0.04 seconds
    file # 3, plain reading it took: 0.03 seconds
    file # 4, plain reading it took: 0.03 seconds

    Notice how small file read latencies have shot up.

    Looks like a single writer is completely starving a reader even without
    any IO going in any of the other groups.

    setup
    =====
    I created an two ioband device of weight 100 each on partition /dev/sdd1
    and /dev/sdd2 respectively. I am doing IO only on partition /dev/sdd2
    (ioband2).

    Following is fio job script.

    [seqread]
    runtime=60
    rw=read
    size=2G
    directory=/mnt/sdd2/fio/
    numjobs=1
    group_reporting


    Following is small file read script.

    echo 3 > /proc/sys/vm/drop_caches

    for ((i=0;i<5;i++)); do
    printf "file #%4d, plain reading it took: " $i
    /usr/bin/time -f "%e seconds" cat /mnt/sdd2/files/$i >/dev/null
    sleep 1
    done


    Thanks
    Vivek



    \
     
     \ /
      Last update: 2009-09-16 01:39    [W:0.027 / U:1.856 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site