lkml.org 
[lkml]   [2005]   [Aug]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    SubjectRe: Where is the performance bottleneck?
    From
    Date
    join the party. ;)

    8 400GB SATA disk on same Marvel 8 port PCIX-133 card. P4 CPU.
    Supermicro SCT board.

    # cat /proc/mdstat
    Personalities : [linear] [raid0] [raid1] [raid5] [multipath] [raid6]
    [raid10] [faulty]
    md0 : active raid0 sdh[7] sdg[6] sdf[5] sde[4] sdd[3] sdc[2] sdb[1] sda
    [0]
    3125690368 blocks 64k chunks

    8 DISK RAID0 from same slot and card. Stripe size is 512KB.

    run oread

    # vmstat 1
    procs -----------memory---------- ---swap-- -----io---- --system-- ----
    cpu----
    r b swpd free buff cache si so bi bo in cs us sy
    id wa
    1 1 0 533216 330424 11004 0 0 7128 1610 1069 77 0 2
    95 3
    1 0 0 298464 560828 11004 0 0 230404 0 2595 1389 1
    23 0 76
    0 1 0 64736 792248 11004 0 0 231420 0 2648 1342 0
    26 0 74
    1 0 0 8948 848416 9696 0 0 229376 0 2638 1337 0
    29 0 71
    0 0 0 868896 768 9696 0 0 29696 48 1224 162 0 19
    73 8

    # time ./oread /dev/md0

    real 0m6.595s
    user 0m0.004s
    sys 0m0.151s

    run dd

    # vmstat 1
    procs -----------memory---------- ---swap-- -----io---- --system-- ----
    cpu----
    r b swpd free buff cache si so bi bo in cs us sy
    id wa
    2 2 0 854008 2932 17108 0 0 7355 1606 1071 80 0 2
    95 3
    0 2 0 848888 3112 21388 0 0 164332 0 2985 3564 2
    7 0 91
    0 2 0 844024 3260 25664 0 0 164040 0 2990 3665 1
    7 0 92
    0 2 0 840328 3380 28920 0 0 164272 0 2932 3791 1
    9 0 90
    0 2 0 836360 3500 32232 0 0 163688 100 3001 5045 2
    7 0 91
    0 2 0 831432 3644 36612 0 0 164120 568 2977 3843 0
    9 0 91
    0 1 0 826056 3752 41688 0 0 7872 0 1267 1474 1 3
    0 96

    # time dd if=/dev/md0 of=/dev/null bs=131072 count=8192
    8192+0 records in
    8192+0 records out

    real 0m4.771s
    user 0m0.005s
    sys 0m0.973s

    so the reasonable thing here is because of O_DIRECT, the sys time
    reduced a lot.

    but the time is longer! the reason i found is...

    i attached a new oread.c which allow to set block size of each read and
    total read count. so i read full strip once a time,

    # time ./oread /dev/md0 524288 2048

    real 0m4.950s
    user 0m0.000s
    sys 0m0.131s

    compared to

    # time ./oread /dev/md0 131072 8192

    real 0m6.633s
    user 0m0.002s
    sys 0m0.191s


    but still, I can get linear speed at 4 DISKS, then no speed gain when
    adding more disk into the RAID.

    Ming

    #include <stdio.h>
    #include <stdlib.h>
    #define __USE_GNU
    #include <fcntl.h>
    #include <stdlib.h>
    #include <unistd.h>

    #define ALIGN(buf) (char *) (((unsigned long) (buf) + 4095) & ~(4095))

    int main(int argc, char *argv[])
    {
    char *p;
    int fd, i;
    int BS, BLOCKS;

    if (argc < 4) {
    printf("%s: <dev> bs cnt\n", argv[0]);
    return 1;
    }

    BS = atoi(argv[2]);
    BLOCKS = atoi(argv[3]);
    fd = open(argv[1], O_RDONLY | O_DIRECT);
    if (fd == -1) {
    perror("open");
    return 1;
    }

    p = ALIGN(malloc(BS + 4095));
    for (i = 0; i < BLOCKS; i++) {
    int r = read(fd, p, BS);

    if (r == BS)
    continue;
    else {
    if (r == -1)
    perror("read");

    break;
    }
    }

    return 0;
    }
    \
     
     \ /
      Last update: 2005-08-31 20:55    [W:0.024 / U:95.512 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site