lkml.org 
[lkml]   [2005]   [Nov]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: 2.6.9 reporting 1 Gigabyte/second throughput on bio's, timer skew possible?
    Jens Axboe wrote:

    >On Sat, Nov 12 2005, Jeff V. Merkey wrote:
    >
    >
    >>Jens Axboe wrote:
    >>
    >>
    >>
    >>>On Fri, Nov 11 2005, Jeff V. Merkey wrote:
    >>>
    >>>
    >>>
    >>>
    >>>>I have allocated 393,216 bio buffers I statically maintain in a chain
    >>>>and am running the dsfs file system with 3 x gigabit links fully
    >>>>saturated. meta-data
    >>>>increases the write sizes to 720 MB/Second on dual 9500 controllers with
    >>>>8 drives each (total of 16) 7200 RPM Drives. I am seeing some
    >>>>congestion and bursting on the bio chains as they are submitted.
    >>>>
    >>>>
    >>>>
    >>
    >>
    >>>16 disks on 2 controllers, I'm 100% sure they are lots of people
    >>>pushing 2.6 much further than that! I wouldn't evne call that a big
    >>>setup.
    >>>
    >>>
    >>>
    >>>
    >>Probably not for this type of application.
    >>
    >>
    >>
    >>>
    >>>
    >>>>DSFS dynamically generates html status files form within the file
    >>>>system. When the system gets somewhat behind, I am seeing bursts > 1
    >>>>GB/Second which exceeds the theoretical limit of the bus. I have a
    >>>>timer function that runs every second and profiles the I/O throughput
    >>>>created by DSFS with bio submissions and captured packets. I am asking
    >>>>if there is clock skew at these data rates with use of the timer
    >>>>functions. The system appears to be sustaining 1GB/Second throughput on
    >>>>dual controllers. I have verified through data rates the system is
    >>>>sustaining 800 megabytes/second with these 1GB/S bursts. I am curious
    >>>>if there is potentially timer skew at these higher rates since I am
    >>>>having a hard time accepting that I can push 1GB/S through a bus rated
    >>>>at only 850 MB/S for DMA based transfers. The unit is accessible by
    >>>>
    >>>>
    >>>>
    >>>>
    >>>Note that the linux io stats accounting in 2.6.9 accounts queued io, not
    >>>io completions. So it's quite possible to have burst rates > bus speeds
    >>>for async io. 2.6.15-rc1 change this.
    >>>
    >>>
    >>>
    >>>
    >>>
    >>So you are willing to log into the unit and validate these numbers? I
    >>would like for an
    >>someone other than me to validate I am seeing these rates.
    >>
    >>
    >
    >If you average the bandwidth over a time long enough to eliminate the
    >bursty queueing rates, your average rage should drop to what the
    >hardware can actually do. Or dig out the patch from 2.6.15-rc1 for
    >ll_rw_blk.c and apply it to 2.6.9, find it here:
    >
    >http://kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=d72d904a5367ad4ca3f2c9a2ce8c3a68f0b28bf0;hp=d83c671fb7023f69a9582e622d01525054f23b66
    >
    >
    >
    Jens,

    Thanks. I'll dig out the patch. I am measuring the rates on the back end
    and they are running at 720-800 MB/S apart from what's being reported from
    the bio submission. At any rate, I ave to say the bio performance is
    stunning in comparison to Windows 2003.

    Jeff
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-11-13 22:30    [W:0.032 / U:0.232 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site