[lkml]   [2005]   [Aug]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: Where is the performance bottleneck?
    On Mon, Aug 29, 2005 at 06:20:56PM +0000, Holger Kiehl wrote:
    > Hello
    > I have a system with the following setup:
    > Board is Tyan S4882 with AMD 8131 Chipset
    > 4 Opterons 848 (2.2GHz)
    > 8 GB DDR400 Ram (2GB for each CPU)
    > 1 onboard Symbios Logic 53c1030 dual channel U320 controller
    > 2 SATA disks put together as a SW Raid1 for system, swap and spares
    > 8 SCSI U320 (15000 rpm) disks where 4 disks (sdc, sdd, sde, sdf)
    > are on one channel and the other four (sdg, sdh, sdi, sdj) on
    > the other channel.
    > The U320 SCSI controller has a 64 bit PCI-X bus for itself, there is
    > no other device on that bus. Unfortunatly I was unable to determine at
    > what speed it is running, here the output from lspci -vv:

    > How does one determine the PCI-X bus speed?

    Usually only the card (in your case the Symbios SCSI controller) can
    tell. If it does, it'll be most likely in 'dmesg'.

    > Anyway, I thought with this system I would get theoretically 640 MB/s using
    > both channels.

    You can never use the full theoretical bandwidth of the channel for
    data. A lot of overhead remains for other signalling. Similarly for PCI.

    > I tested several software raid setups to get the best possible write
    > speeds for this system. But testing shows that the absolute maximum I
    > can reach with software raid is only approx. 270 MB/s for writting.
    > Which is very disappointing.

    I'd expect somewhat better (in the 300-400 MB/s range), but this is not
    too bad.

    To find where the bottleneck is, I'd suggest trying without the
    filesystem at all, and just filling a large part of the block device
    using the 'dd' command.

    Also, trying without the RAID, and just running 4 (and 8) concurrent
    dd's to the separate drives could show whether it's the RAID that's
    slowing things down.

    > The tests where done with kernel from, scheduler
    > is the deadline and distribution is fedora core 4 x86_64 with all
    > updates. Chunksize is always the default from mdadm (64k). Filesystem
    > was always created with the command mke2fs -j -b4096 -O dir_index
    > /dev/mdx.
    > I also have tried with 2.6.13-rc7, but here the speed was much lower,
    > the maximum there was approx. 140 MB/s for writting.

    Now that's very low.

    Vojtech Pavlik
    SuSE Labs, SuSE CR
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2005-08-29 22:30    [W:0.025 / U:35.736 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site