[lkml]   [2005]   [Aug]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Where is the performance bottleneck?
On Mon, Aug 29, 2005 at 06:20:56PM +0000, Holger Kiehl wrote:
> Hello
> I have a system with the following setup:
> Board is Tyan S4882 with AMD 8131 Chipset
> 4 Opterons 848 (2.2GHz)
> 8 GB DDR400 Ram (2GB for each CPU)
> 1 onboard Symbios Logic 53c1030 dual channel U320 controller
> 2 SATA disks put together as a SW Raid1 for system, swap and spares
> 8 SCSI U320 (15000 rpm) disks where 4 disks (sdc, sdd, sde, sdf)
> are on one channel and the other four (sdg, sdh, sdi, sdj) on
> the other channel.
> The U320 SCSI controller has a 64 bit PCI-X bus for itself, there is
> no other device on that bus. Unfortunatly I was unable to determine at
> what speed it is running, here the output from lspci -vv:

> How does one determine the PCI-X bus speed?

Usually only the card (in your case the Symbios SCSI controller) can
tell. If it does, it'll be most likely in 'dmesg'.

> Anyway, I thought with this system I would get theoretically 640 MB/s using
> both channels.

You can never use the full theoretical bandwidth of the channel for
data. A lot of overhead remains for other signalling. Similarly for PCI.

> I tested several software raid setups to get the best possible write
> speeds for this system. But testing shows that the absolute maximum I
> can reach with software raid is only approx. 270 MB/s for writting.
> Which is very disappointing.

I'd expect somewhat better (in the 300-400 MB/s range), but this is not
too bad.

To find where the bottleneck is, I'd suggest trying without the
filesystem at all, and just filling a large part of the block device
using the 'dd' command.

Also, trying without the RAID, and just running 4 (and 8) concurrent
dd's to the separate drives could show whether it's the RAID that's
slowing things down.

> The tests where done with kernel from, scheduler
> is the deadline and distribution is fedora core 4 x86_64 with all
> updates. Chunksize is always the default from mdadm (64k). Filesystem
> was always created with the command mke2fs -j -b4096 -O dir_index
> /dev/mdx.
> I also have tried with 2.6.13-rc7, but here the speed was much lower,
> the maximum there was approx. 140 MB/s for writting.

Now that's very low.

Vojtech Pavlik
SuSE Labs, SuSE CR
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2005-08-29 22:30    [W:0.124 / U:0.148 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site