Messages in this thread | | | Date | Wed, 18 Jul 2007 16:49:21 +0200 | From | "J.A. Magallón" <> | Subject | Re: Slow Soft-RAID 5 performance |
| |
On Wed, 18 Jul 2007 10:56:11 +0100, Rui Santos <rsantos@grupopie.com> wrote:
> Hi, > > I'm getting a strange slow performance behavior on a recently installed > Server. Here are the details: > ... > > I can get a write throughput of 60 MB/sec on each HD by issuing the > command 'time `dd if=/dev/zero of=test.raw bs=4k count=$(( 1024 * 1024 / > 4 )); sync`' > ... > > The RAID device I'm testing on is /dev/md2. Now, by issuing the same > command 'dd if=/dev/zero of=test.raw bs=4k count=$(( 1024 * 1024 / 4 )); > sync`' on the raid device mount point, I get the following speeds: > With stripe_cache_size at default '265': 51 MB/sec > With stripe_cache_size at '8192': 73 MB/sec >
I know many people consider this stupid, but can you post some hdparm -tT data ? The culprit can be the filesystem+pagecache, the md driver or the disk driver, so I think trying just hdparm will show if the disk o md are going nuts...
In my case, I have a box with 2 raids, one with SCSI disks and one with IDE ones.
Some results:
lsscsi: [0:0:0:0] disk IBM DDYS-T18350N S96H /dev/sda [2:0:0:0] disk SEAGATE ST336807LW 0C01 /dev/sdb [2:0:1:0] disk SEAGATE ST336807LW 0C01 /dev/sdc [2:0:2:0] disk SEAGATE ST336807LW 0C01 /dev/sdd [2:0:3:0] disk SEAGATE ST336807LW 0C01 /dev/sde [3:0:0:0] disk ATA ST3120022A 3.06 /dev/sdf [3:0:1:0] cd/dvd HL-DT-ST DVDRAM GSA-4040B A300 /dev/sr0 [4:0:0:0] disk ATA ST3120022A 3.76 /dev/sdg
/dev/md0: Version : 00.90.03 Creation Time : Mon Jun 18 13:40:57 2007 Raid Level : raid5 Array Size : 107522304 (102.54 GiB 110.10 GB) Used Dev Size : 35840768 (34.18 GiB 36.70 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Persistence : Superblock is persistent
Update Time : Wed Jul 18 13:31:22 2007 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0
Layout : left-symmetric Chunk Size : 256K
UUID : 51ad72a7:a4d20d15:0f3ea3a1:5ccb49a0 Events : 0.2
Number Major Minor RaidDevice State 0 8 17 0 active sync /dev/sdb1 1 8 33 1 active sync /dev/sdc1 2 8 49 2 active sync /dev/sdd1 3 8 65 3 active sync /dev/sde1
This is, four scsi disks on a Adaptec U320, doing raid5:
/dev/sdb: Timing cached reads: 904 MB in 2.00 seconds = 451.84 MB/sec Timing buffered disk reads: 228 MB in 3.00 seconds = 75.90 MB/sec /dev/sdc: Timing buffered disk reads: 226 MB in 3.01 seconds = 75.01 MB/sec /dev/sdd: Timing buffered disk reads: 228 MB in 3.00 seconds = 75.88 MB/sec /dev/sde: Timing buffered disk reads: 226 MB in 3.00 seconds = 75.31 MB/sec
/dev/md0: Timing buffered disk reads: 562 MB in 3.01 seconds = 186.88 MB/sec
Nearly 75x3 = 215 Mb/s. And this looks like a small regression, I remember to have seen 200Mb on this setup on previous kernels. Performance is like 186/215 = 86%.
And /dev/md1, raid0 on 2 IDE disks:
/dev/sdf: Timing buffered disk reads: 148 MB in 3.02 seconds = 48.93 MB/sec /dev/sdg: Timing buffered disk reads: 124 MB in 3.00 seconds = 41.33 MB/sec
/dev/md1: Timing buffered disk reads: 204 MB in 3.01 seconds = 67.68 MB/sec
Performance: 67 / 90 = 75%, more or less...not too good.
Now that I read the hdparm man page, perhaps would be better to repeat the tests with hdparm --direct.
-- J.A. Magallon <jamagallon()ono!com> \ Software is like sex: \ It's better when it's free Mandriva Linux release 2008.0 (Cooker) for i586 Linux 2.6.21-jam12 (gcc 4.2.1 20070704 (4.2.1-3mdv2008.0)) SMP PREEMPT 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |