Messages in this thread |  | | Date | Tue, 8 Jan 2002 00:39:53 +0100 | | From | Francois Romieu <> | | Subject | RAID1 oddity on 2.4.17+ |
| |
Greetings,
hda: IC35L060AVER07-0, ATA DISK drive hdc: IC35L060AVER07-0, ATA DISK drive hde: IC35L060AVER07-0, ATA DISK drive hdg: IC35L060AVER07-0, ATA DISK drive
$ lspci 00:00.0 Host bridge: Intel Corporation 82845 845 (Brookdale) Chipset Host Bridge (rev 03) 00:01.0 PCI bridge: Intel Corporation 82845 845 (Brookdale) Chipset AGP Bridge (rev 03) 00:1e.0 PCI bridge: Intel Corporation 82801BAM PCI (rev 12) 00:1f.0 ISA bridge: Intel Corporation 82801BA ISA Bridge (ICH2) (rev 12) 00:1f.1 IDE interface: Intel Corporation 82801BA IDE U100 (rev 12) 01:00.0 VGA compatible controller: ATI Technologies Inc Rage 128 PF 02:06.0 Ethernet controller: Standard Microsystems Corp [SMC] 83C170QF (rev 09) 02:07.0 Ethernet controller: Winbond Electronics Corp W89C940 02:0a.0 Ethernet controller: 3Com Corporation 3c905C-TX [Fast Etherlink] (rev 78) 02:0d.0 Multimedia audio controller: C-Media Electronics Inc CM8738 (rev 10) 02:0e.0 Unknown mass storage controller: Promise Technology, Inc. 20265 (rev 02)
$ cat /proc/interrupts CPU0 0: 89814 XT-PIC timer 1: 4 XT-PIC keyboard 2: 0 XT-PIC cascade 5: 190 XT-PIC eth2 7: 1884 XT-PIC eth0 8: 1 XT-PIC rtc 9: 20487 XT-PIC ide2, ide3 10: 3851 XT-PIC eth1 12: 20 XT-PIC PS/2 Mouse 14: 21445 XT-PIC ide0 15: 22279 XT-PIC ide1 NMI: 0 LOC: 89777 ERR: 0 MIS: 0
$ cat /proc/mdstat Personalities : [raid1] read_ahead 1024 sectors md0 : active raid1 hdc1[0] hda1[1] 1052160 blocks [2/2] [UU]
md1 : active raid1 hdc2[0] hda2[1] 2104448 blocks [2/2] [UU]
md2 : active raid1 hdc3[1] hda3[0] 56894080 blocks [2/2] [UU]
md3 : active raid1 hdg3[1] hde3[0] 56894080 blocks [2/2] [UU]
unused devices: <none>
$ dmesg | less Linux version 2.4.17 (root@se4.intra.cogenit.fr) (gcc version egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)) # 6 Sun Jan 6 17:03:37 CET 2002
$ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 1 model name : Intel(R) Pentium(R) 4 CPU 1.50GHz stepping : 2 cpu MHz : 1513.513 cache size : 256 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm bogomips : 3021.20
# partition table of /dev/hde unit: sectors
/dev/hde1 : start= 63, size= 2104452, Id=fd, bootable /dev/hde2 : start= 2104515, size= 4209030, Id=fd /dev/hde3 : start= 6313545, size=113788395, Id=fd /dev/hde4 : start= 0, size= 0, Id= 0
Ditto for hda, hdc, hdg.
CD-ROM present as hdb.
Output of this script (2.4.17) as first attachment #!/bin/sh
for i in hdc hdg; do hdparm -i /dev/$i hdparm -t /dev/$i sleep 1 done
for i in hdc3 hdg3; do time mke2fs -j -O sparse_super -T largefile -m 0 -R stride=8 -v /dev/$i sleep 1 done
sync
for i in hdc3 hdg3; do mkdir /mnt/$i mount /dev/$i /mnt/$i date; time dd if=/dev/zero of=/mnt/$i/foo bs=1024k count=1000 time sync sleep 1 done
So far, so good.
Turn hdc into hda, hdg into hde, see second attachment.
So far, so good.
Same output available on request for test-2.4.18-pre1-andrew-last-low-latency.
I issue a reboot.
Now I build RAID1 arrays as:
raiddev /dev/md2 raid-level 1 nr-raid-disks 2 chunk-size 128k persistent-superblock 1 nr-spare-disks 0 device /dev/hda3 raid-disk 0 device /dev/hdc3 raid-disk 1
raiddev /dev/md3 raid-level 1 nr-raid-disks 2 chunk-size 128k persistent-superblock 1 nr-spare-disks 0 device /dev/hde3 raid-disk 0 device /dev/hdg3 raid-disk 1
Please, see third attached file. Everything is fast.
/etc/fstab: LABEL=/ / ext3 defaults 1 1 none /dev/pts devpts gid=5,mode=620 0 0 none /proc proc defaults 0 0 none /dev/shm tmpfs defaults 0 0 /dev/md0 swap swap defaults 0 0 /dev/cdrom /mnt/cdrom iso9660 noauto,owner,kudzu,ro 0
Now I run this script: #!/bin/sh
for i in md2 md3; do time mke2fs -j -O sparse_super -T largefile -m 0 -R stride=8 -v /dev/$i sleep 1 done
sync
for i in md3 md2; do mkdir /mnt/$i mount /dev/$i /mnt/$i date; time dd if=/dev/zero of=/mnt/$i/foo bs=1024k count=1000 time sync sleep 1 done
Please, see fourth attached file for output.
Question: where does the slowdown for operations on /dev/md3 come from ?
-- Ueimor [unhandled content-type:application/octet-stream][unhandled content-type:application/octet-stream][unhandled content-type:application/octet-stream]$ cat /proc/mdstat Personalities : [raid1] read_ahead 1024 sectors md3 : active raid1 hdg3[1] hde3[0] 56894080 blocks [2/2] [UU] [>....................] resync = 4.2% (2418372/56894080) finish=24.9min speed=36422K/sec md2 : active raid1 hdc3[1] hda3[0] 56894080 blocks [2/2] [UU] [=======>.............] resync = 37.0% (21059712/56894080) finish=22.8min speed=26088K/sec md0 : active raid1 hdc1[0] hda1[1] 1052160 blocks [2/2] [UU] md1 : active raid1 hdc2[0] hda2[1] 2104448 blocks [2/2] [UU] unused devices: <none>
$ vmstat 1 procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 0 0 3 0 482908 7000 13084 0 0 14 15 1171 1655 0 8 91 0 0 2 0 482908 7000 13084 0 0 0 0 2259 3663 0 26 74 0 0 2 0 482900 7008 13084 0 0 0 28 2248 3602 0 21 79 0 0 2 0 482900 7008 13084 0 0 0 0 2266 3544 0 23 77 0 0 2 0 482900 7008 13084 0 0 0 0 2269 3509 0 14 86 0 0 2 0 482900 7008 13084 0 0 0 0 2296 3655 0 24 76 0 0 2 0 482900 7008 13084 0 0 0 0 2334 3579 0 19 81 0 0 2 0 482900 7008 13084 0 0 0 108 2208 3410 0 20 80 0 0 2 0 482900 7008 13084 0 0 0 0 2215 3493 0 19 81 0 0 2 0 482900 7008 13084 0 0 0 0 2273 3512 0 21 79 0 0 2 0 482900 7008 13084 0 0 0 0 2323 3620 0 23 77 0 0 2 0 482900 7008 13084 0 0 0 0 2302 3619 0 26 74 0 0 2 0 482900 7008 13084 0 0 0 0 2315 3645 0 17 83 0 0 2 0 482900 7008 13084 0 0 0 0 2322 3659 0 25 75 0 0 2 0 482900 7008 13084 0 0 0 0 2286 3570 0 17 83 0 0 2 0 482900 7008 13084 0 0 0 0 2263 3561 0 22 78 0 0 2 0 482900 7008 13084 0 0 0 0 2294 3608 0 18 82
$ cat /proc/mdstat Personalities : [raid1] read_ahead 1024 sectors md3 : active raid1 hdg3[1] hde3[0] 56894080 blocks [2/2] [UU] [==>..................] resync = 12.6% (7210432/56894080) finish=23.0min speed=35899K/sec md2 : active raid1 hdc3[1] hda3[0] 56894080 blocks [2/2] [UU] [========>............] resync = 43.0% (24489792/56894080) finish=20.6min speed=26176K/sec md0 : active raid1 hdc1[0] hda1[1] 1052160 blocks [2/2] [UU] md1 : active raid1 hdc2[0] hda2[1] 2104448 blocks [2/2] [UU] unused devices: <none>
$ cat /proc/mdstat Personalities : [raid1] read_ahead 1024 sectors md3 : active raid1 hdg3[1] hde3[0] 56894080 blocks [2/2] [UU] [=============>.......] resync = 67.2% (38241476/56894080) finish=10.6min speed=29192K/sec md2 : active raid1 hdc3[1] hda3[0] 56894080 blocks [2/2] [UU] [=================>...] resync = 86.1% (49040512/56894080) finish=5.2min speed=24700K/sec md0 : active raid1 hdc1[0] hda1[1] 1052160 blocks [2/2] [UU] md1 : active raid1 hdc2[0] hda2[1] 2104448 blocks [2/2] [UU] unused devices: <none>
Jan 7 23:13:51 se4 kernel: md: syncing RAID array md2 Jan 7 23:24:50 se4 kernel: md: syncing RAID array md3 Jan 7 23:50:05 se4 kernel: md: md2: sync done. Jan 7 23:56:47 se4 kernel: md: md3: sync done.
r b w swpd free buff cache si so bi bo in cs us sy id 1 1 1 0 2880 20216 385436 0 0 0 27060 590 30 0 8 92 0 1 0 0 3256 20216 385040 0 0 0 28296 594 32 0 10 90 1 0 2 0 2292 20232 386800 0 0 4 31300 586 265 0 25 75 0 1 2 0 2440 17808 388268 0 0 0 30032 586 503 0 41 59 0 1 2 0 2708 17808 388004 0 0 0 28004 602 31 0 9 91 0 1 1 0 3220 17808 387492 0 0 0 27604 595 36 0 11 89 1 0 2 0 2652 17812 389036 0 0 0 26924 608 49 0 18 82 1 0 2 0 2292 17844 388608 0 0 8 27824 480 1130 0 74 26 0 1 2 0 2292 17848 388496 0 0 0 29588 588 73 0 12 88 0 1 2 0 2420 17848 388368 0 0 0 27428 592 27 0 3 97 0 1 2 0 2796 17848 387992 0 0 0 27268 584 29 0 8 92 1 0 1 0 2292 17960 388928 0 0 4 11120 442 914 0 62 38 0 1 3 0 2516 17984 388136 0 0 0 31916 544 252 0 20 80 0 1 3 0 2772 17984 387880 0 0 0 27432 593 37 0 7 93 0 1 3 0 3028 17984 387624 0 0 0 30784 597 39 0 7 93 1 0 3 0 2356 18020 388776 0 0 0 29816 586 470 0 37 63 0 1 2 0 2576 18064 387892 0 0 4 27308 587 406 0 32 68 0 1 1 0 3296 18096 386516 0 0 0 32640 659 421 0 30 70 0 1 1 0 3296 18096 386516 0 0 0 27304 593 26 0 9 91 0 1 3 0 2452 18176 388072 0 0 0 28136 580 668 0 52 48 0 1 3 0 2708 18176 387816 0 0 0 27180 588 35 0 4 96
r b w swpd free buff cache si so bi bo in cs us sy id 0 1 2 0 3200 1852 480384 0 0 49 1580 165 50 1 5 94 0 1 2 0 3200 1852 480384 0 0 0 7824 172 14 0 3 97 0 1 2 0 3200 1852 480384 0 0 0 0 132 8 0 0 100 0 1 2 0 3200 1852 480384 0 0 0 2360 130 9 0 1 99 0 1 2 0 3200 1852 480384 0 0 0 0 130 6 0 0 100 0 1 2 0 3320 1860 480256 0 0 0 28 138 14 0 0 100 0 1 2 0 3320 1860 480256 0 0 0 7820 164 14 0 1 99 0 1 2 0 3316 1860 480256 0 0 0 0 132 10 0 1 99 0 1 2 0 3316 1860 480256 0 0 0 2356 129 9 0 1 99 0 1 2 0 3316 1860 480256 0 0 0 0 129 6 0 0 100 0 1 2 0 3316 1860 480256 0 0 0 0 135 8 0 0 100 0 1 2 0 3316 1860 480256 0 0 0 3968 130 10 0 1 99 0 1 2 0 3316 1860 480256 0 0 0 0 129 8 0 0 100 0 1 2 0 3316 1860 480256 0 0 0 7828 173 14 0 1 99 0 1 2 0 3316 1860 480256 0 0 0 0 132 6 0 1 99 0 1 3 0 3212 1876 480368 0 0 0 2704 128 181 0 9 91 0 1 2 0 3212 1876 480368 0 0 0 0 130 5 0 0 100 0 1 2 0 3212 1876 480368 0 0 0 0 134 7 0 1 99 0 1 2 0 3316 1876 480244 0 0 0 7696 166 14 0 1 99 0 1 2 0 3316 1876 480244 0 0 0 0 128 5 0 1 99 0 1 2 0 3316 1876 480244 0 0 0 2728 129 9 0 0 100
|  |