Messages in this thread | | | From | Ian Godin <> | Subject | Re: Drive performance bottleneck | Date | Thu, 3 Feb 2005 10:08:24 -0800 |
| |
On Feb 3, 2005, at 9:40 AM, Nuno Silva wrote:
> Ian Godin wrote: >> I am trying to get very fast disk drive performance and I am seeing >> some interesting bottlenecks. We are trying to get 800 MB/sec or >> more (yes, that is megabytes per second). We are currently using >> PCI-Express with a 16 drive raid card (SATA drives). We have >> achieved that speed, but only through the SG (SCSI generic) driver. >> This is running the stock 2.6.10 kernel. And the device is not >> mounted as a file system. I also set the read ahead size on the >> device to 16KB (which speeds things up a lot): > > I was trying to reproduce but got distracted by this: > (use page down, if you just want to see the odd result) > > puma:/tmp/dd# sg_map > /dev/sg0 /dev/sda > /dev/sg1 /dev/sdb > /dev/sg2 /dev/scd0 > /dev/sg3 /dev/sdc > puma:/tmp/dd# time sg_dd if=/dev/sg1 of=/tmp/dd/sg1 bs=64k count=1000 > Reducing read to 64 blocks per loop > 1000+0 records in > 1000+0 records out > > real 0m0.187s > user 0m0.001s > sys 0m0.141s > puma:/tmp/dd# time dd if=/dev/sdb of=/tmp/dd/sdb bs=64k count=1000 > 1000+0 records in > 1000+0 records out > 65536000 bytes transferred in 1.203468 seconds (54455956 bytes/sec) > > real 0m1.219s > user 0m0.001s > sys 0m0.138s > puma:/tmp/dd# ls -l > total 128000 > -rw-r--r-- 1 root root 65536000 Feb 3 17:16 sdb > -rw-r--r-- 1 root root 65536000 Feb 3 17:16 sg1 > puma:/tmp/dd# md5sum * > ec31224970ddd3fb74501c8e68327e7b sdb > 60d4689227d60e6122f1ffe0ec1b2ad7 sg1 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > See? dd from sdb is not the same as sg1! Is this supposed to happen? > > About the 900MB/sec: > This same sg1 (= sdb, which is a single hitachi sata hdd) performes > like this: > > puma:/tmp/dd# time sg_dd if=/dev/sg1 of=/dev/null bs=64k count=1000000 > time=1 > Reducing read to 64 blocks per loop > time to transfer data was 69.784784 secs, 939.12 MB/sec > 1000000+0 records in > 1000000+0 records out > > real 1m9.787s > user 0m0.063s > sys 0m58.115s > > I can assure you that this drive can't do more than 60MB/sec sustained. > > My only conclusion is that sg (or sg_dd) is broken? ;) > > Peace, > Nuno Silva >
Definitely have been able to repeat that here, so the SG driver definitely appears to be broken. At least I'm glad I am not going insane, I was starting to wonder :)
I'll run some more tests with O_DIRECT and such things, see if I can figure out what the REAL max speed is.
Thanks for the help everyone, Ian.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |