Messages in this thread | | | Date | Tue, 29 Jan 2008 17:15:09 -0800 | From | Vu Pham <> | Subject | Re: [Scst-devel] Integration of SCST in the mainstream Linux kernel |
| |
FUJITA Tomonori wrote: > On Tue, 29 Jan 2008 13:31:52 -0800 > Roland Dreier <rdreier@cisco.com> wrote: > >> > . . STGT read SCST read . STGT read SCST read . >> > . . performance performance . performance performance . >> > . . (0.5K, MB/s) (0.5K, MB/s) . (1 MB, MB/s) (1 MB, MB/s) . >> > . iSER (8 Gb/s network) . 250 N/A . 360 N/A . >> > . SRP (8 Gb/s network) . N/A 421 . N/A 683 . >> >> > On the comparable figures, which only seem to be IPoIB they're showing a >> > 13-18% variance, aren't they? Which isn't an incredible difference. >> >> Maybe I'm all wet, but I think iSER vs. SRP should be roughly >> comparable. The exact formatting of various messages etc. is >> different but the data path using RDMA is pretty much identical. So >> the big difference between STGT iSER and SCST SRP hints at some big >> difference in the efficiency of the two implementations. > > iSER has parameters to limit the maximum size of RDMA (it needs to > repeat RDMA with a poor configuration)? > > > Anyway, here's the results from Robin Humble: > > iSER to 7G ramfs, x86_64, centos4.6, 2.6.22 kernels, git tgtd, > initiator end booted with mem=512M, target with 8G ram > > direct i/o dd > write/read 800/751 MB/s > dd if=/dev/zero of=/dev/sdc bs=1M count=5000 oflag=direct > dd of=/dev/null if=/dev/sdc bs=1M count=5000 iflag=direct >
Both Robin (iser/stgt) and Bart (scst/srp) using ramfs
Robin's numbers come from DDR IB HCAs
Bart's numbers come from SDR IB HCAs: Results with /dev/ram0 configured as backing store on the target (buffered I/O): Read Write Read Write performance performance performance performance (0.5K, MB/s) (0.5K, MB/s) (1 MB, MB/s) (1 MB, MB/s) STGT + iSER 250 48 349 781 SCST + SRP 411 66 659 746
Results with /dev/ram0 configured as backing store on the target (direct I/O): Read Write Read Write performance performance performance performance (0.5K, MB/s) (0.5K, MB/s) (1 MB, MB/s) (1 MB, MB/s) STGT + iSER 7.9 9.8 589 647 SCST + SRP 12.3 9.7 811 794
http://www.mail-archive.com/linux-scsi@vger.kernel.org/msg13514.html
Here are my numbers with DDR IB HCAs, SCST/SRP 5G /dev/ram0 block_io mode, RHEL5 2.6.18-8.el5
direct i/o dd write/read 1100/895 MB/s dd if=/dev/zero of=/dev/sdc bs=1M count=5000 oflag=direct dd of=/dev/null if=/dev/sdc bs=1M count=5000 iflag=direct
buffered i/o dd write/read 950/770 MB/s dd if=/dev/zero of=/dev/sdc bs=1M count=5000 dd of=/dev/null if=/dev/sdc bs=1M count=5000
So when using DDR IB hcas:
stgt/iser scst/srp direct I/O 800/751 1100/895 buffered I/O 1109/350 950/770
-vu > http://www.mail-archive.com/linux-scsi@vger.kernel.org/msg13502.html > > I think that STGT is pretty fast with the fast backing storage. > > > I don't think that there is the notable perfornace difference between > kernel-space and user-space SRP (or ISER) implementations about moving > data between hosts. IB is expected to enable user-space applications > to move data between hosts quickly (if not, what can IB provide us?). > > I think that the question is how fast user-space applications can do > I/Os ccompared with I/Os in kernel space. STGT is eager for the advent > of good asynchronous I/O and event notification interfances. > > > One more possible optimization for STGT is zero-copy data > transfer. STGT uses pre-registered buffers and move data between page > cache and thsse buffers, and then does RDMA transfer. If we implement > own caching mechanism to use pre-registered buffers directly with (AIO > and O_DIRECT), then STGT can move data without data copies. > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Scst-devel mailing list > Scst-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scst-devel >
| |