Messages in this thread | | | Date | Fri, 03 Oct 2003 14:33:01 -0500 | From | Steven Pratt <> | Subject | Re: [Lse-tech] Re: Minutes from 10/1 LSE Call |
| |
Andrew Morton wrote:
>Steven Pratt <slpratt@austin.ibm.com> wrote: > > >> Sure, but why do I only see this is the mm tree, and not the mainline >> tree. >> >> > >Please send a full description of how to reproduce it and I'll take a look. > > > Get the latest rawread from http://www-124.ibm.com/developerworks/opensource/linuxperf/rawread/rawread.html
mkfs devices and mount on /mnt/mntN where N is increasing index. Create file 'foo' in each filesystem of size 1GB (for this example). Unmount and remount the partitions/devices to flush the cache. Filesystems are also umounted and re-mounted between each test run.
The following rawread commands will run the tests for block sizes ranging from 1k-512k. The "-d 1" parameters assumes that you mounted starting at /mnt/mnt1 and the "-m2 -p16" say to run 8 threads on each of 2 devices /mnt/mnt1 and /mnt/mnt2.
rawread -m 2 -p 16 -d 6 -n 20480 -f -c -t 0 -s 1024
rawread -m 2 -p 16 -d 6 -n 10240 -f -c -t 0 -s 2048
rawread -m 2 -p 16 -d 6 -n 5120 -f -c -t 0 -s 4096
rawread -m 2 -p 16 -d 6 -n 2560 -f -c -t 0 -s 8192
rawread -m 2 -p 16 -d 1 -n 1280 -f -c -t 0 -s 16384
rawread -m 2 -p 16 -d 1 -n 640 -f -c -t 0 -s 32768
rawread -m 2 -p 16 -d 1 -n 320 -f -c -t 0 -s 65536
rawread -m 2 -p 16 -d 1 -n 160 -f -c -t 0 -s 131072
rawread -m 2 -p 16 -d 1 -n 80 -f -c -t 0 -s 262144
rawread -m 2 -p 16 -d 1 -n 40 -f -c -t 0 -s 524288
2 devices is the smallest number I have been able to run which shows this problem. With only 1 device I did not see it. My original tests were done with 20 devices. One thing of interest is that with only 2 devices the point at which CPU starts to increase again is at 128k instead of at 32k which I saw with 20 devices. This would support your theory that this is casued by cache misses with more/larger buffers. I'm still not sure this accounts for all of the extra CPU usage, but I am less worried about it.
But as long as I have your attention, there is one other thing about these runs which bothers me, which is that the mm tree is doing horribly on 1k and 2k block sizes. I looks like readahead is not functioning properly for these requst sizes.
Here is a comparison for 2 devices between test6 and test6mm1. You can see that the mm1 tree does great at larger block sizes, but poorly at small ones.
Results:seqread-_vs_.seqread-
tolerance = 0.00 + 3.00% of A test6 test6-mm1 Blocksize KBs/sec KBs/sec %diff diff tolerance ---------- ------------ ------------ -------- ------------ ------------ 1024 44083 22641 -48.64 -21442.00 1322.49 * 2048 45276 26371 -41.76 -18905.00 1358.28 * 4096 44024 45260 2.81 1236.00 1320.72 8192 44519 50073 12.48 5554.00 1335.57 * 16384 46869 51528 9.94 4659.00 1406.07 * 32768 47900 52231 9.04 4331.00 1437.00 * 65536 42803 52183 21.91 9380.00 1284.09 * 131072 36525 49724 36.14 13199.00 1095.75 * 262144 34628 46192 33.39 11564.00 1038.84 * 524288 28997 48005 65.55 19008.00 869.91 *
Results:seqread-_vs_.seqread- tolerance = 0.50 + 3.00% of A test6 test6-mm1 Blocksize %CPU %CPU %diff diff tolerance ---------- ------------ ------------ -------- ------------ ------------ 1024 27.87 11.72 -57.95 -16.15 1.34 * 2048 13.77 8.84 -35.80 -4.93 0.91 * 4096 9 9.99 11.00 0.99 0.77 * 8192 8.07 8.31 2.97 0.24 0.74 16384 5.7 6.63 16.32 0.93 0.67 * 32768 4.93 5.59 13.39 0.66 0.65 * 65536 3.76 4.7 25.00 0.94 0.61 * 131072 3.25 4.53 39.38 1.28 0.60 * 262144 3.23 6.15 90.40 2.92 0.60 * 524288 2.97 8.19 175.76 5.22 0.59 *
Steve
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |