Messages in this thread | | | Date | Sat, 02 Jun 2007 18:12:03 -0400 | From | Bill Davidsen <> | Subject | Re: scheduling oddity on 2.6.20.3 stock |
| |
David Schwartz wrote: >>>>> bunzip2 -c $file.bz2 |gzip -9 >$file.gz > > So here are some actual results from a dual P3-1Ghz machine (2.6.21.1, > CFSv9). First lets time each operation individually: > > $ time bunzip2 -k linux-2.6.21.tar.bz2 > > real 1m5.626s > user 1m2.240s > sys 0m3.144s > > > $ time gzip -9 linux-2.6.21.tar > > real 1m17.652s > user 1m15.609s > sys 0m1.912s > > The compress was the most complex (no surprise there) but they are close > enough that efficient overlap will definitely affect the total wall time. If > we can both decompress and compress in 1:17, we are optimal. First, let's > try the normal way: > > $ time (bunzip2 -c linux-2.6.21.tar.bz2 | gzip -9 > test1) > > real 1m45.051s > user 2m16.945s > sys 0m2.752s > > 1:45, or 1/3 over optimal. Now, with a 32MB non-blocking cache between the > two processes ('accel' creates a 32MB cache and uses 'select' to fill from > stdin and empty to stdout without blocking either direction): > > $ time (bunzip2 -c linux-2.6.21.tar.bz2 | ./accel | gzip -9 > test2) > > real 1m18.361s > user 2m19.589s > sys 0m6.356s > > Within testing accuracy of optimal. > > So it's not the scheduler. It's the fact that bunzip2/gzip have inadequate > input/output buffering. I don't think it's unreasonable to consider this a > defect in those programs. > They are hardly designed to optimize this operation...
For a tunable buffer program allowing the buffer size and buffers in the pool to be set, see www.tmr.com/~public/source program ptbuf. I wrote it as a proof of concept for a pthreads presentation I was giving, and it happened to be useful.
-- Bill Davidsen <davidsen@tmr.com> "We have more to fear from the bungling of the incompetent than from the machinations of the wicked." - from Slashdot - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |