Messages in this thread | | | Date | Wed, 19 Jan 2005 10:34:30 +1100 | From | Nick Piggin <> | Subject | Re: pipe performance regression on ia64 |
| |
Linus Torvalds wrote: > > On Tue, 18 Jan 2005, Luck, Tony wrote: > >>David Mosberger: >> >>>So, when we run bw_pipe on a low load SMP machine, the kernel running in >>>a way load balancer always trying to spread out 2 processes while the >>>wake_up_interruptible_sync() is always trying to draw them back into >>>1 cpu. >>> >>>Linus's patch will reduce the change to call wake_up_interruptible_sync() >>>a lot. >>> >>>For bw_pipe writer or reader, the buffer size is 64k. In a 16k page >>>kernel. The old kernel will call wake_up_interruptible_sync 4 times but >>>the new kernel will call wakeup only 1 time. > > > Yes, it will depend on the buffer size, and on whether the writer actually > does any _work_ to fill it, or just writes it. > > The thing is, in real life, the "wake_up()" tends to be preferable, > because even though we are totally synchronized on the pipe semaphore > (which is a locking issue in itself that might be worth looking into), > most real loads will actually do something to _generate_ the write data in > the first place, and thus you actually want to spread the load out over > CPU's. > > The lmbench pipe benchmark is kind of special, since the writer literally > does nothing but write and the reader does nothing but read, so there is > nothing to parallellize. > > The "wake_up_sync()" hack only helps for the special case where we know > the writer is going to write more. Of course, we could make the pipe code > use that "synchronous" write unconditionally, and benchmarks would look > better, but I suspect it would hurt real life. > > The _normal_ use of a pipe, after all, is having a writer that does real > work to generate the data (like 'cc1'), and a sink that actually does real > work with it (like 'as'), and having less synchronization is a _good_ > thing. > > I don't know how to make the benchmark look repeatable and good, though. > The CPU affinity thing may be the right thing. >
Regarding scheduler balancing behaviour:
The problem could also be magnified in recent -bk kernels by the "wake up to an idle CPU" code in sched.c:try_to_wake_up(). To turn this off, remove SD_WAKE_IDLE from include/linux/topology.h:SD_CPU_INIT and include/asm/topology.h:SD_NODE_INIT
David I remember you reporting a pipe bandwidth regression, and I had a patch for it, but that hurt other workloads, so I don't think we ever really got anywhere. I've recently begun having another look at the multiprocessor balancer, so hopefully I can get a bit further with it this time.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |