lkml.org 
[lkml]   [2005]   [Jan]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: pipe performance regression on ia64
Linus Torvalds wrote:
>
> On Tue, 18 Jan 2005, Luck, Tony wrote:
>
>>David Mosberger:
>>
>>>So, when we run bw_pipe on a low load SMP machine, the kernel running in
>>>a way load balancer always trying to spread out 2 processes while the
>>>wake_up_interruptible_sync() is always trying to draw them back into
>>>1 cpu.
>>>
>>>Linus's patch will reduce the change to call wake_up_interruptible_sync()
>>>a lot.
>>>
>>>For bw_pipe writer or reader, the buffer size is 64k. In a 16k page
>>>kernel. The old kernel will call wake_up_interruptible_sync 4 times but
>>>the new kernel will call wakeup only 1 time.
>
>
> Yes, it will depend on the buffer size, and on whether the writer actually
> does any _work_ to fill it, or just writes it.
>
> The thing is, in real life, the "wake_up()" tends to be preferable,
> because even though we are totally synchronized on the pipe semaphore
> (which is a locking issue in itself that might be worth looking into),
> most real loads will actually do something to _generate_ the write data in
> the first place, and thus you actually want to spread the load out over
> CPU's.
>
> The lmbench pipe benchmark is kind of special, since the writer literally
> does nothing but write and the reader does nothing but read, so there is
> nothing to parallellize.
>
> The "wake_up_sync()" hack only helps for the special case where we know
> the writer is going to write more. Of course, we could make the pipe code
> use that "synchronous" write unconditionally, and benchmarks would look
> better, but I suspect it would hurt real life.
>
> The _normal_ use of a pipe, after all, is having a writer that does real
> work to generate the data (like 'cc1'), and a sink that actually does real
> work with it (like 'as'), and having less synchronization is a _good_
> thing.
>
> I don't know how to make the benchmark look repeatable and good, though.
> The CPU affinity thing may be the right thing.
>

Regarding scheduler balancing behaviour:

The problem could also be magnified in recent -bk kernels by the
"wake up to an idle CPU" code in sched.c:try_to_wake_up(). To turn
this off, remove SD_WAKE_IDLE from include/linux/topology.h:SD_CPU_INIT
and include/asm/topology.h:SD_NODE_INIT

David I remember you reporting a pipe bandwidth regression, and I had
a patch for it, but that hurt other workloads, so I don't think we
ever really got anywhere. I've recently begun having another look at
the multiprocessor balancer, so hopefully I can get a bit further with
it this time.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 14:09    [W:0.410 / U:0.084 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site