lkml.org 
[lkml]   [2008]   [Mar]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: IO CPU affinity test results
Jens Axboe wrote:
> On Fri, Mar 14 2008, Jens Axboe wrote:
>> I think that is encouraging, for such a small setup. The make results
>> are particularly nice. The hangs are a bother, I have no good ideas on
>> why the occur. The fact that it happens on both archs indicates that
>> this is perhaps a generic problem, which is good. The code to support
>> this is relatively simple, so it should be possible to go over it with a
>> fine toothed comb and see if anything shows up.
>>
>> You didn't get any watchdog triggers on the serial console, or anything
>> like that?
>
> Here's something that may explain it - if interrupts aren't disabled
> when generic_smp_call_function_single() is called, we could deadlock
> on the dst->lock. I think that the IPI invoke will have interrupt
> disabled, but I'm not 100% certain.
>
> Can you see if this passes the muster?
>
> diff --git a/kernel/smp.c b/kernel/smp.c
> index 852abd3..65808df 100644
> --- a/kernel/smp.c
> +++ b/kernel/smp.c
> @@ -24,12 +24,13 @@ void __cpuinit generic_init_call_single_data(void)
> void generic_smp_call_function_single_interrupt(void)
> {
> struct call_single_queue *q;
> + unsigned long flags;
> LIST_HEAD(list);
>
> q = &__get_cpu_var(call_single_queue);
> - spin_lock(&q->lock);
> + spin_lock_irqsave(&q->lock);
> list_replace_init(&q->list, &list);
> - spin_unlock(&q->lock);
> + spin_unlock_irqrestore(&q->lock);
>
> while (!list_empty(&list)) {
> struct call_single_data *data;
>

Well, putting in these two small patches (/always/ invoke send_IPI_single + this one, modified to include flags on the two spin calls), things look better after two runs (without profiling enabled), this was done on the 4-way ia64 box:

Part RQ MIN AVG MAX Dev
----- -- ------ ------ ------ ------
mkfs 0 18.151 18.311 18.472 0.227
mkfs 1 18.420 18.456 18.492 0.051

untar 0 18.260 18.420 18.581 0.228
untar 1 18.594 19.477 20.360 1.249

make 0 23.730 24.149 24.567 0.592
make 1 23.401 23.599 23.797 0.280

comb 0 60.141 60.881 61.620 1.046
comb 1 60.810 61.532 62.253 1.020

psys 0 4.16% 4.17% 4.18% 0.011
psys 1 3.80% 3.93% 4.06% 0.180

Will do a longer set of runs and report out on that. As you had noticed earlier Jens, the make results look good...

Alan


\
 
 \ /
  Last update: 2008-03-14 14:43    [W:0.056 / U:4.492 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site