Messages in this thread | | | Date | Fri, 14 Mar 2008 09:41:12 -0400 | From | "Alan D. Brunelle" <> | Subject | Re: IO CPU affinity test results |
| |
Jens Axboe wrote: > On Fri, Mar 14 2008, Jens Axboe wrote: >> I think that is encouraging, for such a small setup. The make results >> are particularly nice. The hangs are a bother, I have no good ideas on >> why the occur. The fact that it happens on both archs indicates that >> this is perhaps a generic problem, which is good. The code to support >> this is relatively simple, so it should be possible to go over it with a >> fine toothed comb and see if anything shows up. >> >> You didn't get any watchdog triggers on the serial console, or anything >> like that? > > Here's something that may explain it - if interrupts aren't disabled > when generic_smp_call_function_single() is called, we could deadlock > on the dst->lock. I think that the IPI invoke will have interrupt > disabled, but I'm not 100% certain. > > Can you see if this passes the muster? > > diff --git a/kernel/smp.c b/kernel/smp.c > index 852abd3..65808df 100644 > --- a/kernel/smp.c > +++ b/kernel/smp.c > @@ -24,12 +24,13 @@ void __cpuinit generic_init_call_single_data(void) > void generic_smp_call_function_single_interrupt(void) > { > struct call_single_queue *q; > + unsigned long flags; > LIST_HEAD(list); > > q = &__get_cpu_var(call_single_queue); > - spin_lock(&q->lock); > + spin_lock_irqsave(&q->lock); > list_replace_init(&q->list, &list); > - spin_unlock(&q->lock); > + spin_unlock_irqrestore(&q->lock); > > while (!list_empty(&list)) { > struct call_single_data *data; >
Well, putting in these two small patches (/always/ invoke send_IPI_single + this one, modified to include flags on the two spin calls), things look better after two runs (without profiling enabled), this was done on the 4-way ia64 box:
Part RQ MIN AVG MAX Dev ----- -- ------ ------ ------ ------ mkfs 0 18.151 18.311 18.472 0.227 mkfs 1 18.420 18.456 18.492 0.051
untar 0 18.260 18.420 18.581 0.228 untar 1 18.594 19.477 20.360 1.249
make 0 23.730 24.149 24.567 0.592 make 1 23.401 23.599 23.797 0.280
comb 0 60.141 60.881 61.620 1.046 comb 1 60.810 61.532 62.253 1.020
psys 0 4.16% 4.17% 4.18% 0.011 psys 1 3.80% 3.93% 4.06% 0.180
Will do a longer set of runs and report out on that. As you had noticed earlier Jens, the make results look good...
Alan
| |