lkml.org 
[lkml]   [2009]   [Jun]   [29]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
SubjectRe: [RFC patch] Use IPI_shortcut for lapic timer broadcast
From
Date
On Sun, 2009-06-28 at 23:47 -0700, Luming Yu wrote:
> Hello,
>
> We need to use IPI shortcut to send lapic timer broadcast
> to avoid the latency of sending IPI one bye one on systems with many
> logical processors when NO_HZ is disabled.
> Without this patch,I have seen upstream kernel with RHEL 5 kernel
> config boot hang .
>
> The patch also changes physflat_send_IPI_all to IPI shortcut mode.
>
> Please review, and apply.
>
> **The patch is enclosed in text attachment*
> **Using web client to send the patch* *
> **below is for review, please apply attached patch*/
>
> Thanks,
> Luming
>
>
> Signed-off-by: Yu Luming <luming.yu@intel.com>
>
> apic.c | 4 +++-
> apic_flat_64.c | 7 ++++++-
> 2 files changed, 9 insertions(+), 2 deletions(-)
>
> --- linux-2.6.30-rc6/arch/x86/kernel/apic/apic.c.0 2009-06-28
> 20:22:55.000000000 -0600
> +++ linux-2.6.30-rc6/arch/x86/kernel/apic/apic.c 2009-06-29
> 00:21:44.000000000 -0600
> @@ -419,7 +419,9 @@
> static void lapic_timer_broadcast(const struct cpumask *mask)
> {
> #ifdef CONFIG_SMP
> - apic->send_IPI_mask(mask, LOCAL_TIMER_VECTOR);
> + if (cpus_empty(*mask))
> + return;
> + apic->send_IPI_all(LOCAL_TIMER_VECTOR);
> #endif
> }
>

Hmm.. this change looks wrong. This will cause unnecessary timer
broadcasts when nohz is actually enabled. I mean, when nohz is enabled,
we want to send this interrupt broadcast to one or some subset of CPUs
and this change sends the interrupt to all CPUs.

On a 16 logical CPUs with nohz enabled, powertop without this patch I
see
Wakeups-from-idle per second : 11.6 interval: 15.0s
and with this patch I see
Wakeups-from-idle per second : 24.7 interval: 15.0s

Also, cat /proc/interrupts | grep LOC shows this problem where all CPUs
seem to get roughly same number of local APIC interrupts over a period
of time with this patch.

We should only do the allbutself when nohz is disabled with a big
unlikely() around that code.

Thanks,
Venki

> --- linux-2.6.30-rc6/arch/x86/kernel/apic/apic_flat_64.c.0 2009-06-29
> 00:13:26.000000000 -0600
> +++ linux-2.6.30-rc6/arch/x86/kernel/apic/apic_flat_64.c 2009-06-29
> 00:11:23.000000000 -0600
> @@ -274,7 +274,12 @@
>
> static void physflat_send_IPI_all(int vector)
> {
> - physflat_send_IPI_mask(cpu_online_mask, vector);
> + if (vector == NMI_VECTOR) {
> + physflat_send_IPI_mask(cpu_online_mask, vector);
> + } else {
> + __default_send_IPI_shortcut(APIC_DEST_ALLINC,
> + vector, apic->dest_logical);
> + }
> }
>
> static unsigned int physflat_cpu_mask_to_apicid(const struct cpumask *cpumask)



\
 
 \ /
  Last update: 2009-06-29 22:39    [W:0.042 / U:4.884 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site