lkml.org 
[lkml]   [2003]   [Apr]   [3]   [last100]   RSS Feed
Views: [more markup]  [less markup]  [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
DateThu, 3 Apr 2003 03:07:23 -0500 (EST)
FromZwane Mwaikambo <>
Subject[PATCH][2.5] smp_call_function needs mb() - oopsable
I have a 3 Processor Pentium 133 system w/ 512k external cache which is 
oopsing reliably in the exact same location. The problem is that the 
current memory barrier in smp_call_function is a simple gcc optimisation 
barrier, we really need a memory barrier there so that the other cpu's 
get the updated value when they get IPI'd immediately afterwards. The 
sequence looks like this;

cpu0					cpu1	
smp_call_function() {			...
	...				...
	call_data = &data		...
	send_IPI()			smp_call_function_interrupt() {
						func = call_data->func <== [1]
[1] at this point cpu1 loads memory, which hasn't been written back yet, 
to a local register.

0xc0114e30 <smp_call_function_interrupt>:       push   %esi
0xc0114e31 <smp_call_function_interrupt+1>:     mov    0xc02e9830,%eax
0xc0114e36 <smp_call_function_interrupt+6>:     push   %ebx
0xc0114e37 <smp_call_function_interrupt+7>:     mov    (%eax),%ecx
0xc0114e39 <smp_call_function_interrupt+9>:     mov    0x4(%eax),%edx
0xc0114e3c <smp_call_function_interrupt+12>:    mov    0x10(%eax),%esi
0xc0114e3f <smp_call_function_interrupt+15>:    xor    %eax,%eax
0xc0114e41 <smp_call_function_interrupt+17>:    xchg   %eax,0xffffd0b0
0xc0114e47 <smp_call_function_interrupt+23>:    lock addl $0x0,0x0(%esp,1)
0xc0114e4d <smp_call_function_interrupt+29>:    mov    0xc02e9830,%eax
0xc0114e52 <smp_call_function_interrupt+34>:    lock incl 0x8(%eax)
0xc0114e56 <smp_call_function_interrupt+38>:    mov    $0xffffe000,%ebx
0xc0114e5b <smp_call_function_interrupt+43>:    and    %esp,%ebx
0xc0114e5d <smp_call_function_interrupt+45>:    mov    0x14(%ebx),%eax
0xc0114e60 <smp_call_function_interrupt+48>:    add    $0x10000,%eax
0xc0114e65 <smp_call_function_interrupt+53>:    mov    %eax,0x14(%ebx)
0xc0114e68 <smp_call_function_interrupt+56>:    push   %edx
0xc0114e69 <smp_call_function_interrupt+57>:    call   *%ecx
The only reason i appear to be seeing this problem is probably due to 
cache controller hardware which is present in the box (old 'highend' MP 
server). I have been unable to reproduce this on much larger SMP boxes 
(saner cache coherency hardware).

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 5
model           : 2
model name      : Pentium 75 - 200
stepping        : 11
cpu MHz         : 133.314
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : yes
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr mce cx8 apic
bogomips        : 266.24

Patch against 2.5.66 appended.

Unable to handle kernel NULL pointer dereference at virtual address 00000200
 printing eip:
00000200
*pde = 00000000
Oops: 0000 [#1]
CPU:    2
EIP:    0060:[<00000200>]    Not tainted
EFLAGS: 00210082
EIP is at 0x200
eax: 00000026   ebx: c7666000   ecx: c7666000   edx: c7666000
esi: 00000200   edi: c034ad74   ebp: c0bc8000   esp: c7667fa8
ds: 007b   es: 007b   ss: 0068
Process rhn-applet (pid: 1654, threadinfo=c7666000 task=c0785340)
Stack: c0116407 c034ad74 40a2d760 08458490 00000004 bfffec78 c010a41a 40a2d760 
       00000000 00000000 08458490 00000004 bfffec78 0830d800 0000007b 0000007b 
       fffffffb 40a23da4 00000073 00200202 bfffec48 0000007b 
Call Trace:
 [<c0116407>] smp_call_function_interrupt+0x57/0xb0
 [<c034ad74>] sr_do_ioctl+0x124/0x250
 [<c010a41a>] call_function_interrupt+0x1a/0x20
Code:  Bad EIP value.
 <0>Kernel panic: Aiee, killing interrupt handler!

Index: linux-2.5.66/arch/i386/kernel/smp.c
===================================================================
RCS file: /build/cvsroot/linux-2.5.66/arch/i386/kernel/smp.c,v
retrieving revision 1.1.1.1
diff -u -p -B -r1.1.1.1 smp.c
--- linux-2.5.66/arch/i386/kernel/smp.c	24 Mar 2003 23:40:27 -0000	1.1.1.1
+++ linux-2.5.66/arch/i386/kernel/smp.c	28 Mar 2003 05:08:54 -0000
@@ -522,7 +521,8 @@ int smp_call_function (void (*func) (voi
 
 	spin_lock(&call_lock);
 	call_data = &data;
-	wmb();
+	mb();
+	
 	/* Send a message to all other CPUs and wait for them to respond */
 	send_IPI_allbutself(CALL_FUNCTION_VECTOR);
 
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
\
 
 \ /
  Last update: 2005-03-22 12:34    [from the cache]
©2003-2008