Messages in this thread |  | | Date | Sun, 3 Sep 2000 11:56:11 -0700 | Subject | Hard lock on boot with both 2.2.16 and 2.4.0-pre7 | From | Nathaniel Smith <> |
| |
I'm not quite certain if this is a user-mode, kernel, or hardware problem, but there is a reproducible hard lock involved (SysRq keys no longer work, etc.) and I managed to get some Oops output, so maybe someone here can give me a clue what's going on. As it is, I have no idea what to do next, and my box is unusable.
Hardware: dual PIII 500, not overclocked, Intel motherboard, IDE, 3com ethernet card, ask if you need any more details... Kernel: originally 2.2.16, then 2.4.0-pre7 Distro: Debian woody
A few days ago, I ran a standard apt-get upgrade, which seemed to complete without problems. A few minutes later, the machine locked completely. After SysRq failed to have effect, rebooted the machine manually, and while starting daemons it froze solid again, repeatedly printing to the screen: "stuck on TLB IPI wait (CPU#1)" (it varies whether it says CPU#1 or CPU#0). It seems that the relevant daemon was mysql; after disabling mysql, it waited until getting to postgres to show the same behaviour; after disabling postgres, it seems to get to sshd before crashing. I can start the box in single user mode and things seem to mostly work fine (I'm typing this email on the box now, the ethernet is up, and I had no problems compiling a new kernel version for testing). Booting kernel 2.4.0-pre7 instead, I saw the same behaviour, but a more detailed error message (may be a typo or two; I had to copy it out by hand):
NMI Watchdog detected LOCKUP on CPU0, registers: CPU: 0 EIP: 0010:[<c01cf0e6>] EFLAGS: 00000086 eax: c1273000 ebx: 00000046 ecx: 00000000 edx: 00000000 esi: 04000001 edi: 00002710 ebp: c0222000 esp: c0223f30 ds: 0018 es: 0018 ss: 0018 Process swapper (pid: 0, stackpage = c0223000) Stack: 000002aa c01729e1 000000f4 00000202 04000001 00000000 0000000c c0274235 c017sb07 c7a30ac0 c010bde1 0000000c c1273000 c0223fa8 c02745e0 c025c980 0000000c c0223fa0 c010bfcb 0000000c c0223fa8 c7a30ac0 c01088c0 c0222000 Call Trace: [<c01729e1>] [<c0172b07>] [<c010bde1>] [<c010bfc6>] [<c01088c0>] [<c01088c0>] [<c010a710>] [<c01088c0>] [<c01088c0>] [<c0100018>] [<c01088ed>] [<c0108952>] [<c0105000>] [<c01001d0>] Code: 80 3d 20 aa 21 c0 00 f3 90 7e f5 e9 a2 3b fa ff 80 3d 20 aa console shuts up ...
Running this through ksymoops (I believe the only modules loaded at the point of the crash were 3c95x and sr_mod, but it's difficult to tell, and ksymoops doesn't seem to mind the lack of module information):
ksymoops 2.3.4 on i686 2.2.16. Options used -v /usr/src/linux-2.4.0-pre7/vmlinux (specified) -K (specified) -L (specified) -o /lib/modules/2.4.0-test7/ (specified) -m /usr/src/linux-2.4.0-pre7/System.map (specified)
No modules in ksyms, skipping objects NMI Watchdog detected LOCKUP on CPU0, registers: CPU: 0 EIP: 0010:[<c01cf0e6>] Using defaults from ksymoops -t elf32-i386 -a i386 EFLAGS: 00000086 eax: c1273000 ebx: 00000046 ecx: 00000000 edx: 00000000 esi: 04000001 edi: 00002710 ebp: c0222000 esp: c0223f30 ds: 0018 es: 0018 ss: 0018 Stack: 000002aa c01729e1 000000f4 00000202 04000001 00000000 0000000c c0274235 c017sb07 c7a30ac0 c010bde1 0000000c c1273000 c0223fa8 c02745e0 c025c980 0000000c c0223fa0 c010bfcb 0000000c c0223fa8 c7a30ac0 c01088c0 c0222000 Call Trace: [<c01729e1>] [<c0172b07>] [<c010bde1>] [<c010bfc6>] [<c01088c0>] [<c01088c0>] [<c010a710>] [<c01088c0>] [<c01088c0>] [<c0100018>] [<c01088ed>] [<c0108952>] [<c0105000>] [<c01001d0>] Code: 80 3d 20 aa 21 c0 00 f3 90 7e f5 e9 a2 3b fa ff 80 3d 20 aa
>>EIP; c01cf0e6 <stext_lock+3522/7a88> <===== Trace; c01729e1 <handle_kbd_event+85/18c> Trace; c0172b07 <keyboard_interrupt+1f/2c> Trace; c010bde1 <handle_IRQ_event+4d/78> Trace; c010bfc6 <do_IRQ+a6/f4> Trace; c01088c0 <default_idle+0/34> Trace; c01088c0 <default_idle+0/34> Trace; c010a710 <ret_from_intr+0/20> Trace; c01088c0 <default_idle+0/34> Trace; c01088c0 <default_idle+0/34> Trace; c0100018 <startup_32+18/cc> Trace; c01088ed <default_idle+2d/34> Trace; c0108952 <cpu_idle+3e/54> Trace; c0105000 <empty_bad_page+0/1000> Trace; c01001d0 <L6+0/2> Code; c01cf0e6 <stext_lock+3522/7a88> 00000000 <_EIP>: Code; c01cf0e6 <stext_lock+3522/7a88> <===== 0: 80 3d 20 aa 21 c0 00 cmpb $0x0,0xc021aa20 <===== Code; c01cf0ed <stext_lock+3529/7a88> 7: f3 90 repz nop Code; c01cf0ef <stext_lock+352b/7a88> 9: 7e f5 jle 0 <_EIP> Code; c01cf0f1 <stext_lock+352d/7a88> b: e9 a2 3b fa ff jmp fffa3bb2 <_EIP+0xfffa3bb2> c0172c98 <aux_write_ack+4/3c> Code; c01cf0f6 <stext_lock+3532/7a88> 10: 80 3d 20 aa 00 00 00 cmpb $0x0,0xaa20
I would appreciate any information at all people can give me about this problem; as I said, I can't even tell for sure whether it's hardware, kernel, or userland. The consistency over kernel versions and completeness of the lock suggest something hardware related, but it seems to have happened immediately following an upgrade (though I'm not sure; mysql and postgres were both upgraded, but ssh was not, so it's unclear why it would suddenly start crashing the box), and is highly sensitive to the program being run.
Thank you in advance for any information you can give.
-- Nathaniel
-- "...these, like all words, have single, decontextualized meanings: everyone knows what each of these words means, everyone knows what constitutes an instance of each of their referents. Language is fixed. Meaning is certain. Santa Claus comes down the chimney at midnight on December 24." -- The Language War, Robin Lakoff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org Please read the FAQ at http://www.tux.org/lkml/
|  |