Messages in this thread |  | | | Date | Thu, 30 Oct 1997 11:09:44 -0500 (EST) | | From | Peter Woit <> | | Subject | 2.0.31 problems (in get_empty_inode?, "unable to handle kernel paging request") |
| |
We've been having periodic crashes of our main linux server due to some sort of consistent kernel problem that is still there in 2.0.31. Crashes typically occur when "find" is being run by updatedb as a cron job, but other pieces of code (rpc.nfsd, ps) seem to also manage to hit this problem. I'm including the latest kernel crash debugging output as well as various info about the the system configuration (the only thing that may not appear there is that we are running netatalk 1.3.3), and some earlier crashes also. I'd greatly appreciate any advice from the experts about what is causing this and what to try to do to get around it.
A related question: Can one get Linux to automatically reboot on this kind of kernel crash? It would be a lot easier to live with if the machine would reboot instead of just staying hung.
Peter Woit woit@math.columbia.edu
Oct 30 07:40:04 cpw kernel: Unable to handle kernel paging request at virtual address d6f6ff44 Oct 30 07:40:04 cpw kernel: current->tss.cr3 = 0521c000, |r3 = 0521c000 Oct 30 07:40:04 cpw kernel: *pde = 00000000 Oct 30 07:40:04 cpw kernel: Oops: 0000 Oct 30 07:40:04 cpw kernel: CPU: 1 Oct 30 07:40:04 cpw kernel: EIP: 0010:[get_empty_inode+68/352] Oct 30 07:40:04 cpw kernel: EFLAGS: 00010202 Oct 30 07:40:04 cpw kernel: eax: 00000b00 ebx: 078b6dc8 ecx: 16f6fec4 edx: 00000006 Oct 30 07:40:04 cpw kernel: esi: 000001b3 edi: 00000001 ebp: 001f8e48 esp: 0521dee0 Oct 30 07:40:04 cpw kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 Oct 30 07:40:04 cpw kernel: Process find (pid: 2192, process nr: 80, stackpage=0521d000) Oct 30 07:40:04 cpw kernel: Stack: 00000000 001e5668 00000000 00129e09 031d5dc8 0521df74 00000007 03d7e000 Oct 30 07:40:04 cpw kernel: 00164525 00164558 001f8e48 0001e62e 00000001 031d5dc8 0521df74 0521df74 Oct 30 07:40:04 cpw kernel: 00000007 031d5dc8 0001e62e 0013109a 031d5dc8 03d7e000 00000007 0521df74 Oct 30 07:40:04 cpw kernel: Call Trace: [__iget+97/516] [ext2_lookup+129/368] [ext2_lookup+180/368] [lookup+238/264] [_namei+91/228] [lnamei+48/72] [sys_newlstat+41/88] Oct 30 07:40:04 cpw kernel: [system_call+258/320] Oct 30 07:40:04 cpw kernel: Code: 66 83 b9 80 00 00 00 00 75 26 ba e7 03 00 00 8a 81 84 00 00 Linux cpw 2.0.31 #2 Wed Oct 29 20:44:01 EST 1997 i686 Kernel modules found Gnu C 2.7.2.3 Binutils linux-2.8.1.0.1 Linux C Library 5.4.38 Dynamic Linker (ld.so) 1.9.6 Linux C++ Library 27.2.8 Procps 1.2 Mount 2.7f Net-tools (1997-09-21) Kbd 0.94 Sh-utils 1.12
Oct 30 10:18:33 cpw kernel: Intel MultiProcessor Specification v1.1 Oct 30 10:18:33 cpw kernel: Virtual Wire compatibility mode. Oct 30 10:18:33 cpw kernel: OEM ID: OEM00000 Product ID: PROD00000000 APIC at: 0xFEE00000 Oct 30 10:18:33 cpw kernel: Processor #1 Pentium(tm) Pro APIC version 17 Oct 30 10:18:33 cpw kernel: Processor #0 Pentium(tm) Pro APIC version 17 Oct 30 10:18:33 cpw kernel: I/O APIC #2 Version 17 at 0xFEC00000. Oct 30 10:18:33 cpw kernel: Processors: 2 Oct 30 10:18:33 cpw kernel: Console: 16 point font, 400 scans Oct 30 10:18:33 cpw kernel: Console: colour VGA+ 80x25, 1 virtual console (max 63) Oct 30 10:18:33 cpw kernel: pcibios_init : BIOS32 Service Directory structure at 0x000fad00 Oct 30 10:18:33 cpw kernel: pcibios_init : BIOS32 Service Directory entry at 0xfb180 Oct 30 10:18:33 cpw kernel: pcibios_init : PCI BIOS revision 2.10 entry at 0xfb1b0 Oct 30 10:18:33 cpw kernel: Probing PCI hardware. Oct 30 10:18:33 cpw kernel: Calibrating delay loop.. ok - 199.07 BogoMIPS Oct 30 10:18:33 cpw kernel: Memory: 127824k/131072k available (760k kernel code, 384k reserved, 2104k data) Oct 30 10:18:33 cpw kernel: Swansea University Computer Society TCP/IP for NET3.034 Oct 30 10:18:33 cpw kernel: IP Protocols: IGMP, ICMP, UDP, TCP Oct 30 10:18:33 cpw kernel: Checking 386/387 coupling... Ok, fpu using exception 16 error reporting. Oct 30 10:18:33 cpw kernel: Checking 'hlt' instruction... Ok. Oct 30 10:18:33 cpw kernel: Linux version 2.0.31 (root@cpw) (gcc version 2.7.2.3) #2 Wed Oct 29 20:44:01 EST 1997 Oct 30 10:18:33 cpw kernel: Booting processor 0 stack 00002000: Calibrating delay loop.. ok - 199.07 BogoMIPS Oct 30 10:18:33 cpw kernel: Total of 2 processors activated (398.13 BogoMIPS). Oct 30 10:18:33 cpw kernel: Starting kswapd v 1.4.2.2 Oct 30 10:18:33 cpw kernel: EATA (Extended Attachment) driver version: 2.59b Oct 30 10:18:36 cpw lpd[64]: /dev/lp: No such file or directory Oct 30 10:18:33 cpw kernel: developed in co-operation with DPT Oct 30 10:18:33 cpw kernel: (c) 1993-96 Michael Neuffer, mike@i-Connect.Net Oct 30 10:18:33 cpw kernel: Registered HBAs: Oct 30 10:18:33 cpw kernel: HBA no. Boardtype Revis EATA Bus BaseIO IRQ DMA Ch ID Pr QS S/G IS Oct 30 10:18:33 cpw kernel: scsi1 : PM3334UW v07L.0 2.0c PCI 0x8210 10 BMST 2 7 N 64 252 Y Oct 30 10:18:33 cpw kernel: scsi0 : BusLogic BT-958 Oct 30 10:18:33 cpw kernel: scsi1 : EATA (Extended Attachment) HBA driver Oct 30 10:18:33 cpw kernel: scsi : 2 hosts. Oct 30 10:18:33 cpw kernel: Vendor: Quantum Model: XP34300W Rev: L912 Oct 30 10:18:33 cpw kernel: Type: Direct-Access ANSI SCSI revision: 02 Oct 30 10:18:33 cpw kernel: Detected scsi disk sda at scsi0, channel 0, id 0, lun 0 Oct 30 10:18:33 cpw kernel: Vendor: TEAC Model: CD-ROM CD-56S Rev: 1.0D Oct 30 10:18:33 cpw kernel: Type: CD-ROM ANSI SCSI revision: 02 Oct 30 10:18:33 cpw kernel: Detected scsi CD-ROM sr0 at scsi0, channel 0, id 5, lun 0 Oct 30 10:18:33 cpw kernel: Vendor: DPT Model: RAID-5 Rev: 07L0 Oct 30 10:18:33 cpw kernel: Type: Direct-Access ANSI SCSI revision: 02 Oct 30 10:18:33 cpw kernel: Detected scsi disk sdb at scsi1, channel 0, id 11, lun 0 Oct 30 10:18:33 cpw kernel: scsi : detected 1 SCSI cdrom 2 SCSI disks total. Oct 30 10:18:33 cpw kernel: SCSI device sda: hdwr sector= 512 bytes. Sectors= 8399520 [4101 MB] [4.1 GB] Oct 30 10:18:33 cpw kernel: SCSI device sdb: hdwr sector= 512 bytes. Sectors= 26659968 [13017 MB] [13.0 GB] Oct 30 10:18:33 cpw kernel: Found Intel i82557 PCI Speedo at I/O 0x8100, IRQ 9. Oct 30 10:18:33 cpw kernel: PCI latency timer (CFLT) is 0x20. Oct 30 10:18:33 cpw kernel: eepro100.c:v0.36 10/20/97 Donald Becker linux-eepro100@cesdis.gsfc.nasa.gov Oct 30 10:18:33 cpw kernel: eepro100.c:v0.36 10/20/97 Donald Becker linux-eepro100@cesdis.gsfc.nasa.gov Oct 30 10:18:33 cpw kernel: Partition check: Oct 30 10:18:33 cpw kernel: sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 > Oct 30 10:18:33 cpw kernel: sdb: sdb1 sdb2 Oct 30 10:18:33 cpw kernel: VFS: Mounted root (ext2 filesystem) readonly. Oct 30 10:18:33 cpw kernel: Adding Swap: 128516k swap-space (priority -1) Oct 30 10:18:33 cpw kernel: Adding Swap: 120452k swap-space (priority -2) Oct 30 10:18:33 cpw kernel: Adding Swap: 120452k swap-space (priority -3)
Older crashes
Jun 14 14:53:01 cpw kernel: Unable to handle kernel paging request at virtual address d63d4764 Jun 14 14:53:01 cpw kernel: current->tss.cr3 = 07a8f000, |r3 = 07a8f000 Jun 14 14:53:01 cpw kernel: *pde = 00000000 Jun 14 14:53:01 cpw kernel: Oops: 0000 Jun 14 14:53:01 cpw kernel: CPU: 1 Jun 14 14:53:01 cpw kernel: EIP: 0010:[get_empty_inode+68/352] Jun 14 14:53:01 cpw kernel: EFLAGS: 00010202 Jun 14 14:53:01 cpw kernel: eax: 00000c00 ebx: 04aaa4ec ecx: 163d46e4 edx: 00000002 Jun 14 14:53:01 cpw kernel: esi: 000002ec edi: 00000001 ebp: 0020d2f4 esp: 07a91ee0 Jun 14 14:53:01 cpw kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 Jun 14 14:53:01 cpw kernel: Process rpc.nfsd (pid: 114, process nr: 44, stackpage=07a91000) Jun 14 14:53:01 cpw kernel: Stack: 00000000 001f9530 00000000 00128a69 03fd82f4 02c9da98 00000018 06dcc01f Jun 14 14:53:01 cpw kernel: 02c9da98 00161cbf 0020d2f4 000578e7 00000001 03fd82f4 07a91f74 07a91f74 Jun 14 14:53:01 cpw kernel: 00000018 00ba64b4 000578e7 0012fb7e 03fd82f4 06dcc01f 00000018 07a91f74 Jun 14 14:53:01 cpw kernel: Call Trace: [__iget+97/516] [ext2_lookup+343/368] [lookup+238/264] [_namei+91/196] [lnamei+48/72] [sys_newlstat+41/88] [system_call+207/272] Jun 14 14:53:01 cpw kernel: Code: 66 83 b9 80 00 00 00 00 75 26 ba e7 03 00 00 8a 81 84 00 00
Jul 22 07:40:11 cpw kernel: Unable to handle kernel paging request at virtual address d3f0095c Jul 22 07:40:11 cpw kernel: current->tss.cr3 = 077aa000, |r3 = 077aa000 Jul 22 07:40:11 cpw kernel: *pde = 00000000 Jul 22 07:40:11 cpw kernel: Oops: 0000 Jul 22 07:40:11 cpw kernel: CPU: 1 Jul 22 07:40:11 cpw kernel: EIP: 0010:[get_empty_inode+68/352] Jul 22 07:40:11 cpw kernel: EFLAGS: 00010216 Jul 22 07:40:11 cpw kernel: eax: 00000c00 ebx: 050e2dc8 ecx: 13f008dc edx: 0000000d Jul 22 07:40:11 cpw kernel: esi: 0000015f edi: 00000001 ebp: 0020d160 esp: 077c0ee0 Jul 22 07:40:11 cpw kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 Jul 22 07:40:11 cpw kernel: Process find (pid: 3900, process nr: 61, stackpage=077c0000) Jul 22 07:40:11 cpw kernel: Stack: 00000000 001f9318 00000000 00128a69 0050d2f4 077c0f74 00000008 013e7000 Jul 22 07:40:11 cpw kernel: 00161be9 00161c1c 0020d160 0002b69e 00000001 0050d2f4 077c0f74 077c0f74 Jul 22 07:40:11 cpw kernel: 00000008 0050d2f4 0002b69e 0012fb7e 0050d2f4 013e7000 00000008 077c0f74 Jul 22 07:40:11 cpw kernel: Call Trace: [__iget+97/516] [ext2_lookup+129/368] [ext2_lookup+180/368] [lookup+238 /264] [_namei+91/196] [lnamei+48/72] [sys_newlstat+41/88] Jul 22 07:40:11 cpw kernel: [system_call+207/272] Jul 22 07:40:11 cpw kernel: Code: 66 83 b9 80 00 00 00 00 75 26 ba e7 03 00 00 8a 81 84 00 00
Aug 23 07:40:06 cpw kernel: Unable to handle kernel paging request at virtual address d47b656c Aug 23 07:40:06 cpw kernel: current->tss.cr3 = 03589000, |r3 = 03589000 Aug 23 07:40:06 cpw kernel: *pde = 00000000 Aug 23 07:40:06 cpw kernel: Oops: 0000 Aug 23 07:40:06 cpw kernel: CPU: 1 Aug 23 07:40:06 cpw kernel: EIP: 0010:[get_empty_inode+68/352] Aug 23 07:40:06 cpw kernel: EFLAGS: 00010202 Aug 23 07:40:06 cpw kernel: eax: 00000c00 ebx: 010664ec ecx: 147b64ec edx: 00000001 Aug 23 07:40:06 cpw kernel: esi: 0000033d edi: 00000001 ebp: 0020cfcc esp: 02ca1ee0 Aug 23 07:40:06 cpw kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 Aug 23 07:40:06 cpw kernel: Process find (pid: 2253, process nr: 64, stackpage=02ca1000) Aug 23 07:40:06 cpw kernel: Stack: 00000000 001f8f88 00000000 00128a69 02e839d8 02ca1f74 00000009 03579000 Aug 23 07:40:06 cpw kernel: 00161be9 00161c1c 0020cfcc 0001f62a 00000001 02e839d8 02ca1f74 02ca1f74 Aug 23 07:40:06 cpw kernel: 00000009 02e839d8 0001f62a 0012fb7e 02e839d8 03579000 00000009 02ca1f74 Aug 23 07:40:06 cpw kernel: Call Trace: [__iget+97/516] [ext2_lookup+129/368] [ext2_lookup+180/368] [lookup+238 /264] [_namei+91/196] [lnamei+48/72] [sys_newlstat+41/88] Aug 23 07:40:06 cpw kernel: [system_call+207/272] Aug 23 07:40:06 cpw kernel: Code: 66 83 b9 80 00 00 00 00 75 26 ba e7 03 00 00 8a 81 84 00 00
Aug 25 02:31:52 cpw kernel: Unable to handle kernel paging request at virtual address d29bedc8 Aug 25 02:31:52 cpw kernel: current->tss.cr3 = 00c5a000, |r3 = 00c5a000 Aug 25 02:31:52 cpw kernel: *pde = 00000000 Aug 25 02:31:52 cpw kernel: Oops: 0000 Aug 25 02:31:52 cpw kernel: CPU: 1 Aug 25 02:31:52 cpw kernel: EIP: 0010:[__iget+60/516] Aug 25 02:31:52 cpw kernel: EFLAGS: 00010202 Aug 25 02:31:52 cpw kernel: eax: 00000807 ebx: 129bedc8 ecx: 000678fa edx: 000678fa Aug 25 02:31:52 cpw kernel: esi: 001f9618 edi: 00000000 ebp: 0020d2f4 esp: 00cceee8 Aug 25 02:31:52 cpw kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 Aug 25 02:31:52 cpw kernel: Process tkdesksh (pid: 1508, process nr: 59, stackpage=00cce000) Aug 25 02:31:52 cpw kernel: Stack: 050694ec 00ccef6c 00000002 07d6b006 00161be9 00161c1c 0020d2f4 000678fa Aug 25 02:31:52 cpw kernel: 00000001 050694ec 00ccef6c 00ccef6c 00000002 050694ec 000678fa 0012fb7e Aug 25 02:31:52 cpw kernel: 050694ec 07d6b006 00000002 00ccef6c 00000000 00ccef6c 00ccefb4 000004c2 Aug 25 02:31:52 cpw kernel: Call Trace: [ext2_lookup+129/368] [ext2_lookup+180/368] [lookup+238/264] [_namei+91 /196] [namei+48/72] [sys_access+195/292] [system_call+207/272] Aug 25 02:31:52 cpw kernel: Code: 66 39 03 75 0d 8b 4c 24 1c 39 4b 04 0f 84 fa 00 00 00 8b 5b Sep 8 07:48:12 cpw kernel: Unable to handle kernel paging request at virtual address d549256c Sep 8 07:48:12 cpw kernel: current->tss.cr3 = 012b7000, |r3 = 012b7000 Sep 8 07:48:12 cpw kernel: *pde = 00000000 Sep 8 07:48:12 cpw kernel: Oops: 0000 Sep 8 07:48:12 cpw kernel: CPU: 1 Sep 8 07:48:12 cpw kernel: EIP: 0010:[get_empty_inode+68/352] Sep 8 07:48:12 cpw kernel: EFLAGS: 00010202 Sep 8 07:48:12 cpw kernel: eax: 00000c00 ebx: 031a13f0 ecx: 154924ec edx: 00000006 Sep 8 07:48:12 cpw kernel: esi: 000002f2 edi: 00000001 ebp: 0020d2f4 esp: 01338ee0 Sep 8 07:48:12 cpw kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 Sep 8 07:48:12 cpw kernel: Process find (pid: 13197, process nr: 58, stackpage=01338000) Sep 8 07:48:12 cpw kernel: Stack: 00000000 001f9260 00000000 00128a69 05af80fc 038b1118 00000018 030c9000 Sep 8 07:48:12 cpw kernel: 038b1118 00161cbf 0020d2f4 0002f081 00000001 05af80fc 01338f74 01338f74 Sep 8 07:48:12 cpw kernel: 00000018 038b0d14 0002f081 0012fb7e 05af80fc 030c9000 00000018 01338f74 Sep 8 07:48:12 cpw kernel: Call Trace: [__iget+97/516] [ext2_lookup+343/368] [lookup+238/264] [_namei+91/196] [lnamei+48/72] [sys_newlstat+41/88] [system_call+207/272] Sep 8 07:48:12 cpw kernel: Code: 66 83 b9 80 00 00 00 00 75 26 ba e7 03 00 00 8a 81 84 00 00
Sep 15 20:03:25 cpw kernel: Unable to handle kernel paging request at virtual address d120ee48 Sep 15 20:03:25 cpw kernel: current->tss.cr3 = 00a91000, |r3 = 00a91000 Sep 15 20:03:25 cpw kernel: *pde = 00000000 Sep 15 20:03:25 cpw kernel: Oops: 0000 Sep 15 20:03:25 cpw kernel: CPU: 1 Sep 15 20:03:25 cpw kernel: EIP: 0010:[get_empty_inode+68/352] Sep 15 20:03:25 cpw kernel: EFLAGS: 00010206 Sep 15 20:03:25 cpw kernel: eax: 00000c00 ebx: 05aecbd0 ecx: 1120edc8 edx: 00000001 Sep 15 20:03:25 cpw kernel: esi: 00000228 edi: 00000001 ebp: 00212ee0 esp: 00a9cee0 Sep 15 20:03:25 cpw kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 Sep 15 20:03:25 cpw kernel: Process rpc.nfsd (pid: 98, process nr: 20, stackpage=00a9c000) Sep 15 20:03:25 cpw kernel: Stack: 00000000 001fef68 00000000 00129cb9 00d428dc 00a9cf74 0000000e 041f901a Sep 15 20:03:25 cpw kernel: 001641a9 001641dc 00212ee0 000578bd 00000001 00d428dc 00a9cf74 00a9cf74 Sep 15 20:03:25 cpw kernel: 0000000e 00d428dc 000578bd 00130e5a 00d428dc 041f901a 0000000e 00a9cf74 Sep 15 20:03:25 cpw kernel: Call Trace: [__iget+97/516] [ext2_lookup+129/368] [ext2_lookup+180/368] [lookup+238 /264] [_namei+91/228] [lnamei+48/72] [sys_newlstat+41/88] Sep 15 20:03:25 cpw kernel: [system_call+255/320] Sep 15 20:03:25 cpw kernel: Code: 66 83 b9 80 00 00 00 00 75 26 ba e7 03 00 00 8a 81 84 00 00
Sep 16 10:44:16 cpw kernel: Unable to handle kernel paging request at virtual address d1b89e48 Sep 16 10:44:16 cpw kernel: current->tss.cr3 = 00b16000, |r3 = 00b16000 Sep 16 10:44:16 cpw kernel: *pde = 00000000 Sep 16 10:44:16 cpw kernel: Oops: 0000 Sep 16 10:44:16 cpw kernel: CPU: 1 Sep 16 10:44:16 cpw kernel: EIP: 0010:[get_empty_inode+68/352] Sep 16 10:44:16 cpw kernel: EFLAGS: 00010206 Sep 16 10:44:16 cpw kernel: eax: 00000c00 ebx: 028cfbd0 ecx: 11b89dc8 edx: 00000004 Sep 16 10:44:16 cpw kernel: esi: 00000453 edi: 00000004 ebp: 00212ee0 esp: 00b1dee0 Sep 16 10:44:16 cpw kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 Sep 16 10:44:16 cpw kernel: Process rpc.nfsd (pid: 134, process nr: 20, stackpage=00b1d000) Sep 16 10:44:16 cpw kernel: Stack: 00000000 001ff2b0 00000000 00129cb9 023343f0 00b1df74 0000000d 066fe01e Sep 16 10:44:16 cpw kernel: 001641a9 001641dc 00212ee0 00083124 00000001 023343f0 00b1df74 00b1df74 Sep 16 10:44:16 cpw kernel: 0000000d 023343f0 00083124 00130e5a 023343f0 066fe01e 0000000d 00b1df74 Sep 16 10:44:16 cpw kernel: Call Trace: [__iget+97/516] [ext2_lookup+129/368] [ext2_lookup+180/368] [lookup+238 /264] [_namei+91/228] [lnamei+48/72] [sys_newlstat+41/88] Sep 16 10:44:16 cpw kernel: [system_call+255/320] Sep 16 10:44:16 cpw kernel: Code: 66 83 b9 80 00 00 00 00 75 26 ba e7 03 00 00 8a 81 84 00 00
|  |