Messages in this thread | | | Date | Tue, 7 Sep 1999 12:51:48 -0700 (PDT) | From | Mark Hanson <> | Subject | Triple Oops with 2.2.11 SMP |
| |
Just got a triple oops on our SMP machine. This is the same machine I reported an oops with a couple of months ago. At that time the general feeling was that we had faulty memory. So we swapped 1 128MB DIMM out with 2 64MB DIMMS from another machine which had been running NT for a long time without incident. (Personally, I tend to discount the faulty memory theory, because the only problem we ever have with this machine is the crashes. I do complicated builds on it all the time (sometimes with make -j), and the compiler never crashes, which is a typical indication that you have faulty memory. I do lots of other file thrashing on it as well, and can't recall any programs crashing for no reason. (Well, Netscape will bomb with a bus-error once in a while, but I think that's par for the course.))
Thanks to the serial console feature, I was able to capture the Oops messages. After the crash, the machine was non-responsive to all the Alt-SysRq keys and CTRL-ALT-DEL.
It's a RedHat 5.2 machine running a 2.2.11 kernel with the four patches in Alan's release notes.
Please CC me any ideas or requests for more information. Ksymoops output follows, boot messages, etc. follow that.
Thanks, Mark
Unable to handle kernel paging request at virtual address 36383369 current->tss.cr3 = 01121000, %cr3 = 01121000 *pde = 00000000 Oops: 0000 CPU: 1 EIP: 0010:[<c01261fe>] Using defaults from ksymoops -t elf32-i386 -a i386 EFLAGS: 00010206 eax: 36383369 ebx: 001a02d2 ecx: 00000813 edx: 36383369 esi: 00000400 edi: 001a02d2 ebp: 00000813 esp: c6221d8c ds: 0018 es: 0018 ss: 0018 Process ld (pid: 10706, process nr: 174, stackpage=c6221000) Stack: c012622d 00000813 001a02d2 00000400 c0126493 00000813 001a02d2 00000400 00000813 001a02d2 00000813 00000008 00000006 c013cf87 00000813 001a02d2 00000400 001a02b0 c209c880 c6221f14 c7c49e00 00000400 00000400 c7c49e00 Call Trace: [<c012622d>] [<c0126493>] [<c013cf87>] [<c013f264>] [<c013f791>] [<c013f9c4>] [<c013dd9e>] [<c013db38>] [<c011c60a>] [<c011c6cc>] [<c011c618>] [<c010cd64>] [<c01249a1>] [<c0124d08>] [<c0107b1c>] [<c010002b>] Code: 8b 12 39 58 04 75 f3 39 70 08 75 ee 66 39 48 0c 75 e8 89 c2
>>EIP; c01261fe <find_buffer+2a/44> <===== Trace; c012622d <get_hash_table+15/20> Trace; c0126493 <getblk+1f/14c> Trace; c013cf87 <ext2_new_block+7bb/944> Trace; c013f264 <ext2_alloc_block+134/144> Trace; c013f791 <block_getblk+145/274> Trace; c013f9c4 <ext2_getblk+104/22c> Trace; c013dd9e <ext2_file_write+266/5f8> Trace; c013db38 <ext2_file_write+0/5f8> Trace; c011c60a <do_generic_file_read+5f6/604> Trace; c011c6cc <generic_file_read+64/80> Trace; c011c618 <file_read_actor+0/50> Trace; c010cd64 <old_mmap+c4/108> Trace; c01249a1 <sys_lseek+a9/e0> Trace; c0124d08 <sys_write+d8/114> Trace; c0107b1c <system_call+34/38> Trace; c010002b <startup_32+2b/a4> Code; c01261fe <find_buffer+2a/44> 00000000 <_EIP>: Code; c01261fe <find_buffer+2a/44> <===== 0: 8b 12 movl (%edx),%edx <===== Code; c0126200 <find_buffer+2c/44> 2: 39 58 04 cmpl %ebx,0x4(%eax) Code; c0126203 <find_buffer+2f/44> 5: 75 f3 jne fffffffa <_EIP+0xfffffffa> c01261f8 <find_buffer+24/44> Code; c0126205 <find_buffer+31/44> 7: 39 70 08 cmpl %esi,0x8(%eax) Code; c0126208 <find_buffer+34/44> a: 75 ee jne fffffffa <_EIP+0xfffffffa> c01261f8 <find_buffer+24/44> Code; c012620a <find_buffer+36/44> c: 66 39 48 0c cmpw %cx,0xc(%eax) Code; c012620e <find_buffer+3a/44> 10: 75 e8 jne fffffffa <_EIP+0xfffffffa> c01261f8 <find_buffer+24/44> Code; c0126210 <find_buffer+3c/44> 12: 89 c2 movl %eax,%edx
Unable to handle kernel paging request at virtual address 4004aae4 current->tss.cr3 = 00101000, %cr3 = 00101000 *pde = 00000000 Oops: 0000 CPU: 1 EIP: 0010:[<c012734e>] EFLAGS: 00010203 eax: 4004aad0 ebx: c03ab2b8 ecx: 00000006 edx: 00010000 esi: 4004aad0 edi: c1190800 ebp: c03ab2b8 esp: c7fdffa8 ds: 0018 es: 0018 ss: 0018 Process kswapd (pid: 5, process nr: 6, stackpage=c7fdf000) Stack: 00000030 c7fde000 c011bc4e c03ab2b8 0000001a 00000006 c0120ec7 00000006 00000030 00000000 c01f86eb c7fde2b9 c0120f8b 00000030 00000f00 c7ff3fc8 c03af0c0 00001000 c01064a3 00000000 00000f00 c0239fc0 Call Trace: [<c011bc4e>] [<c0120ec7>] [<c01f86eb>] [<c0120f8b>] [<c01064a3>] Code: 8b 76 14 83 78 20 00 75 06 f6 40 18 46 74 13 6a 00 e8 4c 01
>>EIP; c012734e <try_to_free_buffers+12/88> <===== Trace; c011bc4e <shrink_mmap+de/130> Trace; c0120ec7 <do_try_to_free_pages+3f/ac> Trace; c01f86eb <tvecs+1027/2190> Trace; c0120f8b <kswapd+57/e0> Trace; c01064a3 <kernel_thread+23/30> Code; c012734e <try_to_free_buffers+12/88> 00000000 <_EIP>: Code; c012734e <try_to_free_buffers+12/88> <===== 0: 8b 76 14 movl 0x14(%esi),%esi <===== Code; c0127351 <try_to_free_buffers+15/88> 3: 83 78 20 00 cmpl $0x0,0x20(%eax) Code; c0127355 <try_to_free_buffers+19/88> 7: 75 06 jne f <_EIP+0xf> c012735d <try_to_free_buffers+21/88> Code; c0127357 <try_to_free_buffers+1b/88> 9: f6 40 18 46 testb $0x46,0x18(%eax) Code; c012735b <try_to_free_buffers+1f/88> d: 74 13 je 22 <_EIP+0x22> c0127370 <try_to_free_buffers+34/88> Code; c012735d <try_to_free_buffers+21/88> f: 6a 00 pushl $0x0 Code; c012735f <try_to_free_buffers+23/88> 11: e8 4c 01 00 00 call 162 <_EIP+0x162> c01274b0 <wakeup_bdflush+0/90>
Unable to handle kernel paging request at virtual address 00007110 current->tss.cr3 = 063ae000, %cr3 = 063ae000 *pde = 00000000 Oops: 0000 CPU: 0 EIP: 0010:[<c012000f>] EFLAGS: 00010082 eax: c106801c ebx: c7fff020 ecx: 00007108 edx: c0295298 esi: c1067fe0 edi: 00000282 ebp: c1067fe0 esp: c0aa1e48 ds: 0018 es: 0018 ss: 0018 Process cvs (pid: 10841, process nr: 170, stackpage=c0aa1000) Stack: c7fff740 c1067fe0 00000001 c106801c c1067fe0 c011f3f6 c7fff020 c1067fe0 c7fff748 c7fff740 c7fff748 00000000 c7570804 c01208c3 c7fff740 c1067fe0 00000020 00000013 00000013 00000000 c1067fe0 00000001 c0120eb5 00000013 Call Trace: [<c011f3f6>] [<c01208c3>] [<c0120eb5>] [<c0121037>] [<c0121832>] [<c011bdf7>] [<c011c2fa>] [<c011c6cc>] [<c011c618>] [<c0124bfe>] [<c0107b1c>] Code: 8b 69 08 81 fd 2b 2f c3 a5 0f 85 fe 00 00 00 8b 69 0c 85 ed
>>EIP; c012000f <kmem_cache_free+43/19c> <===== Trace; c011f3f6 <kmem_slab_destroy+ba/c8> Trace; c01208c3 <kmem_cache_reap+1e7/21c> Trace; c0120eb5 <do_try_to_free_pages+2d/ac> Trace; c0121037 <try_to_free_pages+23/30> Trace; c0121832 <__get_free_pages+72/3bc> Trace; c011bdf7 <try_to_read_ahead+2f/128> Trace; c011c2fa <do_generic_file_read+2e6/604> Trace; c011c6cc <generic_file_read+64/80> Trace; c011c618 <file_read_actor+0/50> Trace; c0124bfe <sys_read+d2/104> Trace; c0107b1c <system_call+34/38> Code; c012000f <kmem_cache_free+43/19c> 00000000 <_EIP>: Code; c012000f <kmem_cache_free+43/19c> <===== 0: 8b 69 08 movl 0x8(%ecx),%ebp <===== Code; c0120012 <kmem_cache_free+46/19c> 3: 81 fd 2b 2f c3 cmpl $0xa5c32f2b,%ebp Code; c0120017 <kmem_cache_free+4b/19c> 8: a5 Code; c0120018 <kmem_cache_free+4c/19c> 9: 0f 85 fe 00 00 jne 10d <_EIP+0x10d> c012011c <kmem_cache_free+150/19c> Code; c012001d <kmem_cache_free+51/19c> e: 00 Code; c012001e <kmem_cache_free+52/19c> f: 8b 69 0c movl 0xc(%ecx),%ebp Code; c0120021 <kmem_cache_free+55/19c> 12: 85 ed testl %ebp,%ebp
Linux version 2.2.11 (root@mercury) (gcc version 2.7.2.3) #6 SMP Tue Sep 7 07:01:33 PDT 1999 Intel MultiProcessor Specification v1.4 Virtual Wire compatibility mode. OEM ID: INTEL Product ID: 440LX APIC at: 0xFEE00000 Processor #0 Pentium(tm) Pro APIC version 17 Processor #1 Pentium(tm) Pro APIC version 17 I/O APIC #2 Version 17 at 0xFEC00000. Processors: 2 mapped APIC to ffffe000 (fee00000) mapped IOAPIC to ffffd000 (fec00000) Detected 331928246 Hz processor. Console: colour VGA+ 80x25 Calibrating delay loop... 330.96 BogoMIPS Memory: 127904k/131072k available (1136k kernel code, 420k reserved, 1564k data, 48k init) Checking 386/387 coupling... OK, FPU using exception 16 error reporting. Checking 'hlt' instruction... OK. POSIX conformance testing by UNIFIX per-CPU timeslice cutoff: 100.27 usecs. CPU0: Intel Pentium II (Deschutes) stepping 00 calibrating APIC timer ... ..... CPU clock speed is 331.9136 MHz. ..... system bus clock speed is 66.3824 MHz. Booting processor 1 eip 2000 Calibrating delay loop... 331.78 BogoMIPS OK. CPU1: Intel Pentium II (Deschutes) stepping 00 Total of 2 processors activated (662.73 BogoMIPS). enabling symmetric IO mode... ...done. ENABLING IO-APIC IRQs init IO_APIC IRQs IO-APIC pin 0, 10, 11, 16, 17, 20, 21, 22, 23 not connected. number of MP IRQ sources: 17. number of IO-APIC registers: 24. testing the IO APIC....................... .... register #00: 02000000 ....... : physical APIC id: 02 .... register #01: 00170011 ....... : max redirection entries: 0017 ....... : IO APIC version: 0011 .... register #02: 00000000 ....... : arbitration: 00 .... IRQ redirection table: NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: 00 000 00 1 0 0 0 0 0 0 00 01 000 00 0 0 0 0 0 1 1 59 02 0FF 0F 0 0 0 0 0 1 1 51 03 000 00 0 0 0 0 0 1 1 61 04 000 00 0 0 0 0 0 1 1 69 05 000 00 0 0 0 0 0 1 1 71 06 000 00 0 0 0 0 0 1 1 79 07 000 00 0 0 0 0 0 1 1 81 08 000 00 0 0 0 0 0 1 1 89 09 000 00 0 0 0 0 0 1 1 91 0a 000 00 1 0 0 0 0 0 0 00 0b 000 00 1 0 0 0 0 0 0 00 0c 000 00 0 0 0 0 0 1 1 99 0d 000 00 1 0 0 0 0 0 0 00 0e 000 00 0 0 0 0 0 1 1 A1 0f 000 00 0 0 0 0 0 1 1 A9 10 000 00 1 0 0 0 0 0 0 00 11 000 00 1 0 0 0 0 0 0 00 12 0FF 0F 1 1 0 1 0 1 1 B1 13 0FF 0F 1 1 0 1 0 1 1 B9 14 000 00 1 0 0 0 0 0 0 00 15 000 00 1 0 0 0 0 0 0 00 16 000 00 1 0 0 0 0 0 0 00 17 000 00 1 0 0 0 0 0 0 00 .................................... done. PCI: PCI BIOS revision 2.10 entry at 0xfdba1 PCI: Using configuration type 1 PCI: Probing PCI hardware PCI: Enabling I/O for device 00:3a Linux NET4.0 for Linux 2.2 Based upon Swansea University Computer Society NET3.039 NET4: Unix domain sockets 1.0 for Linux NET4.0. NET4: Linux TCP/IP 1.0 for NET4.0 IP Protocols: ICMP, UDP, TCP Starting kswapd v 1.5 parport0: PC-style at 0x378 [SPP,ECP,ECPPS2] Detected PS/2 Mouse Port. Serial driver version 4.27 with no serial options enabled ttyS00 at 0x03f8 (irq = 4) is a 16550A ttyS01 at 0x02f8 (irq = 3) is a 16550A Real Time Clock Driver v1.09 loop: registered device at major 7 PIIX4: IDE controller on PCI bus 00 dev 39 PIIX4: not 100% native mode: will probe irqs later ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hda:pio, hdb:pio hda: IOMEGA ZIP 100 ATAPI, ATAPI FLOPPY drive ide0 at 0x1f0-0x1f7,0x3f6 on irq 14 hda: 98304kB, 96/64/32 CHS, 4096 kBps, 512 sector size, 2941 rpm hda: The drive reports both 100663296 and 0 bytes as its capacity Floppy drive(s): fd0 is 1.44M FDC 0 is a National Semiconductor PC87306 (scsi0) <Adaptec AHA-294X Ultra SCSI host adapter> found at PCI 14/0 (scsi0) Wide Channel, SCSI ID=7, 16/255 SCBs (scsi0) Downloading sequencer code... 413 instructions downloaded scsi0 : Adaptec AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI) 5.1.19/3.2.4 <Adaptec AHA-294X Ultra SCSI host adapter> scsi : 1 host. (scsi0:0:0:0) Synchronous at 40.0 Mbyte/sec, offset 8. Vendor: SEAGATE Model: ST19101W Rev: 0014 Type: Direct-Access ANSI SCSI revision: 02 Detected scsi disk sda at scsi0, channel 0, id 0, lun 0 (scsi0:0:1:0) Synchronous at 40.0 Mbyte/sec, offset 8. Vendor: SEAGATE Model: ST19101W Rev: 0014 Type: Direct-Access ANSI SCSI revision: 02 Detected scsi disk sdb at scsi0, channel 0, id 1, lun 0 (scsi0:0:2:0) Synchronous at 20.0 Mbyte/sec, offset 15. Vendor: PLEXTOR Model: CD-ROM PX-32TS Rev: 1.01 Type: CD-ROM ANSI SCSI revision: 02 Detected scsi CD-ROM sr0 at scsi0, channel 0, id 2, lun 0 (scsi0:0:4:0) Synchronous at 5.0 Mbyte/sec, offset 15. Vendor: SONY Model: SDT-9000 Rev: 0200 Type: Sequential-Access ANSI SCSI revision: 02 Detected scsi tape st0 at scsi0, channel 0, id 4, lun 0 scsi : detected 1 SCSI tape 1 SCSI cdrom 2 SCSI disks total. Uniform CDROM driver Revision: 2.55 SCSI device sda: hdwr sector= 512 bytes. Sectors= 17783240 [8683 MB] [8.7 GB] SCSI device sdb: hdwr sector= 512 bytes. Sectors= 17783240 [8683 MB] [8.7 GB] 3c59x.c:v0.99H 11/17/98 Donald Becker http://cesdis.gsfc.nasa.gov/linux/drivers/vortex.html eth0: 3Com 3c905B Cyclone 100baseTx at 0xe880, 00:10:4b:75:c9:f4, IRQ 10 8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface. MII transceiver found at address 24, status 786d. MII transceiver found at address 0, status 786d. Enabling bus-master transmits and whole-frame receives. Partition check: sda: sda1 sda2 sda3 sda4 sdb: sdb1 sdb2 sdb3 NTFS version 990411 VFS: Mounted root (ext2 filesystem) readonly. Freeing unused kernel memory: 48k freed Adding Swap: 130748k swap-space (priority -1) Adding Swap: 130748k swap-space (priority -2)
/proc/cpu: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 5 model name : Pentium II (Deschutes) stepping : 0 cpu MHz : 331.928246 cache size : 512 KB fdiv_bug : no hlt_bug : no sep_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx osfxsr bogomips : 330.96
processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 5 model name : Pentium II (Deschutes) stepping : 0 cpu MHz : 331.928246 cache size : 512 KB fdiv_bug : no hlt_bug : no sep_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx osfxsr bogomips : 331.78
/proc/scsi/aic7xxx/0: Adaptec AIC7xxx driver version: 5.1.19/3.2.4 Compile Options: TCQ Enabled By Default : Enabled AIC7XXX_PROC_STATS : Disabled AIC7XXX_RESET_DELAY : 5
Adapter Configuration: SCSI Adapter: Adaptec AHA-294X Ultra SCSI host adapter Ultra Wide Controller PCI MMAPed I/O Base: 0xfebef000 Adapter SEEPROM Config: SEEPROM found and used. Adaptec SCSI BIOS: Enabled IRQ: 11 SCBs: Active 0, Max Active 32, Allocated 45, HW 16, Page 255 Interrupts: 127933 BIOS Control Word: 0x18b6 Adapter Control Word: 0x005e Extended Translation: Enabled Disconnect Enable Flags: 0xffff Ultra Enable Flags: 0x0007 Tag Queue Enable Flags: 0x0003 Ordered Queue Tag Flags: 0x0003 Default Tag Queue Depth: 16 Tagged Queue By Device array for aic7xxx host instance 0: {0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0} Actual queue depth per device for aic7xxx host instance 0: {16,16,1,1,1,1,1,1,1,1,1,1,1,1,1,1}
Statistics:
(scsi0:0:0:0) Device using Wide/Sync transfers at 40.0 MByte/sec, offset 8 Transinfo settings: current(12/8/1/0), goal(12/8/1/0), user(12/15/1/0) Total transfers 43677 (39407 reads and 4270 writes)
(scsi0:0:1:0) Device using Wide/Sync transfers at 40.0 MByte/sec, offset 8 Transinfo settings: current(12/8/1/0), goal(12/8/1/0), user(12/15/1/0) Total transfers 84522 (79288 reads and 5234 writes)
(scsi0:0:2:0) Device using Narrow/Sync transfers at 20.0 MByte/sec, offset 15 Transinfo settings: current(12/15/0/0), goal(12/15/0/0), user(12/15/1/0) Total transfers 0 (0 reads and 0 writes)
(scsi0:0:4:0) Device using Narrow/Sync transfers at 5.0 MByte/sec, offset 15 Transinfo settings: current(50/15/0/0), goal(50/15/0/0), user(50/15/0/0) Total transfers 0 (0 reads and 0 writes)
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |