Messages in this thread | | | From | (G.W. Wettstein) | Date | Tue, 22 Dec 1998 09:22:19 -0600 | Subject | 2.0.36/Etherpower II hang and oops. |
| |
We have been chasing a problem with our production servers and the SMC Etherpower II cards and their drivers for a couple of months now. The cards are running at 10Mbit/sec and under intermediate to high-load the driver/card will hang. Upping and downing the interface once or sometimes twice will free the card and it will resume operation.
We have been able to tickle the bug with both uni-processor and SMP kernels in both the 2.0.35 and 2.0.36 kernels. Note that the driver version in 2.0.35 was the same as in 2.0.36 since I built the 2.0.35 kernels with updated drivers (v1.03) from Donald's site. The 2.0.36 kernel does seem marginally more resistant to the problem.
This morning I was able to get an OOPS out of an SMP-2.0.36 kernel when I was trying to revive a hung interface on one of our news servers. I am appending a piece from the syslog file which has all the kernel errors which were generated when we were working on the interface. Below this I am inserting a diagnostic dump from the epic-diag utilities that Donald has on his site. I am not sure whether this is useful to anyone other than Donald but I wanted to be complete.
I have been able to reproduce this hang on two separate servers and on three separate SMC cards. At this point I am reasonably convinced that we have a driver problem in the kernel. An interesting observation is that we have not been able to reproduce the hang with non-SCSI kernels. An IDE based machine with the same card plugged into the same hub has never offerred to hang. This may not be completely valid since the IDE based machine has only a single card while the SCSI machines have two SMC cards in them. The cards are on different interrupts so it shouldn't be an IRQ sharing problem.
The hang seems to involve the card being unable to handle receive packet interrupts. After a hang ifconfig reports n number of errors and reports an identical n number of dropped packets on the interface. The problem may also be secondary to networks that have intermediate to high collision rates. I can expand on my findings there if anyone is interested.
In the FWIW department I think that anyone needing absolutely bulletproof network connections would have to not use the SMC Etherpower II cards at this point until the problem can be tracked down.
syslog dump starts here. -------------------------------------------------- Dec 22 08:09:22 news kernel: Warning: kfree_skb passed an skb still on a list (from 0fd3ce20). Dec 22 08:09:22 news kernel: Warning: kfree_skb passed an skb still on a list (from 03671628). Dec 22 08:09:22 news kernel: general protection: 0000 Dec 22 08:09:22 news kernel: CPU: 0 Dec 22 08:09:22 news kernel: EIP: 0010:[kfree_skb+146/248] Dec 22 08:09:22 news kernel: EFLAGS: 00010282 Dec 22 08:09:22 news kernel: eax: 00000000 ebx: 865504c1 ecx: 0fe3b810 edx: 001a300a Dec 22 08:09:22 news kernel: esi: 03671628 edi: 00000000 ebp: 00001043 esp: 033cfec4 Dec 22 08:09:22 news kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 Dec 22 08:09:22 news kernel: Process ifconfig (pid: 18392, process nr: 44, stackpage=033cf000) Dec 22 08:09:22 news kernel: Stack: 0000000a 00000000 0000f800 0014104e 03671628 00000000 0000000a 0008a018 Dec 22 08:09:22 news kernel: 0019191e 03671628 00000000 001bef24 00000000 001bef24 0000000a 001417a8 Dec 22 08:09:22 news kernel: 001bef24 001bef24 bffffc84 033cff5c 00142185 001bef24 bffffc64 03337810 Dec 22 08:09:22 news kernel: Call Trace: [dev_kfree_skb+62/76] [epic_close+290/364] [dev_close+28/184] [dev_ifsioc+605/1540] [dev_ioctl+443/544] [inet_ioctl+867/936] [sock_ioctl+33/40] Dec 22 08:09:22 news kernel: [sys_ioctl+323/344] [system_call+258/320] Dec 22 08:09:22 news kernel: Code: 83 bb cc 00 00 00 00 74 25 85 ff 74 11 56 53 e8 f6 fc ff ff Dec 22 08:09:52 news kernel: eth0: Transmit timeout using MII device, Tx status 0003. Dec 22 08:10:03 news last message repeated 2 times Dec 22 08:11:03 news last message repeated 2 times Dec 22 08:11:20 news kernel: Trying to free free IRQ10 Dec 22 08:11:20 news kernel: kfree of non-kmalloced memory: 03671810, next= 00000000, order=6 syslog dump ends here. ----------------------------------------------------
epic-diag dump starts here. ----------------------------------------------- EPIC chip registers at 0xf800 0x000: 00000008 00cc84c0 0000733f 00000512 0000001d 00000061 00000000 00000000 0x020: 00000000 00000000 00000000 00085241 00080650 00080021 00080014 00083c60 0x040: 0008e000 00081c29 00083ad2 0008cf00 00080000 00080002 00080000 0008fff9 0x060: 00080004 00080021 00080077 00080074 00083c79 00082003 00082003 00081003 0x080: 0008a028 00088048 0fd3c9f6 0fd00820 0fd8020a 0fd80cdf 03640d62 03640c80 0x0A0: 036798ea fffc11d3 ffffffff fffc02ef fffc06ff c0200410 c0200758 03640798 0x0C0: 0008a228 0008a228 0008a31c 03bc6018 0390002a 0390003f 03900240 03900100 0x0E0: 03900100 03900003 e000ffff 0008a228 00008000 00000000 00000002 00000000 Interrupt sources are pending. Tx chain done indication. Tx Queue empty indication. Rx threshold crossed indication. Rx status valid indication. EEPROM contents: e000 1c29 3ad2 cf00 001d 1c08 10b8 a011 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0010 0000 1980 2100 0000 0000 0003 0000 0701 0000 0000 0000 4d53 3943 3334 5432 2058 2020 0000 0000 0280 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 The word-wide EEPROM checksum is 0x6571. Parsing the EEPROM of a EPIC/100: Station Address 00:E0:29:1C:D2:3A. Board name 'SMC9432TX ', revision 58. Calculated checksum is 00. MII PHY found at address 3. MII PHY #3 transceiver registers: 1000 782d 0181 4401 01e1 0021 0000 ffff ffff ffff ffff ffff ffff ffff ffff ffff 0040 0018 ffff ffff ffff ffff ffff ffff ffff ffff ffff 003e ffff 0040 0000 1dc4. epic-diag dump starts here. -----------------------------------------------
Have a pleasant remainder of the week.
Greg
As always, Dr. G.W. Wettstein Enjellic Systems Development - Specialists in 4206 N. 19th Ave. intranet based enterprise information solutions. Fargo, ND 58102 Phone: 701-281-1686 EMAIL: greg@wind.enjellic.com ------------------------------------------------------------------------------ "There are two kinds of people, those who do the work and those who take the credit. Try to be in the first group; there is less competition there." -- Indira Gandhi
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |