Messages in this thread | | | Date | Mon, 20 Sep 1999 17:44:06 -0500 | From | Brandon L Black <> | Subject | Strange 2.2.X hard hang.... |
| |
I've tried to sort this out myself, but I'm in over my head and beating it against the wall... I'm hoping some of whatever I say will ring a bell in someone else's head who might have some clues...
If you don't have time to read a long messy ramble, skip this email :)
I'll start with an overview of the hardware and software involved:
ASUS P2B-S m/b (bios rev 1.08) Built-in AIC7890 controller w/ Seagate Cheetah 9LP 9G 80MB/s LVD drive. PIII-500 cpu 128MB SDRAM IDE CDROM on builtin PIIX4 controller Voodoo3 graphics (have replaced with Permedia2 and TNT with no affect on problem, just in case) all other peripherals are removed to cut down on variables debugging this problem.
OS is RedHat 6.0 with all the up to dates rpm-ed fixes from their errata page.
Compiler is egcs-1.1.2. as86/ld86 is the latest Dev86 package
With the stock redhat-supplied 2.2.5-22-i686 kernel binary, things go smoothly the vast majority of the time, but the machine freezes on rare occasions (for this whole email, "Freeze" means the computer just freezes... all activity stops, no oops, no gpf, no errors.... no alt-sysreq... no ctrl-alt-del... no numlock light... etc...)
I grabbed stock 2.2.12 and 2.2.13pre9, compiled them without patches or wierd settings. I also tried various compiler options, and the PIII patches from Doug Ledford's page.
In all permutations, I randomly ran into the following two problems:
1) LILO loads kernel, kernel starts decompressing, and quickly hangs with "Out of Memory" ... some kernels that I build just always fail like this, some kernels that I build happen to get past decompression without problems.... I can't see any obvious pattern to which ones have this problem
2) On the ones that actually boot, they generally boot sanely all the way up. However, I get the same random hangs as the redhat 2.2.5-22 kernel, but they are _much_ more frequent (can't go much more than 15 minutes without it happening).
Ignoring the whacko "Out of Memory" thing for now.... I eventually was able to reproduce the random hangs fairly well. All I had to do was fsck large filesystems to induce the hang (or let the init scripts fsck large filesystems on bootup after a previous crash). In the final stage of the fsck (or maybe just after fsck finishes?), the hang happens.
I went back to the 2.2.5-22 kernel, and I could reproduce the hang via the same fsck method, but not as reliably (i.e. it was about a 20-30% chance the fsck would kill me in 2.2.5-22, vs 80-90% in the 2.2.12/13pre kernel)....
The other clue I got was that _one_ single time, I actually got a kernel GPF error in an fsck run (which coredumped that fsck).. then the system proceeded to fsck another filesystem, which then hung the system.
The GPF (and this happened once out of probably hundreds of boots during this debugging I've been doing over several days) 's call trace led down to being in getblk() according to System.map.
I tried to insert a few printk()'s into getblk() (one at the beginning, then one just before each exit point) to trace each entry and exit of that function, hoping I would see the machine hang between the printk's, which might indicate that whatever happened was happening in there... but just after I added the printk's I started getting kernel builds that had the "Out of Memory" on startup again (which hand't been happening for a while.....) ... and it was late and I gave up for the time being....
The next day (yesterday), I grabbed and compiled/installed 2.3.18. It runs like a champ. No hangs, no freezes... had it up all night last night and this morning, ran many-many-many fsck's, rebuilt glibc and egcs 10 or 15 times in row each, in parallel, as a stress test, from xterms with KDE fully up and running... no problems.
Prior to seeing 2.3.18 work flawlessly, I was prepared to give up and decide that I must have a faulty peice of hardware somewhere.... and maybe its just that I have a slightly-faulty peice of hardware, and 2.3.18 doesn't step on it as bad as 2.2.5-22, which doesn't step on it as bad as 2.2.12/13pre..... I don't know... I'm lost :)
Any help?
Brandon (please cc: brandon.black@wcom.com)
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |