lkml.org 
[lkml]   [1996]   [Nov]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
Subject2.0.25 Frozen


Greetings,

This post ends my "lurk-only" mode for this newsgroup.

Since this is my first post, let me start out by THANKING the
principle players in the Linux community for being willing to
apply their skills to this effort.

WARNING: This post will be relatively long.
-------------------------------------------------------------------
----------- My background -----------------------------------------
-------------------------------------------------------------------
I am not a "kernel hacker". However, I have worked with many flavors
of Unix (since 1985) in both systems administration and programming
capacities. I have programmed in 'C' for several years, but have used
mostly Perl for the last year and a half. My kernel knowledge comes
from the usual reference works and a few classes. I am writing this
only to give some context to my post.
-------------------------------------------------------------------
----------- Problem Overview/Description --------------------------
-------------------------------------------------------------------
A co-worker and I were getting ready to put a Linux machine into a
production role. Before doing so, we wanted to "stress test" it a
little. Here is what we did (high level):

01) Installed Slackware 96 as a baseline
02) Got latest Linux production kernel (2.0.25), built, and booted it.
03) Ran the following Perl script from "/usr/src/linux" to repeatedly
build 2.0.25. one-thousand times:
-------------------------------------------------------------------
#!/usr/bin/perl -w
###################################################################
# Note - This script is a Perl incarnation of the script
# suggested at:
# http://www.bitwizard.nl/sig11/
###################################################################
$good_builds = 0;

for($x = 1; $x <= 1000; $x++)
{
$my_temp_file = sprintf("%s-%04.4d-TMP", $^T, $x);
`make clean 1>$my_temp_file 2>&1`;
if ($? == 0)
{
`make zImage 1>>$my_temp_file 2>&1`;
if ($? == 0)
{
$good_builds++;
unlink($my_temp_file);
}
}
$date_time = `date`;
chop($date_time);
printf "[%06.6d] of [%06.6d] builds GOOD [%s]\n",
$good_builds, $x, $date_time;
}
exit;
-------------------------------------------------------------------
----------- The Problem -------------------------------------------
-------------------------------------------------------------------
The kernel building stopped at build #72.

The machine was basically frozen [ I am in Minnesota though *8^) ].

I could switch VC's, and was able to do a "Shift-Scrollock" to
show memory and a "Ctrl-Scrollock" to show state. Then, the machine
totally froze.

Here are the pertinent(?) "syslog" messages (with first few fields
"cut" out and some formatting):
-------------------------------------------------------------------
free sibling
task PC stack pid father child younger older
swapper 0 R 00000000 4096 0 0 1
init -1 R current 4 1 0 16779
kflushd 2 S 00000000 4 2 1 3
kswapd 3 S 00000000 4 3 1 4 2
nfsiod 4 S 00000000 3520 4 1 5 3
nfsiod 5 S 00000000 3520 5 1 6 4
nfsiod 6 S 00000000 3520 6 1 7 5
nfsiod 7 S 00000000 3520 7 1 13 6
bash 8 S 00000000 4 74 1 1985 76 73
agetty 9 R FFFFFFFF 4 554 1 16779 79
crond 10 R 00000000 4 37 1 51 13
syslogd 11 R FFFFFFFF 4 51 1 53 37
update 12 R 00000000 4 13 1 37 7
klogd 13 R 00000000 1968 53 1 55 51
rpc.portmap 14 S FFFFFFFF 3236 55 1 57 53
inetd 15 R FFFFFFFF 1548 57 1 59 55
lpd 16 S FFFFFFFF 3376 59 1 62 57
rpc.nfsd 17 R FFFFFFFF 4 64 1 70 62
rpc.mountd 18 R FFFFFFFF 3516 62 1 64 59
sendmail 19 R 00000000 3244 70 1 73 64
script 20 S FFFFFFFF 4 1985 74 1986
gpm 21 R 001E1708 3404 73 1 74 70
agetty 22 R FFFFFFFF 4 76 1 77 74
agetty 23 S FFFFFFFF 1500 77 1 78 76
agetty 24 S FFFFFFFF 4 78 1 79 77
agetty 25 S FFFFFFFF 2772 79 1 554 78
script 26 S FFFFFFFF 4 1986 1985 1987
bash 27 S 00000000 12 1987 1986 1988
djw-test.pl 28 S 00000000 4 1988 1987 16782
djw-test.pl 29 R 00000000 4 16782 1988
cpp 37 R 00000000 1968 16779 1 554
Mem-info:
Free pages: 5120kB
( 346*4kB 241*8kB 75*16kB 3*32kB 0*64kB 4*128kB = 5120kB)
Swap cache: add 40/40, delete 26789394/40, find 74/0
Free swap: 200724kB
16384 pages of RAM
1294 free pages
556 reserved pages
1810 pages shared
Buffer memory: 39420kB
Buffer heads: 40980
Buffer blocks: 39420
Unable to handle kernel paging request at virtual address c7478124
current->tss.cr3 = 00fda000, |r3 = 00fda000
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[show_buffers+137/392]
EFLAGS: 00010207
eax: 0001882f ebx: 00000487 ecx: 07478110 edx: 001f7000
esi: 00000000 edi: 00005a81 ebp: 00000000 esp: 0001ce9c
ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018
Process init (pid: 1, process nr: 1, stackpage=0001c000)
Stack: 00004000 0000050e 0000022c 00000712 00000001 0001882f 001f7000 00000000
0000046b 0000004b 00000000 0010f8bb 00000002 0000f203 00000046 001e6ae8
001f7000 00185fa5 00185b35 00000003 00000000 0009e098 00000000 00000001
Call Trace: [show_mem+187/200] [do_spec+29/32] [handle_scancode+833/904]
[timer_bh+172/332] [keyboard_interrupt+46/72] [do_IRQ+45/80]
[IRQ1_interrupt+86/128]
[sync_buffers+76/396] [fsync_dev+14/48]
[sys_sync+7/16] [system_call+82/128]
Code: 8b 41 14 a8 04 74 04 ff 44 24 28 a8 40 74 04 ff 44 24 1c a8
Aiee, killing interrupt handler
-------------------------------------------------------------------

Here is the "tail" end of the build #72 temp file (slightly formatted):
-------------------------------------------------------------------
gcc -D__KERNEL__ -I/usr/src/linux-2.0.25/include -Wall
-Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strength-reduce
-pipe -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2
-DCPU=686 -c -o tcp.o tcp.c
gcc -D__KERNEL__ -I/usr/src/linux-2.0.25/include -Wall
-Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strength-reduce
-pipe -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2
-DCPU=686 -c -o tcp_input.o tcp_input.c
gcc: Internal compiler error: program cc1 got fatal signal 11
make[3]: *** [tcp_input.o] Error 1
make[3]: Leaving directory `/usr/src/linux-2.0.25/net/ipv4'
make[2]: *** [first_rule] Error 2
make[2]: Leaving directory `/usr/src/linux-2.0.25/net/ipv4'
make[1]: *** [sub_dirs] Error 2
make[1]: Leaving directory `/usr/src/linux-2.0.25/net'
make: *** [linuxsubdirs] Error 2
-------------------------------------------------------------------
-------------------------------------------------------------------
----------- Hardware ----------------------------------------------
-------------------------------------------------------------------
The machine is a brand new Micron Millennia Pro Plus.

Memory: 63312k/65536k available (660k kernel code, 384k reserved, 1180k data)
tty00 at 0x03f8 (irq = 4) is a 16550A
tty01 at 0x02f8 (irq = 3) is a 16550A
PS/2 auxiliary pointing device detected -- driver installed.
Floppy drive(s): fd0 is 1.44M
AHA-2940 Ultra (PCI-bus), I/O 0xfc00, Mem 0xffbef000:
Vendor: SEAGATE Model: ST51080N Rev: 0958
Vendor: MICROP Model: 1991-27Q4D Rev: Q4D
Vendor: SONY Model: SDT-5000 Rev: 3.30
Vendor: PLEXTOR Model: CD-ROM PX-8XCS Rev: 1.02
eth0: 3Com 3c595 Vortex 100baseTX at 0xff80, 00:60:97:24:40:6e, IRQ 5

[/proc/cpuinfo]
processor : 0
cpu : 686
model : Pentium Pro
vendor_id : GenuineIntel
stepping : 7
fdiv_bug : no
hlt_bug : no
fpu : yes
fpu_exception : yes
cpuid : yes
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 11 mtrr pge mca cmov
bogomips : 199.07

[/proc/interrupts]
0: 938188 timer
1: 1961 keyboard
2: 0 cascade
5: 33564 3c595 Vortex 100baseTX
10: 14543 + aic7xxx
12: 72 PS/2 Mouse
13: 1 math error

[/proc/ioports]
0000-001f : dma1
0020-003f : pic1
0040-005f : timer
0060-006f : keyboard
0080-009f : dma page reg
00a0-00bf : pic2
00c0-00df : dma2
00f0-00ff : npu
02f8-02ff : serial(auto)
03c0-03df : vga+
03f0-03f5 : floppy
03f7-03f7 : floppy DIR
03f8-03ff : serial(auto)
fc00-fcbe : aic7xxx
ff80-ff9f : 3c595 Vortex 100baseTX

[/proc/pci] (slightly formatted)
PCI devices found:
Bus 0, device 17, function 0:
SCSI storage controller: Adaptec AIC-7881U (rev 0).
Medium devsel. Fast back-to-back capable. IRQ 10. Master Capable.
Latency=64. Min Gnt=8.Max Lat=8.
I/O at 0xfc00.
Non-prefetchable 32 bit memory at 0xffbef000.
Bus 0, device 15, function 0:
Ethernet controller: 3Com 3C595 100bTX (rev 0).
Medium devsel. IRQ 5. Master Capable.
Latency=248. Min Gnt=3.Max Lat=8.
I/O at 0xff80.
Bus 0, device 11, function 0:
VGA compatible controller: S3 Inc. ViRGE (rev 5).
Medium devsel. IRQ 11. Master Capable.
Latency=64. Min Gnt=4.Max Lat=255.
Non-prefetchable 32 bit memory at 0xf8000000.
Bus 0, device 7, function 1:
IDE interface: Intel 82371SB Natoma/Triton II PIIX3 (rev 0).
Medium devsel. Fast back-to-back capable. Master Capable.
Latency=64.
I/O at 0xffa0.
Bus 0, device 7, function 0:
ISA bridge: Intel 82371SB Natoma/Triton II PIIX3 (rev 1).
Medium devsel. Fast back-to-back capable. Master Capable.
No bursts.
Bus 0, device 0, function 0:
Host bridge: Intel 82441FX Natoma (rev 2).
Medium devsel. Fast back-to-back capable. Master Capable.
Latency=64.
-------------------------------------------------------------------
----------- OS/Configuration --------------------------------------
-------------------------------------------------------------------
Linux version 2.0.25 (root@spare) (gcc version 2.7.2)
#1 Wed Nov 20 13:24:33 CST 1996
-------------------------------------------------------------------
Configuration (ONLY affirmative)

CONFIG_MODULES=y
CONFIG_NET=y
CONFIG_PCI=y
CONFIG_SYSVIPC=y
CONFIG_BINFMT_AOUT=y
CONFIG_BINFMT_ELF=y
CONFIG_KERNEL_ELF=y
CONFIG_M686=y
CONFIG_BLK_DEV_FD=y
CONFIG_INET=y
CONFIG_IP_NOSR=y
CONFIG_SKB_LARGE=y
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
CONFIG_CHR_DEV_ST=y
CONFIG_BLK_DEV_SR=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_AIC7XXX=y
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
CONFIG_NET_ETHERNET=y
CONFIG_NET_VENDOR_3COM=y
CONFIG_VORTEX=y
CONFIG_MINIX_FS=y
CONFIG_EXT2_FS=y
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_PROC_FS=y
CONFIG_NFS_FS=y
CONFIG_ISO9660_FS=y
CONFIG_SERIAL=y
CONFIG_MOUSE=y
CONFIG_PSMOUSE=y
-------------------------------------------------------------------
Some other comments. We have looked at the "SIG11 problem" material
located at:
http://www.bitwizard.nl/sig11/

We have been running a stand-alone memory test utility,
"MEMTEST-86 v1.1", to perceive errors. We have run this for periods
as long as twelve(12) hours without any errors being flagged.

At this point in time, any/all suggestions welcome. If more
information is needed, let me know. In the meantime, we will
keep trying to get a stable H/W & S/W combination.

TIA,

Dave Worden
<dworden@us.teltech.com>

\
 
 \ /
  Last update: 2005-03-22 13:38    [W:0.043 / U:0.352 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site