lkml.org 
[lkml]   [2009]   [Jul]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectOOM killer in 2.6.31-rc2
Date
Greetings all;

I had to hard reset this box just now as there was no response to a ctl-alt-
bksp when X was un-responsive this morning.

I had built a 2nd version of the 2.6.31-rc2 kernel last night when I found the
video stuff appeared to have been moved in the .config and my pcHDTV-3000
cards modules were not being built, but are now. That kernel was installed,
and if at some time in the night a module was needed, it would have been
available, but I can't make a solid connection.

This machine will always be marked as 'tainted' because any bios update that
fixes the very early boot time oops, also leaves me with a machine that will
crash hard in 30 seconds to 3 or 4 hours. The fixes done by the oops make it
generally dead stable for weeks. That oops:

Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-provided physical RAM map:
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 0000000000100000 - 00000000dfee0000 (usable)
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000dfee0000 - 00000000dfee3000 (ACPI NVS)
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000dfee3000 - 00000000dfef0000 (ACPI data)
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000dfef0000 - 00000000dff00000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000f0000000 - 00000000f4000000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] BIOS-e820: 0000000100000000 - 0000000120000000 (usable)
Jul 6 10:03:58 coyote kernel: [ 0.000000] DMI 2.4 present.
Jul 6 10:03:58 coyote kernel: [ 0.000000] Phoenix BIOS detected: BIOS may corrupt low RAM, working around it.
Jul 6 10:03:58 coyote kernel: [ 0.000000] last_pfn = 0x120000 max_arch_pfn = 0x1000000
Jul 6 10:03:58 coyote kernel: [ 0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
Jul 6 10:03:58 coyote kernel: [ 0.000000] ------------[ cut here ]------------
Jul 6 10:03:58 coyote kernel: [ 0.000000] WARNING: at arch/x86/kernel/cpu/mtrr/generic.c:456
generic_get_mtrr+0x12c/0x150()
Jul 6 10:03:58 coyote kernel: [ 0.000000] Hardware name: System Product Name
Jul 6 10:03:58 coyote kernel: [ 0.000000] mtrr: your BIOS has set up an incorrect mask, fixing it up.
Jul 6 10:03:58 coyote kernel: [ 0.000000] Modules linked in:
Jul 6 10:03:58 coyote kernel: [ 0.000000] Pid: 0, comm: swapper Not tainted 2.6.31-rc2 #2
Jul 6 10:03:58 coyote kernel: [ 0.000000] Call Trace:
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c101449c>] ? generic_get_mtrr+0x12c/0x150
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c103693d>] warn_slowpath_common+0x7d/0xe0
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c101449c>] ? generic_get_mtrr+0x12c/0x150
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c1036a13>] warn_slowpath_fmt+0x33/0x50
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c101449c>] generic_get_mtrr+0x12c/0x150
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c1422e1a>] mtrr_trim_uncached_memory+0x85/0x368
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c142191e>] ? mtrr_bp_init+0x1d9/0x2bb
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c141c369>] setup_arch+0x52c/0xa33
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c11c0020>] ? thermal_get_trip_type+0x0/0x9c
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c1418bb4>] start_kernel+0xb2/0x38b
Jul 6 10:03:58 coyote kernel: [ 0.000000] [<c1418394>] i386_start_kernel+0x84/0xb0
Jul 6 10:03:58 coyote kernel: [ 0.000000] ---[ end trace a7919e7f17c0a725 ]---
Jul 6 10:03:58 coyote kernel: [ 0.000000] Scanning 0 areas for low memory corruption
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified physical RAM map:
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 0000000000000000 - 0000000000010000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 0000000000010000 - 000000000009f000 (usable)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 000000000009f000 - 00000000000a0000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000000f0000 - 0000000000100000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 0000000000100000 - 00000000dfee0000 (usable)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000dfee0000 - 00000000dfee3000 (ACPI NVS)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000dfee3000 - 00000000dfef0000 (ACPI data)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000dfef0000 - 00000000dff00000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000f0000000 - 00000000f4000000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 00000000fec00000 - 0000000100000000 (reserved)
Jul 6 10:03:58 coyote kernel: [ 0.000000] modified: 0000000100000000 - 0000000120000000 (usable)
Jul 6 10:03:58 coyote kernel: [ 0.000000] init_memory_mapping: 0000000000000000-00000000379fe000
Jul 6 10:03:58 coyote kernel: [ 0.000000] NX (Execute Disable) protection: active

You all have seen this one before, several times. I have asked that
since its a good fix, that the kernel not be marked tainted in that
instance. I would run the asus bios that didn't do that _IF_ it was
stable. 2 newer versions are _not_ stable, this is stable after the fix.

The machine has 4G of ram & is I believe 'pae'

The oom's first stanza:

Jul 6 06:45:01 coyote kernel: [78748.106803] X invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0
Jul 6 06:45:01 coyote kernel: [78748.106808] Pid: 3068, comm: X Tainted: G W 2.6.31-rc2 #1
Jul 6 06:45:01 coyote kernel: [78748.106811] Call Trace:
Jul 6 06:45:01 coyote kernel: [78748.106818] [<c1308513>] ? printk+0x23/0x40
Jul 6 06:45:01 coyote kernel: [78748.106823] [<c107e268>] oom_kill_process+0x178/0x270
Jul 6 06:45:01 coyote kernel: [78748.106827] [<c107e6ad>] ? badness+0x14d/0x220
Jul 6 06:45:01 coyote kernel: [78748.106830] [<c107e8c2>] __out_of_memory+0x142/0x170
Jul 6 06:45:01 coyote kernel: [78748.106834] [<c107e949>] out_of_memory+0x59/0xc0
Jul 6 06:45:01 coyote kernel: [78748.106837] [<c1081d17>] __alloc_pages_nodemask+0x4f7/0x510
Jul 6 06:45:01 coyote kernel: [78748.106841] [<c1081db3>] __get_free_pages+0x23/0x50
Jul 6 06:45:01 coyote kernel: [78748.106845] [<c10bf8b2>] __pollwait+0xb2/0xf0
Jul 6 06:45:01 coyote kernel: [78748.106848] [<c12f6528>] unix_poll+0x28/0xc0
Jul 6 06:45:01 coyote kernel: [78748.106851] [<c1281b7e>] sock_poll+0x1e/0x40
Jul 6 06:45:01 coyote kernel: [78748.106853] [<c10bee8e>] do_select+0x34e/0x6b0
Jul 6 06:45:01 coyote kernel: [78748.106871] [<c10bf800>] ? __pollwait+0x0/0xf0
Jul 6 06:45:01 coyote kernel: [78748.106874] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106877] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106879] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106882] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106884] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106887] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106890] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106892] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106895] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106897] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106900] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106902] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106905] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106907] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106910] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106913] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106915] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106918] [<c10bf8f0>] ? pollwake+0x0/0x90
Jul 6 06:45:01 coyote kernel: [78748.106920] [<c10bf402>] core_sys_select+0x212/0x350
Jul 6 06:45:01 coyote kernel: [78748.106925] [<c10be292>] ? poll_select_set_timeout+0x82/0x90
Jul 6 06:45:01 coyote kernel: [78748.106928] [<c10bf761>] sys_select+0x51/0xf0
Jul 6 06:45:01 coyote kernel: [78748.106931] [<c10031b7>] sysenter_do_call+0x12/0x22
Jul 6 06:45:01 coyote kernel: [78748.106933] Mem-Info:
Jul 6 06:45:01 coyote kernel: [78748.106935] DMA per-cpu:
Jul 6 06:45:01 coyote kernel: [78748.106937] CPU 0: hi: 0, btch: 1 usd: 0
Jul 6 06:45:01 coyote kernel: [78748.106939] CPU 1: hi: 0, btch: 1 usd: 0
Jul 6 06:45:01 coyote kernel: [78748.106941] CPU 2: hi: 0, btch: 1 usd: 0
Jul 6 06:45:01 coyote kernel: [78748.106943] CPU 3: hi: 0, btch: 1 usd: 0
Jul 6 06:45:01 coyote kernel: [78748.106944] Normal per-cpu:
Jul 6 06:45:01 coyote kernel: [78748.106946] CPU 0: hi: 186, btch: 31 usd: 118
Jul 6 06:45:01 coyote kernel: [78748.106948] CPU 1: hi: 186, btch: 31 usd: 171
Jul 6 06:45:01 coyote kernel: [78748.106950] CPU 2: hi: 186, btch: 31 usd: 159
Jul 6 06:45:01 coyote kernel: [78748.106952] CPU 3: hi: 186, btch: 31 usd: 172
Jul 6 06:45:01 coyote kernel: [78748.106954] HighMem per-cpu:
Jul 6 06:45:01 coyote kernel: [78748.106955] CPU 0: hi: 186, btch: 31 usd: 56
Jul 6 06:45:01 coyote kernel: [78748.106957] CPU 1: hi: 186, btch: 31 usd: 20
Jul 6 06:45:01 coyote kernel: [78748.106959] CPU 2: hi: 186, btch: 31 usd: 53
Jul 6 06:45:01 coyote kernel: [78748.106961] CPU 3: hi: 186, btch: 31 usd: 180
Jul 6 06:45:01 coyote kernel: [78748.106965] Active_anon:90702 active_file:136927 inactive_anon:26328
Jul 6 06:45:01 coyote kernel: [78748.106966] inactive_file:1956 unevictable:25 dirty:4 writeback:0 unstable:0
Jul 6 06:45:01 coyote kernel: [78748.106967] free:560899 slab:206505 mapped:19048 pagetables:3220 bounce:0
Jul 6 06:45:01 coyote kernel: [78748.106971] DMA free:3496kB min:64kB low:80kB high:96kB active_anon:0kB
inactive_anon:0kB acti
ve_file:12kB inactive_file:8kB unevictable:0kB present:15804kB pages_scanned:0 all_unreclaimable? yes
Jul 6 06:45:01 coyote kernel: [78748.106983] lowmem_reserve[]: 0 0 25406 25406
Jul 6 06:45:01 coyote kernel: [78748.106988] HighMem free:2236464kB min:512kB low:3928kB high:7348kB
active_anon:362808kB inact
ive_anon:105308kB active_file:547220kB inactive_file:7704kB unevictable:100kB present:3252052kB pages_scanned:0
all_unreclaimabl
e? no
Jul 6 06:45:01 coyote kernel: [78748.106991] lowmem_reserve[]: 0 0 0 0
Jul 6 06:45:01 coyote kernel: [78748.106994] DMA: 310*4kB 204*8kB 27*16kB 6*32kB 0*64kB 0*128kB 0*256kB 0*512kB
0*1024kB 0*2048
kB 0*4096kB = 3496kB
Jul 6 06:45:01 coyote kernel: [78748.107002] Normal: 1*4kB 0*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB
1*1024kB 1*2048kB
0*4096kB = 3636kB
Jul 6 06:45:01 coyote kernel: [78748.107009] HighMem: 45888*4kB 44682*8kB 32844*16kB 18479*32kB 6641*64kB 1114*128kB
37*256kB 1
*512kB 1*1024kB 0*2048kB 0*4096kB = 2236464kB
Jul 6 06:45:01 coyote kernel: [78748.107017] 139771 total pagecache pages
Jul 6 06:45:01 coyote kernel: [78748.107019] 616 pages in swap cache
Jul 6 06:45:01 coyote kernel: [78748.107021] Swap cache stats: add 7937, delete 7321, find 2362/2515
Jul 6 06:45:01 coyote kernel: [78748.107023] Free swap = 8360652kB
Jul 6 06:45:01 coyote kernel: [78748.107024] Total swap = 8385912kB
Jul 6 06:45:01 coyote kernel: [78748.121323] 1179632 pages RAM
Jul 6 06:45:01 coyote kernel: [78748.121325] 951810 pages HighMem
Jul 6 06:45:01 coyote kernel: [78748.121327] 146165 pages reserved
Jul 6 06:45:01 coyote kernel: [78748.121328] 149491 pages shared
Jul 6 06:45:01 coyote kernel: [78748.121329] 441615 pages non-shared
Jul 6 06:45:01 coyote kernel: [78748.121332] Out of memory: kill process 2385 (mysqld) score 15451 or a child
Jul 6 06:45:01 coyote kernel: [78748.121334] Killed process 2385 (mysqld)

and continued to:

Jul 6 06:45:01 coyote kernel: [78748.137525] Killed process 30192 (spamd)
Jul 6 06:45:01 coyote kernel: [78748.154292] Killed process 2506 (httpd)
Jul 6 06:45:01 coyote kernel: [78748.170851] Killed process 2507 (httpd)
Jul 6 06:45:01 coyote kernel: [78748.187519] Killed process 2508 (httpd)
Jul 6 06:45:01 coyote kernel: [78748.320880] Killed process 2510 (httpd)
Jul 6 06:45:01 coyote kernel: [78748.337529] Killed process 2511 (httpd)
Jul 6 06:45:05 coyote kernel: [78751.467158] Killed process 948 (spamd)
Jul 6 06:45:05 coyote kernel: [78751.483535] Killed process 963 (httpd)
Jul 6 06:45:05 coyote kernel: [78751.500194] Killed process 964 (httpd)
Jul 6 06:45:05 coyote kernel: [78751.502688] Killed process 965 (httpd)
Jul 6 06:45:05 coyote kernel: [78751.506025] Killed process 2512 (httpd)
Jul 6 06:45:05 coyote kernel: [78751.509376] Killed process 2513 (httpd)
Jul 6 06:45:05 coyote kernel: [78751.512713] Killed process 2514 (httpd)
Jul 6 06:45:05 coyote kernel: [78751.516057] Killed process 1995 (krunner_lock)
Jul 6 06:45:05 coyote kernel: [78751.519481] Killed process 3391 (kwin)
Jul 6 06:45:05 coyote kernel: [78751.522674] Killed process 3396 (plasma)
Jul 6 06:45:05 coyote kernel: [78751.526006] Killed process 6931 (kmail)
Jul 6 06:45:05 coyote kernel: [78751.542786] Killed process 16309 (spamd)
Jul 6 06:45:05 coyote kernel: [78751.546042] Killed process 3421 (krunner)
Jul 6 06:45:05 coyote kernel: [78751.549521] Killed process 3357 (klauncher)
Jul 6 06:45:05 coyote kernel: [78751.552760] Killed process 3526 (kcalc)
Jul 6 06:45:05 coyote kernel: [78751.759570] Killed process 3068 (X)
Jul 6 06:45:05 coyote kernel: [78751.765648] Killed process 3389 (ksmserver)

The oom started at 6:45:01 this morning. mysqld wasn't doing anything & the
only reason its even started is for mythtv, which is how I found my
pcHDTV-3000 was on the missing list even if occupying a slot.

I've no idea if the rebuilt (with v4l drivers now) will also crash.
It feels normal. And looks normal in htop's display, using 538M of 4096M,
no swap used yet.

From my .config:

# grep MEM .config
CONFIG_SHMEM=y
# CONFIG_MEMTEST is not set
# CONFIG_NOHIGHMEM is not set
# CONFIG_HIGHMEM4G is not set
CONFIG_HIGHMEM64G=y
CONFIG_HIGHMEM=y
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_SPARSEMEM_STATIC=y
CONFIG_X86_BOOTPARAM_MEMORY_CORRUPTION_CHECK=y
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_INPUT_FF_MEMLESS is not set
CONFIG_DEVKMEM=y
CONFIG_FIX_EARLYCON_MEM=y
# CONFIG_HW_RANDOM_TIMERIOMEM is not set
# CONFIG_MEMSTICK is not set
CONFIG_FIRMWARE_MEMMAP=y
CONFIG_DEBUG_MEMORY_INIT=y
CONFIG_HAVE_ARCH_KMEMCHECK=y
CONFIG_STRICT_DEVMEM=y
CONFIG_HAS_IOMEM=y

In case I have some option miss-set in that, plz advise.

Thanks.

--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
The NRA is offering FREE Associate memberships to anyone who wants them.
<https://www.nrahq.org/nrabonus/accept-membership.asp>

Mal: "Shouldn't have been you - they shoulda hit us."

Book: "That crossed my mind..."
--"Serenity"



\
 
 \ /
  Last update: 2009-07-06 17:01    [W:0.123 / U:0.088 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site