[lkml]   [2004]   [Oct]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [OOPS] 2.6.9-rc4, dual Opteron, NUMA, 8GB
On Wed, 13 Oct 2004, Andi Kleen wrote:

> On Wed, 13 Oct 2004 13:34:01 -0700 "Randy.Dunlap" <> wrote:
> >
> > > Who's responsible for the K8_NUMA stuff? I'd love to work with them to
> > > narrow this down.
> >
> > Andi Kleen (SUSE). Copied.
> It looks like memory corruption somewhere. I suspect not directly related to NUMA,
> but the different memory layout with NUMA may trigger it.
> I would enable CONFIG_DEBUG_SLAB and CONFIG_DEBUG_PAGEALLOC and see if that
> triggers it elsewhere.
> First suspection would be the device driver. Perhaps you can test it with
> a different block device?
> -Andi



It turns out that NUMA is the culprit and LVM has no effect on any
configuration. The machine has 6 memory slots. If I have 4 sticks of
2GB in the machine, it makes zone 0 w/ 8GB and zone 1 disabled. If I
add two more 2GB sticks (total 12GB, filling all possible 6 slots),
then I have 8GB in zone 0 and 4GB in zone 1, and then a mke2fs on a
280GB /dev/sdb1 works fine.

In conclusion:

NUMA + 12 GB -> works
no numa + 8 GB -> works

And I suspect that without CONFIG_SMP, 8GB will also work, by virtue
of there only being one NUMA zone. I haven't tested that yet, but
Randy referred me to these posts, where somebody with the same problem
confirms that disabling NUMA and SMP fixes his problem:

When I boot NUMA w/ 12GB, I get:

Scanning NUMA topology in Northbridge 24
Number of nodes 2 (10010)
Node 0 MemBase 0000000000000000 Limit 00000001ffffffff
Node 1 MemBase 0000000200000000 Limit 00000002ffffffff
node 1 shift 24 addr 200000000 conflict 0
node 1 shift 25 addr 200000000 conflict 0
Using node hash shift of 26
Bootmem setup node 0 0000000000000000-00000001ffffffff
Bootmem setup node 1 0000000200000000-00000002ffffffff
No mptable found.
On node 0 totalpages: 2097151
DMA zone: 4096 pages, LIFO batch:1
Normal zone: 2093055 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1
On node 1 totalpages: 1048575
DMA zone: 0 pages, LIFO batch:1
Normal zone: 1048575 pages, LIFO batch:16
HighMem zone: 0 pages, LIFO batch:1

When I boot NUMA w/ 8GB, k8topology.c prints out:

Node 0 MemBase 0000000000000000 Limit 00000001ffffffff
Skipping disabled node 1

I can get you the full dmesg of both configs tomorrow.

Anything else you'd like to see?

- Brad
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2005-03-22 14:07    [W:0.041 / U:7.144 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site