lkml.org 
[lkml]   [2012]   [Feb]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Memory issues with Opteron 6220

* Anders Ossowicki <aowi@novozymes.com> wrote:

> Hey,
>
> We're seeing unexpected slowdowns and other memory issues with a new system.
> Enough to render it unusable. For example:
>
> Error: open3: fork failed: Cannot allocate memory
>
> at times where there's no real memory pressure:
> total used free shared buffers cached
> Mem: 132270720 131942388 328332 0 299768 103334420
> -/+ buffers/cache: 28308200 103962520
> Swap: 7811068 13760 7797308
>
> [...]

> The system is a Dell Poweredge R715, with two eight-core
> Opteron 6220 processors and 128G of memory. We have several
> similar systems, such as the one this should replace: R715,
> 2x8 core Opteron 6140, 128G memory, and they do not exhibit
> any similar symptoms.

130 MB of RAM visible to Linux isn't the expected bootup default
indeed. Around 130 *GB* would be expected ...

> We have tried with 2.6.37, 2.6.38, 3.2.5 and 3.3-rc1 with no luck. The
> microcode updates from AMD have not helped either.

Nasty.

No smoking gun in the dmesg:

> dmesg is available at http://dev.exherbo.org/~arkanoid/atlas-dmesg-3.2.5.txt

[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
[ 0.000000] BIOS-e820: 0000000000100000 - 00000000df679000 (usable)
[ 0.000000] BIOS-e820: 00000000df679000 - 00000000df68f000 (reserved)
[ 0.000000] BIOS-e820: 00000000df68f000 - 00000000df6ce000 (ACPI data)
[ 0.000000] BIOS-e820: 00000000df6ce000 - 00000000e0000000 (reserved)
[ 0.000000] BIOS-e820: 00000000f0000000 - 00000000f4000000 (reserved)
[ 0.000000] BIOS-e820: 00000000fe000000 - 00000000fec90000 (reserved)
[ 0.000000] BIOS-e820: 00000000fec94000 - 00000000fecd0000 (reserved)
[ 0.000000] BIOS-e820: 00000000fecd4000 - 0000000100000000 (reserved)
[ 0.000000] BIOS-e820: 0000000100000000 - 000000201f000000 (usable)

that 0x201f000000 is slightly above 128 GB.

The lowlevel x86 RAM init code seems to be fine:

[ 0.000000] last_pfn = 0x201f000 max_arch_pfn = 0x400000000

that 0x201f000 correctly points to slighly above 128 GB
physical.

[ 0.000000] init_memory_mapping: 0000000100000000-000000201f000000

that too shows that the lowlevel x86 platform memory init code
still sees 128 GB.

it's spread out amongst 4 nodes, 32 GB each:

[ 0.000000] Initmem setup node 0 0000000000000000-0000000820000000
[ 0.000000] NODE_DATA [000000081fffb000 - 000000081fffffff]
[ 0.000000] Initmem setup node 1 0000000820000000-0000001020000000
[ 0.000000] NODE_DATA [000000101fffb000 - 000000101fffffff]
[ 0.000000] Initmem setup node 2 0000001020000000-0000001820000000
[ 0.000000] NODE_DATA [000000181fffb000 - 000000181fffffff]
[ 0.000000] Initmem setup node 3 0000001820000000-000000201f000000
[ 0.000000] NODE_DATA [000000201effa000 - 000000201effefff]

the NORMAL zone gets set up properly:

[ 0.000000] Normal 0x00100000 -> 0x0201f000

and each node zone got 32 GB of RAM:

[ 0.000000] Normal zone: 7354368 pages, LIFO batch:31
[ 0.000000] Normal zone: 8257536 pages, LIFO batch:31
[ 0.000000] Normal zone: 8257536 pages, LIFO batch:31
[ 0.000000] Normal zone: 8253504 pages, LIFO batch:31


and it's all visible in the end to the MM:

[ 0.000000] Built 4 zonelists in Zone order, mobility grouping on. Total pages: 33021506

that's still 125 GB. (cgroup_page appears to pick up 1GB of RAM
btw.)

So where is the rest of RAM gone? How does /proc/meminfo look
like?

Thanks,

Ingo


\
 
 \ /
  Last update: 2012-02-09 09:35    [W:0.166 / U:0.332 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site