lkml.org 
[lkml]   [2003]   [Jan]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: kexec reboot code buffer
From
Date
Dave Hansen <haveblue@us.ibm.com> writes:

> On my system, it appears to lock up in:
> kimage_alloc_reboot_code_pages()
> after the kexec -l.

O.k. It should come out of it eventually from what I have
seen described, the current algorithm is definitely inefficient on
your machine.

> I put a little printk in the loop:
> list_for_each_safe(pos, next, &extra_pages) {
> struct page *page;
> int i;
> if( (listcount++%1000000) == 0 )
> printk("listcount:%d\n", listcount);
> page = list_entry(pos, struct page, list);
> for(i = 0; i < count; i++) {
> ClearPageReserved(pages +i);
> }
> list_del(&extra_pages);
> __free_pages(page, order);
> }
>
> I stopped it when it hit 1.2 billion:
> kimage_alloc_reboot_code_pages(): listcount:1213000001
>
> First of all, this is a 16-way, 4-node NUMA-Q with 32GB of RAM.
> If the alloc_pages(GFP_HIGHUSER, order) doesn't happen on node 0,
> CPU[0-3], it will be guaranteed to get physical addresses >8GB, until
> that node is out of memory, when it will start falling other to other
> nodes' highmem.

Thanks I was afraid this might be a problem, but I was not certain
how much a practical problem it would be. On a NUMA machine on x86
with > 4GB of memory the fact that this hangs shows it is very
definitely a problem.

> So, my real question is why you bother to allocate from HIGHMEM at all,
> when you know that you probably won't be getting what you want?

Actually I did not know. The common case is non-NUMA with PAE not
enabled. In which case the odds are fairly high I will get what I
want. And being able to allocate from 3GB instead of just 1GB is
much more polite. The question then is how do I specify the zones
properly.

Additionally I did not start using high memory until recently when, I
rewrote my generic code. With the rewrite it is more obvious
what I am trying to accomplish but it obviously works much less
well in the NUMA-Q corner case.

Looking at this code a little more there is also another related
bug. I use TASK_SIZE to figure out how large an address I can
allocate but on 64bit architectures with a 32bit subset TASK_SIZE
can be a variable instead of a constant. For the moment I am
going to just ignore that issue as sys_kexec_load looks
like one system call that it does not make sense to emulate. The
expected behavior on a 64bit vs. a 32bit architecture are quite
different.

> What you want is RAM with physical addresses <3GB, right?

In this case, and then later I want to allocate from physical
addresses < 4GB. The rest of the allocations will suffer
from the same problem on the NUMA-Q.

The problem is that I have not figured out how to tell the memory
allocator just what I need, and now it has been confirmed that
with two many false positives my code takes forever. My goal has
always been to not make kexec an undue burden on the rest of the
kernel because I am not the common case.

Allocating DMA able memory that you can use for pci devices
has the same sort of issues. And so far it appears there the
solution is not to allocate from memory outside of the kernels virtual
address space. As that case is improved I may be able to ride
it's coat tails.

I guess what I want to do then is add to asm-i386/kexec.h
#ifndef HIGHMEM64G
#define GFP_KEXEC GFP_HIGHUSER
#else
#define GFP_KEXEC GFP_KERNEL
#endif

And then in kernel/kexec.c
s/GFP_HIGHUSER/GFP_KEXEC/g

That should provide a usable mechanism to control this kind of thing,
it is not perfect, but at least I will be guaranteed to get back
memory that I can use.

I wonder if it is worth it to setup a special zone and zone list for
use with kexec?

I guess I would make the standard zones something like:
/*
* ZONE_DMA < 16 MB ISA DMA capable memory
* ZONE_NORMAL 16-896 MB direct mapped by the kernel
* ZONE_PHYSMEM 896-4096 MB memory that is accessible with the MMU disabled.
* ZONE_HIGHMEM > 4096MB only page cache and user processes
*/


Or something to that effect, so I could separate out memory
below 4GB and memory above 4GB. For the reboot code buffer it really
does not matter, as that is just one chunk of physically continuous
memory. For the normal allocation being able to get memory anywhere
between 0-4GB on a 32bit platform is something I don't want to give
up.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:32    [W:0.064 / U:0.476 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site