lkml.org 
[lkml]   [2004]   [Oct]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [RFC] memory defragmentation to satisfy high order allocations
On Fri, Oct 01, 2004 at 02:57:03PM -0700, Dave Hansen wrote:
> On Fri, 2004-10-01 at 12:04, Marcelo Tosatti wrote:
> > On Fri, Oct 01, 2004 at 01:11:47PM -0700, Andrew Morton wrote:
> > > Presumably this duplicates some of the memory hot-remove patches.
> >
> > As far as I have researched, the memory moving/remapping code
> > on the hot remove patches dont work correctly. Please correct me.
>
> I definitely see some commonality, but Marcelo's approach has handling
> for the different kinds of pages broken out much more nicely. Can't
> tell yet if this produces extra code, or is just plain better.
>
> We worked pretty hard to try and copy as little code as possible. Was
> there any reason that there was so much stuff copied out of rmap.c?
> Just for proof-of-concept?

Just proof of concept really, to have an equivalent of "try_to_unmap()" -
which you call from the migrate page code.

Just that "try_to_remap_{file,anon}" do the pte clearing + remapping in
one function.
> Here's one of the recent patch sets that we're working on:
>
> http://sprucegoose.sr71.net/patches/2.6.9-rc2-mm4-mhp-test2/
>
> In that directory, the K* patches hijack some of the swap code (but
> require memory pressure to work last time I tried), and the p000*
> patches (by Hirokazu Takahashi) actively migrate pages around. Both
> approaches work, but the K* one is smaller and less intrusive, while the
> p000* one is much more complete. They may end up being able to coexist
> in the end.

The page migration code (p000*) looks nice - quite complete indeed (nice error
handling, etc) but somewhat specific to the migration procedure, which is more
critical (cannot fail so easily as) then the remapping for high-order allocations.

For example this in migrate_page_common


+ switch (ret) {
+ case 0:
+ case -ENOENT:
+ copy_highpage(newpage, page);
+ return ret;
+ case -EBUSY:
+ return ret;
+ case -EAGAIN:
+ writeback_and_free_buffers(page);
+ unlock_page(page);
+ msleep(10);
+ timeout -= 10;
+ lock_page(page);
+ continue;
Which retries undefinately to migrate the page

For the "defragmentation" operation we want to do an "easy" try - ie if we
can't remap giveup.

I feel we should try to "untie" the code which checks for remapping availability /
does the remapping from the page migration - so to be able to share the most
code between it and other users of the same functionality.

Curiosity: How did you guys test the migration operation? Several threads on
several processors operating on the memory, etc?

> I don't work for Fujitsu :) Please take a look at the patches in the
> above directory and see what you think. I'm sure you have some very
> good stuff in your patch, but I need to take a closer look.
>
> I'm just about to head out of town for the weekend, but I'll take a much
> more detailed look on Monday.

Cool. I'll take a closer look at the relevant parts of memory hotplug patches
this weekend, hopefully. See if I can help with testing of these patches too.

Andrew, what are your thoughts wrt merging this to mainline?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 14:06    [from the cache]
©2003-2011 Jasper Spaans