Messages in this thread | | | Date | Thu, 08 Apr 2004 22:32:32 +0900 (JST) | Subject | Re: Migrate pages from a ccNUMA node to another - patch | From | Hirokazu Takahashi <> |
| |
Hello,
> > I guess aruguments src_node, mm and pte would be redundant since > > they can be looked up from old_p with the reverse mapping scheme. > > In my version 0.2, I can do with only the following arguments: > * node: Destination NUMA node > * mm: -> victim "mm_struct" > * pte: -> PTE of the page to be moved > (If I have "mm" at hand, why not to use it ? Why not to avoid fetching the r-map > page struct ?) > > > >Notes: "pte" can be NULL if I do not know it apriori > > > I cannot release "mm->page_table_lock" otherwise I have to re-scan the "mm->pgd". > > > > Re-schan plicy would be much better since migrating pages is heavy work. > > I don't think that holding mm->page_table_lock for long time would be > > good idea. > > Re-scanning is "cache killer", at least on IA64 with huge user memory size. > I have more than 512 Mbytes user memory and its PTEs do not fit into the L2 cache. > > In my current design, I have the outer loops: PGD, PMD and PTE walking; and once > I find a valid PTE, I check it against the list of max. 2048 physical addresses as > the inner loop. > I reversed them: walking through the list of max. 2048 physical addresses as outer > loop and the PGD - PMD - PTE scans as inner loops resulted in 4 to 5 times slower > migration.
I've been thinking about it.
I guess our page remap patches would be overkill for your purpose. Point of our patches is that: 1. Blocks new access to a specified page. 2. Waits for the page going into quiescent state. 3. Copies data from the page to a new page and exchanges them.
In my understanding you want to handle only anonymous pages which don't have backing store yet. This means that you only need step 3.
> > How do you think about following algorism: > > 1. get mm->page_table_lock > > 2. chose some pages. > > 3. release mm->page_table_lock > > 4. call remap_onepage() against each page. > > 5. goto step1 if there remain pages to be migrated. > > I want to move the most frequently used pages - at least with the HW assisted > hot page detection. > I take "mm->page_table_lock", I nuke the PTE. We've got a good chance that the CPU > using the page observes a page fault almost immediately. It enters the page fault > handler and gets blocked by "mm->page_table_lock". If I released the lock, the CPU > could continue and realize that there is nothing to do, the page fault has already > been repaired. In the mean time, it is me who wait for "mm->page_table_lock".
If you use the HW assisted hot page detection, just notify our remapping functions of hot pages directly. Everything would be handled well and pagefault handler would be blocked by PG_lock bit.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |