Messages in this thread | | | Date | Wed, 30 Jan 2013 11:00:10 +0800 | From | Tang Chen <> | Subject | Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory |
| |
On 01/30/2013 10:48 AM, Simon Jeons wrote: > On Wed, 2013-01-30 at 10:32 +0800, Tang Chen wrote: >> On 01/29/2013 08:52 PM, Simon Jeons wrote: >>> Hi Tang, >>> >>> On Wed, 2013-01-09 at 17:32 +0800, Tang Chen wrote: >>>> Here is the physical memory hot-remove patch-set based on 3.8rc-2. >> >> Hi Simon, >> >> I'll summarize all the info and answer you later. :) >> >> Thanks for asking. :) > > Thanks Tang, IIRC, there's qemu feature support memory hot-add/remove > emulation if we don't have machine which supports memory hot-add/remove > to test. Is that qemu feature merged? Otherwise where can I get that > patchset?
Hi Simon,
There are patches to support hot-add/remove in qemu, but they are not merged yet. You can get the latest patches here: http://lists.nongnu.org/archive/html/qemu-devel/2012-12/msg02693.html
BTY, it is unstable and full of problems, and you need to compile your own seabios too.
Thanks. :)
> >> >>> >>> Some questions ask you, not has relationship with this patchset, but is >>> memory hotplug stuff. >>> >>> 1. In function node_states_check_changes_online: >>> >>> comments: >>> * If we don't have HIGHMEM nor movable node, >>> * node_states[N_NORMAL_MEMORY] contains nodes which have zones of >>> * 0...ZONE_MOVABLE, set zone_last to ZONE_MOVABLE. >>> >>> How to understand it? Why we don't have HIGHMEM nor movable node and >>> node_staes[N_NORMAL_MEMORY] contains 0...ZONE_MOVABLE, IIUC, >>> N_NORMAL_MEMORY only means the node has regular memory. >>> >>> * If we don't have movable node, node_states[N_NORMAL_MEMORY] >>> * contains nodes which have zones of 0...ZONE_MOVABLE, >>> * set zone_last to ZONE_MOVABLE. >>> >>> How to understand? >>> >>> 2. In function move_pfn_range_left, why end<= z2->zone_start_pfn is not >>> correct? The comments said that must include/overlap, why? >>> >>> 3. In function online_pages, the normal case(w/o online_kenrel, >>> online_movable), why not check if the new zone is overlap with adjacent >>> zones? >>> >>> 4. Could you summarize the difference implementation between hot-add and >>> logic-add, hot-remove and logic-remove? >>> >>> >>>> >>>> This patch-set aims to implement physical memory hot-removing. >>>> >>>> The patches can free/remove the following things: >>>> >>>> - /sys/firmware/memmap/X/{end, start, type} : [PATCH 4/15] >>>> - memmap of sparse-vmemmap : [PATCH 6,7,8,10/15] >>>> - page table of removed memory : [RFC PATCH 7,8,10/15] >>>> - node and related sysfs files : [RFC PATCH 13-15/15] >>>> >>>> >>>> Existing problem: >>>> If CONFIG_MEMCG is selected, we will allocate memory to store page cgroup >>>> when we online pages. >>>> >>>> For example: there is a memory device on node 1. The address range >>>> is [1G, 1.5G). You will find 4 new directories memory8, memory9, memory10, >>>> and memory11 under the directory /sys/devices/system/memory/. >>>> >>>> If CONFIG_MEMCG is selected, when we online memory8, the memory stored page >>>> cgroup is not provided by this memory device. But when we online memory9, the >>>> memory stored page cgroup may be provided by memory8. So we can't offline >>>> memory8 now. We should offline the memory in the reversed order. >>>> >>>> When the memory device is hotremoved, we will auto offline memory provided >>>> by this memory device. But we don't know which memory is onlined first, so >>>> offlining memory may fail. >>>> >>>> In patch1, we provide a solution which is not good enough: >>>> Iterate twice to offline the memory. >>>> 1st iterate: offline every non primary memory block. >>>> 2nd iterate: offline primary (i.e. first added) memory block. >>>> >>>> And a new idea from Wen Congyang<wency@cn.fujitsu.com> is: >>>> allocate the memory from the memory block they are describing. >>>> >>>> But we are not sure if it is OK to do so because there is not existing API >>>> to do so, and we need to move page_cgroup memory allocation from MEM_GOING_ONLINE >>>> to MEM_ONLINE. And also, it may interfere the hugepage. >>>> >>>> >>>> >>>> How to test this patchset? >>>> 1. apply this patchset and build the kernel. MEMORY_HOTPLUG, MEMORY_HOTREMOVE, >>>> ACPI_HOTPLUG_MEMORY must be selected. >>>> 2. load the module acpi_memhotplug >>>> 3. hotplug the memory device(it depends on your hardware) >>>> You will see the memory device under the directory /sys/bus/acpi/devices/. >>>> Its name is PNP0C80:XX. >>>> 4. online/offline pages provided by this memory device >>>> You can write online/offline to /sys/devices/system/memory/memoryX/state to >>>> online/offline pages provided by this memory device >>>> 5. hotremove the memory device >>>> You can hotremove the memory device by the hardware, or writing 1 to >>>> /sys/bus/acpi/devices/PNP0C80:XX/eject. >>> >>> Is there a similar knode to hot-add the memory device? >>> >>>> >>>> >>>> Note: if the memory provided by the memory device is used by the kernel, it >>>> can't be offlined. It is not a bug. >>>> >>>> >>>> Changelogs from v5 to v6: >>>> Patch3: Add some more comments to explain memory hot-remove. >>>> Patch4: Remove bootmem member in struct firmware_map_entry. >>>> Patch6: Repeatedly register bootmem pages when using hugepage. >>>> Patch8: Repeatedly free bootmem pages when using hugepage. >>>> Patch14: Don't free pgdat when offlining a node, just reset it to 0. >>>> Patch15: New patch, pgdat is not freed in patch14, so don't allocate a new >>>> one when online a node. >>>> >>>> Changelogs from v4 to v5: >>>> Patch7: new patch, move pgdat_resize_lock into sparse_remove_one_section() to >>>> avoid disabling irq because we need flush tlb when free pagetables. >>>> Patch8: new patch, pick up some common APIs that are used to free direct mapping >>>> and vmemmap pagetables. >>>> Patch9: free direct mapping pagetables on x86_64 arch. >>>> Patch10: free vmemmap pagetables. >>>> Patch11: since freeing memmap with vmemmap has been implemented, the config >>>> macro CONFIG_SPARSEMEM_VMEMMAP when defining __remove_section() is >>>> no longer needed. >>>> Patch13: no need to modify acpi_memory_disable_device() since it was removed, >>>> and add nid parameter when calling remove_memory(). >>>> >>>> Changelogs from v3 to v4: >>>> Patch7: remove unused codes. >>>> Patch8: fix nr_pages that is passed to free_map_bootmem() >>>> >>>> Changelogs from v2 to v3: >>>> Patch9: call sync_global_pgds() if pgd is changed >>>> Patch10: fix a problem int the patch >>>> >>>> Changelogs from v1 to v2: >>>> Patch1: new patch, offline memory twice. 1st iterate: offline every non primary >>>> memory block. 2nd iterate: offline primary (i.e. first added) memory >>>> block. >>>> >>>> Patch3: new patch, no logical change, just remove reduntant codes. >>>> >>>> Patch9: merge the patch from wujianguo into this patch. flush tlb on all cpu >>>> after the pagetable is changed. >>>> >>>> Patch12: new patch, free node_data when a node is offlined. >>>> >>>> >>>> Tang Chen (6): >>>> memory-hotplug: move pgdat_resize_lock into >>>> sparse_remove_one_section() >>>> memory-hotplug: remove page table of x86_64 architecture >>>> memory-hotplug: remove memmap of sparse-vmemmap >>>> memory-hotplug: Integrated __remove_section() of >>>> CONFIG_SPARSEMEM_VMEMMAP. >>>> memory-hotplug: remove sysfs file of node >>>> memory-hotplug: Do not allocate pdgat if it was not freed when >>>> offline. >>>> >>>> Wen Congyang (5): >>>> memory-hotplug: try to offline the memory twice to avoid dependence >>>> memory-hotplug: remove redundant codes >>>> memory-hotplug: introduce new function arch_remove_memory() for >>>> removing page table depends on architecture >>>> memory-hotplug: Common APIs to support page tables hot-remove >>>> memory-hotplug: free node_data when a node is offlined >>>> >>>> Yasuaki Ishimatsu (4): >>>> memory-hotplug: check whether all memory blocks are offlined or not >>>> when removing memory >>>> memory-hotplug: remove /sys/firmware/memmap/X sysfs >>>> memory-hotplug: implement register_page_bootmem_info_section of >>>> sparse-vmemmap >>>> memory-hotplug: memory_hotplug: clear zone when removing the memory >>>> >>>> arch/arm64/mm/mmu.c | 3 + >>>> arch/ia64/mm/discontig.c | 10 + >>>> arch/ia64/mm/init.c | 18 ++ >>>> arch/powerpc/mm/init_64.c | 10 + >>>> arch/powerpc/mm/mem.c | 12 + >>>> arch/s390/mm/init.c | 12 + >>>> arch/s390/mm/vmem.c | 10 + >>>> arch/sh/mm/init.c | 17 ++ >>>> arch/sparc/mm/init_64.c | 10 + >>>> arch/tile/mm/init.c | 8 + >>>> arch/x86/include/asm/pgtable_types.h | 1 + >>>> arch/x86/mm/init_32.c | 12 + >>>> arch/x86/mm/init_64.c | 390 +++++++++++++++++++++++++++++ >>>> arch/x86/mm/pageattr.c | 47 ++-- >>>> drivers/acpi/acpi_memhotplug.c | 8 +- >>>> drivers/base/memory.c | 6 + >>>> drivers/firmware/memmap.c | 96 +++++++- >>>> include/linux/bootmem.h | 1 + >>>> include/linux/firmware-map.h | 6 + >>>> include/linux/memory_hotplug.h | 15 +- >>>> include/linux/mm.h | 4 +- >>>> mm/memory_hotplug.c | 459 +++++++++++++++++++++++++++++++--- >>>> mm/sparse.c | 8 +- >>>> 23 files changed, 1094 insertions(+), 69 deletions(-) >>>> >>>> -- >>>> To unsubscribe, send a message with 'unsubscribe linux-mm' in >>>> the body to majordomo@kvack.org. For more info on Linux MM, >>>> see: http://www.linux-mm.org/ . >>>> Don't email:<a href=mailto:"dont@kvack.org"> email@kvack.org</a> >>> >>> >>> > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-acpi" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >
| |