lkml.org 
[lkml]   [2012]   [Mar]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH -V2 0/9] memcg: add HugeTLB resource tracking
On Thu,  1 Mar 2012 14:46:11 +0530
"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> wrote:

> Hi,
>
> This patchset implements a memory controller extension to control
> HugeTLB allocations. It is similar to the existing hugetlb quota
> support in that, the limit is enforced at mmap(2) time and not at
> fault time. HugeTLB's quota mechanism limits the number of huge pages
> that can allocated per superblock.
>

Thank you, I think memcg-extension is better than hugetlbfs cgroup.


> For shared mappings we track the regions mapped by a task along with the
> memcg. We keep the memory controller charged even after the task
> that did mmap(2) exits. Uncharge happens during truncate. For Private
> mappings we charge and uncharge from the current task cgroup.
>

What "current" means here ? current task's cgroup ?


> A sample strace output for an application doing malloc with hugectl is given
> below. libhugetlbfs will fall back to normal pagesize if the HugeTLB mmap fails.
>
> open("/mnt/libhugetlbfs.tmp.uhLMgy", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
> unlink("/mnt/libhugetlbfs.tmp.uhLMgy") = 0
>
> .........
>
> mmap(0x20000000000, 50331648, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) = -1 ENOMEM (Cannot allocate memory)
> write(2, "libhugetlbfs", 12libhugetlbfs) = 12
> write(2, ": WARNING: New heap segment map" ....
> mmap(NULL, 42008576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xfff946c0000
> ....
>
>
> Goals:
>
> 1) We want to keep the semantic closer to hugelb quota support. ie, we want
> to extend quota semantics to a group of tasks. Currently hugetlb quota
> mechanism allows one to control number of hugetlb pages allocated per
> hugetlbfs superblock.
>
> 2) Applications using hugetlbfs always fallback to normal page size allocation when they
> fail to allocate huge pages. libhugetlbfs internally handles this for malloc(3). We
> want to retain this behaviour when we enforce the controller limit. ie, when huge page
> allocation fails due to controller limit, applications should fallback to
> allocation using normal page size. The above implies that we need to enforce
> limit at mmap(2).
>

Hm, ok.

> 3) HugeTLBfs doesn't support page reclaim. It also doesn't support write(2). Applications
> use hugetlbfs via mmap(2) interface. Important point to note here is hugetlbfs
> extends file size in mmap.
>
> With shared mappings, the file size gets extended in mmap and file will remain in hugetlbfs
> consuming huge pages until it is truncated. We want to make sure we keep the controller
> charged until the file is truncated. This implies, that the controller will be charged
> even after the task that did mmap exit.
>

O.K. hugetlbfs is charged until the file is removed.
Then, next question will be 'can we destory cgroup....'

> Implementation details:
>
> In order to achieve the above goals we need to track the cgroup information
> along with mmap range in a charge list in inode for shared mapping and in
> vm_area_struct for private mapping. We won't be using page to track cgroup
> information because with the above goals we are not really tracking the pages used.
>
> Since we track cgroup in charge list, if we want to remove the cgroup, we need to update
> the charge list to point to the parent cgroup. Currently we take the easy route
> and prevent a cgroup removal if it's non reclaim resource usage is non zero.
>

As Andrew pointed out, there are some ongoing works about page-range tracking.
Please check.

Thanks,
-Kame

> Changes from V1:
> * Changed the implementation as a memcg extension. We still use
> the same logic to track the cgroup and range.
>
> Changes from RFC post:
> * Added support for HugeTLB cgroup hierarchy
> * Added support for task migration
> * Added documentation patch
> * Other bug fixes
>
> -aneesh
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>



\
 
 \ /
  Last update: 2012-03-02 06:53    [W:0.143 / U:1.272 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site