[lkml]   [2012]   [Feb]   [13]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    Patch in this message
    Subject[PATCH 6/6 v4] memcg: fix performance of mem_cgroup_begin_update_page_stat()
    From 3377fd7b6e23a5d2a368c078eae27e2b49c4f4aa Mon Sep 17 00:00:00 2001
    From: KAMEZAWA Hiroyuki <>
    Date: Mon, 6 Feb 2012 12:14:47 +0900
    Subject: [PATCH 6/6] memcg: fix performance of mem_cgroup_begin_update_page_stat()

    mem_cgroup_begin_update_page_stat() should be very fast because
    it's called very frequently. Now, it needs to look up page_cgroup
    and its memcg....this is slow.

    This patch adds a global variable to check "a memcg is moving or not".
    By this, the caller doesn't need to visit page_cgroup and memcg.

    Here is a test result. A test program makes page faults onto a file,
    MAP_SHARED and makes each page's page_mapcount(page) > 1, and free
    the range by madvise() and page fault again. This program causes
    26214400 times of page fault onto a file(size was 1G.) and shows
    shows the cost of mem_cgroup_begin_update_page_stat().

    Before this patch for mem_cgroup_begin_update_page_stat()
    [kamezawa@bluextal test]$ time ./mmap 1G

    real 0m21.765s
    user 0m5.999s
    sys 0m15.434s

    27.46% mmap mmap [.] reader
    21.15% mmap [kernel.kallsyms] [k] page_fault
    9.17% mmap [kernel.kallsyms] [k] filemap_fault
    2.96% mmap [kernel.kallsyms] [k] __do_fault
    2.83% mmap [kernel.kallsyms] [k] __mem_cgroup_begin_update_page_stat

    After this patch
    [root@bluextal test]# time ./mmap 1G

    real 0m21.373s
    user 0m6.113s
    sys 0m15.016s

    In usual path, calls to __mem_cgroup_begin_update_page_stat() goes away.

    Note: we may be able to remove this optimization in future if
    we can get pointer to memcg directly from struct page.

    Signed-off-by: KAMEZAWA Hiroyuki <>
    include/linux/memcontrol.h | 5 +++--
    mm/memcontrol.c | 7 ++++++-
    2 files changed, 9 insertions(+), 3 deletions(-)

    diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
    index 3f3ef33..3df9979 100644
    --- a/include/linux/memcontrol.h
    +++ b/include/linux/memcontrol.h
    @@ -143,7 +143,7 @@ static inline bool mem_cgroup_disabled(void)

    void __mem_cgroup_begin_update_page_stat(struct page *page,
    bool *lock, unsigned long *flags);
    +extern atomic_t memcg_moving;
    static inline void mem_cgroup_begin_update_page_stat(struct page *page,
    bool *lock, unsigned long *flags)
    @@ -151,7 +151,8 @@ static inline void mem_cgroup_begin_update_page_stat(struct page *page,
    *lock = false;
    - return __mem_cgroup_begin_update_page_stat(page, lock, flags);
    + if (atomic_read(&memcg_moving))
    + return __mem_cgroup_begin_update_page_stat(page, lock, flags);

    void __mem_cgroup_end_update_page_stat(struct page *page, bool *lock,
    diff --git a/mm/memcontrol.c b/mm/memcontrol.c
    index ec55880..fd02fab 100644
    --- a/mm/memcontrol.c
    +++ b/mm/memcontrol.c
    @@ -1273,8 +1273,11 @@ int mem_cgroup_swappiness(struct mem_cgroup *memcg)
    * start move here.

    +/* for quick checking without looking up memcg */
    +atomic_t memcg_moving __read_mostly;
    static void mem_cgroup_start_move(struct mem_cgroup *memcg)
    + atomic_inc(&memcg_moving);
    @@ -1285,8 +1288,10 @@ static void mem_cgroup_end_move(struct mem_cgroup *memcg)
    * Now, mem_cgroup_clear_mc() may call this function with NULL.
    * We check NULL in callee rather than caller.
    - if (memcg)
    + if (memcg) {
    + atomic_dec(&memcg_moving);
    + }


     \ /
      Last update: 2012-02-14 04:21    [W:0.025 / U:140.268 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site