lkml.org 
[lkml]   [2007]   [May]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    Date
    From
    Subject[PATCH 09/40] mm: optimize gfp_to_rank()
    The gfp_to_rank() call in the slab allocator severely impacts performance.
    Hence reduce it to the bone, keeping only what is needed to make the reserve
    work.

    [more AIM9 results go here]

    AIM9 test 2.6.21-rc5 2.6.21-rc5-slab1
    CONFIG_SLAB_FAIR=y

    54 tcp_test 2124.48 +/- 10.85 2137.43 +/- 9.22 12.95
    55 udp_test 5204.43 +/- 45.13 5231.59 +/- 56.66 27.16
    56 fifo_test 20991.42 +/- 46.71 19675.97 +/- 56.35 1315.44
    57 stream_pipe 10024.16 +/- 119.88 9912.53 +/- 75.52 111.63
    58 dgram_pipe 9460.18 +/- 119.50 9502.75 +/- 89.06 42.57
    59 pipe_cpy 30719.81 +/- 117.01 27885.52 +/- 46.81 2834.28

    2.6.21-rc5-slab2
    CONFIG_SLAB_FAIR=y

    54 tcp_test 2124.48 +/- 10.85 2122.80 +/- 4.70 1.68
    55 udp_test 5204.43 +/- 45.13 5136.98 +/- 62.31 67.45
    56 fifo_test 20991.42 +/- 46.71 19646.81 +/- 53.61 1344.60
    57 stream_pipe 10024.16 +/- 119.88 9940.87 +/- 280.73 83.29
    58 dgram_pipe 9460.18 +/- 119.50 9432.69 +/- 250.27 27.49
    59 pipe_cpy 30719.81 +/- 117.01 27870.70 +/- 65.50 2849.10

    Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
    ---
    mm/internal.h | 33 +++++++++++++++++++++++++++++++--
    1 file changed, 31 insertions(+), 2 deletions(-)

    Index: linux-2.6-git/mm/internal.h
    ===================================================================
    --- linux-2.6-git.orig/mm/internal.h 2007-02-22 14:09:39.000000000 +0100
    +++ linux-2.6-git/mm/internal.h 2007-02-22 14:24:34.000000000 +0100
    @@ -105,9 +105,38 @@ static inline int alloc_flags_to_rank(in
    return rank;
    }

    -static inline int gfp_to_rank(gfp_t gfp_mask)
    +static __always_inline int gfp_to_rank(gfp_t gfp_mask)
    {
    - return alloc_flags_to_rank(gfp_to_alloc_flags(gfp_mask));
    + /*
    + * Although correct this full version takes a ~3% performance hit
    + * on the network test in aim9.
    + *
    + * return alloc_flags_to_rank(gfp_to_alloc_flags(gfp_mask));
    + *
    + * So we go cheat a little. We'll only focus on the correctness of
    + * rank 0.
    + */
    +
    + if (likely(!(gfp_mask & __GFP_NOMEMALLOC))) {
    + if (gfp_mask & __GFP_EMERGENCY)
    + return 0;
    + else if (!in_irq() && (current->flags & PF_MEMALLOC))
    + return 0;
    + /*
    + * We skip the TIF_MEMDIE test:
    + *
    + * if (!in_interrupt() && unlikely(test_thread_flag(TIF_MEMDIE)))
    + * return 0;
    + *
    + * this will force an alloc but since we are allowed the memory
    + * that will succeed. This will make this very rare occurence
    + * very expensive when under severe memory pressure, but it
    + * seems a valid tradeoff.
    + */
    + }
    +
    + /* Cheat by lumping everybody else in rank 1. */
    + return 1;
    }

    #endif
    --

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2007-05-04 12:53    [W:4.194 / U:0.020 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site