lkml.org 
[lkml]   [2009]   [Mar]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
Subject[patch 3/3] slub: sort parital list when thrashing
Caches that are cpu slab thrashing will scan their entire partial list
until a slab is found that will satisfy at least the requisite number of
allocations so that it will not be considered thrashing (as defined by
/sys/kernel/slab/cache/slab_thrash_ratio).

The partial list can be extremely long and its scanning requires that
list_lock is held for that particular node. This can be inefficient if
slabs at the head of the list are not appropriate cpu slab replacements.

When an object is freed, the number of free objects for its slab is
calculated if the cpu cache is currently thrashing. If it can satisfy
the requisite number of allocations so that the slab thrash ratio is
exceeded, it is moved to the head of the partial list. This minimizes
the time spent holding list_lock and can help cacheline optimizations
for recently freed objects.

Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: David Rientjes <rientjes@google.com>
---
mm/slub.c | 16 ++++++++++++++++
1 files changed, 16 insertions(+), 0 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1264,6 +1264,13 @@ static void remove_partial(struct kmem_cache *s, struct page *page)
spin_unlock(&n->list_lock);
}

+static void move_partial_to_head(struct kmem_cache_node *n, struct page *page)
+{
+ spin_lock(&n->list_lock);
+ list_move(&page->lru, &n->partial);
+ spin_unlock(&n->list_lock);
+}
+
/*
* Remove from the partial list.
*
@@ -1726,6 +1733,15 @@ checks_ok:
if (unlikely(!prior)) {
add_partial(get_node(s, page_to_nid(page)), page, 1);
stat(c, FREE_ADD_PARTIAL);
+ } else if (c->slowpath_allocs >= SLAB_THRASHING_THRESHOLD) {
+ /*
+ * If the cache is actively slab thrashing, it's necessary to
+ * move partial slabs to the head of the list so there isn't
+ * excessive partial list scanning while holding list_lock.
+ */
+ if (!skip_partial(s, page))
+ move_partial_to_head(get_node(s, page_to_nid(page)),
+ page);
}

out_unlock:

\
 
 \ /
  Last update: 2009-03-26 10:47    [W:0.026 / U:0.276 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site