lkml.org 
[lkml]   [2007]   [Mar]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [RFC] SLAB : NUMA cache_free_alien() very expensive because of virt_to_slab(objp); nodeid = slabp->nodeid;
Christoph Lameter a écrit :
> On Tue, 20 Mar 2007, Eric Dumazet wrote:
>
>> I understand we want to do special things (fallback and such tricks) at
>> allocation time, but I believe that we can just trust the real nid of memory
>> at free time.
>
> Sorry no. The node at allocation time determines which node specific
> structure tracks the slab. If we fall back then the node is allocated
> from one node but entered in the node structure of another. Thus you
> cannot free the slab without knowing the node at allocation time.

I think you dont understand my point.

When we enter kmem_cache_free(), we are not freeing a slab, but an object,
knowing a pointer to this object.

The fast path is to put the pointer, into the cpu array cache. This object
might be given back some cycles later, because of a kmem_cache_alloc() : No
need to access the two cache lines (struct page, struct slab)

This fast path could be entered checking the node of the page, which is
faster, but eventually different from the virt_to_slab(obj)->nodeid. Do we
care ? Definitly not : Node is guaranted to be correct.

Then, if we must flush the cpu array cache because it is full, we *may* access
the slabs of the objects we are flushing, and then check the
virt_to_slab(obj)->nodeid to be able to do the correct thing.

Fortunatly, flushing cache array is not a frequent event, and the cost of
access to cache lines (truct page, struct slab) can be amortized because
several 'transfered or freed' objects might share them at this time.


Actually I had to disable NUMA on my platforms because it is just overkill and
slower. Maybe its something OK for very big machines, and not dual nodes
Opterons ? Let me know so that I dont waste your time (and mine)


Thank you
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2007-03-21 07:31    [W:0.914 / U:0.220 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site