Messages in this thread | | | Date | Sun, 14 Sep 2003 02:28:10 +0530 | From | Dipankar Sarma <> | Subject | Re: [patch] Make slab allocator work with SLAB_MUST_HWCACHE_ALIGN |
| |
On Sat, Sep 13, 2003 at 10:06:08PM +0200, Manfred Spraul wrote: > Ravikiran G Thirumalai wrote: > >I wouldn't be using the slab at all because using slabs would mean using > >NR_CPUs pointers and one extra dereference which is bad as we had found out > >earlier. But I guess slab will have to do node local allocations for > >other applications. > > > > > Interesting. Slab internally uses lots of large per-cpu arrays. > Alltogether something like around 40 kB/cpu. Right now implemented with > NR_CPUs pointers. In the long run I'll try to switch to your allocator. > > But back to the patch that started this thread: Do you still need the > ability to set an explicit alignment for slab allocations? If yes, then > I'd polish my patch, double check all kmem_cache_create callers and then > send the patch to akpm. Otherwise I'd wait - the patch is not a bugfix.
This is the problem - the current dynamic per-cpu allocator (alloc_percpu()) is broken. I uses kmalloc() to allocate each CPU's data, but kmalloc() doesn't gurantee cache line alignment (only SLAB_HWCACHE_ALIGN). This may result in some per-CPU statistics bouncing between CPUs specially on the ones with large L1 cache lines.
We have a number of options -
1. Force kmalloc() to strictly align on cache line boundary, but will result in wastage of space elsewhere (with your strict align patch) but alloc_percpu() will never result in cache line sharing. 2. Make alloc_percpu() use its own caches for various sizes with your strictly align patch. The rest of the kernel is not affected. 3. Let alloc_percpu() use its own allocator which supports NUMA and does not use an offset table.
#2 and #3 has less impact in the kernel and we should consider those, IMO.
Thanks Dipankar - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |