Messages in this thread | | | Date | Fri, 27 Jan 2006 23:30:23 +0100 | From | Eric Dumazet <> | Subject | Re: [patch 3/4] net: Percpufy frequently used variables -- proto.sockets_allocated |
| |
Andrew Morton a écrit : > Ravikiran G Thirumalai <kiran@scalex86.org> wrote: >> On Fri, Jan 27, 2006 at 09:53:53AM +0100, Eric Dumazet wrote: >>> Ravikiran G Thirumalai a écrit : >>>> Change the atomic_t sockets_allocated member of struct proto to a >>>> per-cpu counter. >>>> >>>> Signed-off-by: Pravin B. Shelar <pravins@calsoftinc.com> >>>> Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org> >>>> Signed-off-by: Shai Fultheim <shai@scalex86.org> >>>> >>> Hi Ravikiran >>> >>> If I correctly read this patch, I think there is a scalability problem. >>> >>> On a big SMP machine, read_sockets_allocated() is going to be a real killer. >>> >>> Say we have 128 Opterons CPUS in a box. >> read_sockets_allocated is being invoked when when /proc/net/protocols is read, >> which can be assumed as not frequent. >> At sk_stream_mem_schedule(), read_sockets_allocated() is invoked only >> certain conditions, under memory pressure -- on a large CPU count machine, >> you'd have large memory, and I don't think read_sockets_allocated would get >> called often. It did not atleast on our 8cpu/16G box. So this should be OK >> I think. > > That being said, the percpu_counters aren't a terribly successful concept > and probably do need a revisit due to the high inaccuracy at high CPU > counts. It might be better to do some generic version of vm_acct_memory() > instead.
There are several issues here :
alloc_percpu() current implementation is a a waste of ram. (because it uses slab allocations that have a minimum size of 32 bytes)
Currently we cannot use per_cpu(&some_object, cpu), so a generic version of vm_acct_memory() would need a rework of percpu.h and maybe this is not possible on every platform ?
#define per_cpu(var, cpu) (*RELOC_HIDE(&per_cpu__##var, __per_cpu_offset[cpu]))
-->
#define per_cpu_name(var) per_cpu__##var #define per_cpu_addr(var) &per_cpu_name(var) #define per_cpu(var, cpu) (*RELOC_HIDE(per_cpu_addr(var), __per_cpu_offset[cpu])
But this could render TLS migration difficult...
Eric - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |