Messages in this thread | | | Subject | Re: [PATCH 1/2]percpu: introduce read mostly percpu API | From | Eric Dumazet <> | Date | Wed, 20 Oct 2010 07:18:00 +0200 |
| |
Le mercredi 20 octobre 2010 à 11:07 +0800, Shaohua Li a écrit : > Add a new readmostly percpu section and api, next patch will use it. > > Signed-off-by: Shaohua Li <shaohua.li@intel.com> > ---
Could you precisely describe why grouping together read mostly percpu variables is a win ? Especially when you add in your next patch a single variable ?
> include/asm-generic/vmlinux.lds.h | 4 ++++ > include/linux/percpu-defs.h | 9 +++++++++ > 2 files changed, 13 insertions(+) > > Index: linux/include/asm-generic/vmlinux.lds.h > =================================================================== > --- linux.orig/include/asm-generic/vmlinux.lds.h 2010-10-20 09:32:52.000000000 +0800 > +++ linux/include/asm-generic/vmlinux.lds.h 2010-10-20 10:03:38.000000000 +0800 > @@ -677,6 +677,8 @@ > - LOAD_OFFSET) { \ > VMLINUX_SYMBOL(__per_cpu_start) = .; \ > *(.data..percpu..first) \ > + . = ALIGN(PAGE_SIZE); \ > + *(.data..percpu..readmostly) \ > *(.data..percpu..page_aligned) \ > *(.data..percpu) \ > *(.data..percpu..shared_aligned) \
So percpu..page_aligned is not any more aligned to a PAGE ? or we have a big hole before it ? Hmm....
Maybe you should put first data..percpu..page_aligned, then align to one cache line (L1_CACHE_BYTES), then data..percpu..readmostly, so that hole is small.
We should take care of not introducing too much holes in percpu zone. Maybe using a new subzone ".data..percpu..small_objects" to put small objects in it.
nm -v vmlinux | grep -10 sockets_in_use # select one random part
0000000000011e0c d cpu_min_freq 0000000000011e10 d cpu_cur_freq 0000000000011e14 d cpu_set_freq 0000000000011e18 d cpu_is_managed 0000000000011e20 d od_cpu_dbs_info 0000000000011fa0 d cs_cpu_dbs_info 00000000000120f0 d cpufreq_show_table 0000000000012100 D cpuidle_devices 0000000000012120 d ladder_devices 0000000000012200 d menu_devices 00000000000122c0 d sockets_in_use # object_size = 4 , hole = 28bytes 00000000000122e0 d prot_inuse 00000000000123e0 D nf_conntrack_untracked 0000000000012560 d rt_cache_stat 00000000000125a0 d ipv4_cookie_scratch 0000000000012740 D init_tss 0000000000014a00 D irq_stat 0000000000014a40 D cpu_info 0000000000014b00 d hv_clock 0000000000014b40 D cpu_tlbstate 0000000000014b80 d runqueues
We can see many holes because of 2^5 alignments of individual .o .data..percpu sections.
find . -name "*.o"|xargs objdump -h|grep percpu
Linker promotes a section alignment from natural alignment to 2^5 as soon as the size reaches 2^5
For example in net/ipv4/route.o, we have a per_cpu structure (rt_cache_stat), that is an array of 16 integers. The natural alignement should be 4 (alignof(int)), but we get :
# objdump -h net/ipv4/route.o|grep percpu 19 .data..percpu 00000040 0000000000000000 0000000000000000 00007a80 2**5
For a section replicated N times, this really is a concern.
Thanks !
-- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |