lkml.org 
[lkml]   [2008]   [Mar]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: hackbench regression since 2.6.25-rc
From
Date
On Fri, 2008-03-14 at 11:30 +0800, Zhang, Yanmin wrote:
> On Fri, 2008-03-14 at 11:04 +0800, Zhang, Yanmin wrote:
> > On Thu, 2008-03-13 at 17:16 -0700, Christoph Lameter wrote:
> > > Could you recompile the kernel with slub performance statistics and post
> > > the output of
> > >
> > > slabinfo -AD
> > Before testing with kernel 2.6.25-rc5:
> > Name Objects Alloc Free %Fast
> > vm_area_struct 2795 135185 132587 93 29
> > :0004096 25 119045 119043 99 98
> > :0000064 12257 119671 107742 98 50
> > :0000192 3312 78563 75370 92 21
> > :0000128 4648 48143 43738 97 53
> > dentry 15217 46675 31527 95 72
> > :0000080 12784 33674 21206 99 97
> > :0000016 4367 25871 23705 99 78
> > :0000096 3001 22591 20084 99 92
> > buffer_head 5536 18147 12884 97 42
> > anon_vma 1729 14948 14130 99 73
> >
> >
> > After testing:
> > Name Objects Alloc Free %Fast
> > :0000192 3428 80093958 80090708 92 8
> > :0000512 374 80016030 80015715 68 7
> > vm_area_struct 2875 224524 221868 94 20
> > :0000064 12408 134273 122227 98 47
> > :0004096 24 127397 127395 99 98
> > :0000128 4596 57837 53432 97 48
> > dentry 15659 51402 35824 95 64
> > :0000016 4584 29327 27161 99 76
> > :0000080 12784 33674 21206 99 97
> > :0000096 2998 26264 23757 99 93
> >
> >
> > So block 192 and 512's and very active and their fast free percentage is low.
> On my 8-core stoakley, there is no such regression. Below data is after testing.
>
> [root@lkp-st02-x8664 ~]# slabinfo -AD
> Name Objects Alloc Free %Fast
> :0000192 3170 80055388 80052280 92 1
> :0000512 316 80012750 80012466 69 1
> vm_area_struct 2642 194700 192193 94 16
> :0000064 3846 74468 70820 97 53
> :0004096 15 69014 69012 98 97
> :0000128 1447 32920 31541 91 8
> dentry 13485 33060 19652 92 42
> :0000080 10639 23377 12953 98 98
> :0000096 1662 16496 15036 99 94
> :0000832 232 14422 14203 85 10
> :0000016 2733 15102 13372 99 14
>
> So the block 192 and 512's fast free percentage is even smaller than the ones on tigerton.
>
> Oprofile data on stoakley:
>
> CPU: Core 2, speed 2660 MHz (estimated)
> Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (Unhalted core cycles) count 100000
> samples % app name symbol name
> 2897265 25.7603 linux-2.6.25-rc5 __slab_alloc
> 2689900 23.9166 linux-2.6.25-rc5 add_partial
> 629355 5.5957 linux-2.6.25-rc5 copy_user_generic_string
> 552309 4.9107 linux-2.6.25-rc5 __slab_free
> 514792 4.5771 linux-2.6.25-rc5 sock_alloc_send_skb
> 500879 4.4534 linux-2.6.25-rc5 unix_stream_recvmsg
> 274798 2.4433 linux-2.6.25-rc5 __kmalloc_track_caller
> 230283 2.0475 linux-2.6.25-rc5 kfree
> 222286 1.9764 linux-2.6.25-rc5 unix_stream_sendmsg
> 217413 1.9331 linux-2.6.25-rc5 memset_c
> 211589 1.8813 linux-2.6.25-rc5 kmem_cache_alloc
> 151500 1.3470 linux-2.6.25-rc5 system_call
> 132262 1.1760 linux-2.6.25-rc5 sock_def_readable
> 123130 1.0948 linux-2.6.25-rc5 kmem_cache_free
> 109518 0.9738 linux-2.6.25-rc5 sock_wfree
On tigerton, if I add "slub_max_order=3 slub_min_objects=16" to kernel boot cmdline,
the result is improved significantly and it takes just 1/10 time of the original testing.

Below is the new output of slabino -AD.

Name Objects Alloc Free %Fast
:0000192 3192 80087199 80084141 92 8
kmalloc-512 773 80016203 80015888 97 9
vm_area_struct 2787 223100 220525 94 17
:0004096 68 118322 118320 99 98
:0000064 12215 123575 111669 98 42
:0000128 4616 53826 49422 97 45
dentry 12373 49568 37286 95 65
:0000080 12823 33755 21206 99 97


So kmalloc-512 is the key.


Then, I tested it on stoakley with the same kernel commandline. Improvement is about 50%.
One important thing is without the boot parameter, hackbench on stoakey takes only 1/4 time
of the one on tigerton. With the boot parameter, hackbench on tigerton is faster than the one
on stoakely.

Is it possible to initiate slub_min_objects based on possible cpu number? I mean,
cpu_possible_map(). We could calculate slub_min_objects by a formular.

-yanmin




\
 
 \ /
  Last update: 2008-03-14 06:39    [W:0.062 / U:23.424 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site