lkml.org 
[lkml]   [2015]   [Oct]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Date
SubjectRe: [RFC PATCH 0/3] restartable sequences benchmarks
On Thu, Oct 22, 2015 at 11:06 AM, Dave Watson <davejwatson@fb.com> wrote:
> We've been testing out restartable sequences + malloc changes for use
> at Facebook. Below are some test results, as well as some possible
> changes based on Paul Turner's original patches

Thanks! I'll stare at this some time between now and Kernel Summit.

>
> https://lkml.org/lkml/2015/6/24/665
>
> I ran one service with several permutations of various mallocs. The
> service is CPU-bound, and hits the allocator quite hard. Requests/s
> are held constant at the source, so we use cpu idle time and latency
> as an indicator of service quality. These are average numbers over
> several hours. Machines were dual E5-2660, total 16 cores +
> hyperthreading. This service has ~400 total threads, 70-90 of which
> are doing work at any particular time.
>
> RSS CPUIDLE LATENCYMS
> jemalloc 4.0.0 31G 33% 390
> jemalloc + this patch 25G 33% 390
> jemalloc + this patch using lsl 25G 30% 420
> jemalloc + PT's rseq patch 25G 32% 405
> glibc malloc 2.20 27G 30% 420
> tcmalloc gperftools trunk (2.2) 21G 30% 480

Slightly confused. This is showing a space efficiency improvement but
not a performance improvement? Is the idea that percpu free lists are
more space efficient than per-thread free lists?

>
> jemalloc rseq patch used for testing:
> https://github.com/djwatson/jemalloc
>
> lsl test - using lsl segment limit to get cpu (i.e. inlined vdso
> getcpu on x86) instead of using the thread caching as in this patch.
> There has been some suggestions to add the thread-cached getcpu()
> feature separately. It does seem to move the needle in a real service
> by about ~3% to have a thread-cached getcpu vs. not. I don't think we
> can use restartable sequences in production without a faster getcpu.

If nothing else, I'd like to replace the thread-cached getcpu thing
with percpu gsbase, at least on x86. That doesn't necessarily have to
be exclusive with restartable sequences.

>
> GS-segment / migration only tests
>
> There's been some interest in seeing if we can do this with only gs
> segment, here's some numbers for those. This doesn't have to be gs,
> it could just be a migration signal sent to userspace as well, the
> same approaches would apply.
>
> GS patch: https://lkml.org/lkml/2014/9/13/59
>
> RSS CPUIDLE LATENCYMS
> jemalloc 4.0.0 31G 33% 390
> jemalloc + percpu locking 25G 25% 420
> jemalloc + preempt lock / signal 25G 32% 415

Neat!

--Andy


\
 
 \ /
  Last update: 2015-10-22 21:21    [W:0.174 / U:0.640 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site