lkml.org 
[lkml]   [2011]   [Jul]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
Date
Subject[RFC] syscall calling convention, stts/clts, and xstate latency
I was trying to understand the FPU/xstate saving code, and I ran some
benchmarks with surprising results. These are all on Sandy Bridge
i7-2600. Please take all numbers with a grain of salt -- they're in
tight-ish loops and don't really take into account real-world cache
effects.

A clts/stts pair takes about 80 ns. Accessing extended state from
userspace with TS set takes 239 ns. A kernel_fpu_begin /
kernel_fpu_end pair with no userspace xstate access takes 80 ns
(presumably 79 of those 80 are the clts/stts). (Note: The numbers in
this paragraph were measured using a hacked-up kernel and KVM.)

With nonzero ymm state, xsave + clflush (on the first cacheline of
xstate) + xrstor takes 128 ns. With hot cache, xsave = 24ns, xsaveopt
(with unchanged state) = 16 ns, and xrstor = 40 ns.

With nonzero xmm state but zero ymm state, xsave+xrstor drops to 38 ns
and xsaveopt saves another 5 ns.

Zeroing the state completely with vzeroall adds 2 ns. Not sure what's going on.

All of this makes me think that, at least on Sandy Bridge, lazy xstate
saving is a bad optimization -- if the cache is being nice,
save/restore is faster than twiddling the TS bit. And the cost of the
trap when TS is set blows everything else away.


Which brings me to another question: what do you think about declaring
some of the extended state to be clobbered by syscall? Ideally, we'd
treat syscall like a regular function and clobber everything except
the floating point control word and mxcsr. More conservatively, we'd
leave xmm and x87 state but clobber ymm. This would let us keep the
cost of the state save and restore down when kernel_fpu_begin is used
in a syscall path and when a context switch happens as a result of a
syscall.

glibc does *not* mark the xmm registers as clobbered when it issues
syscalls, but I suspect that everything everywhere that issues
syscalls does it from a function, and functions are implicitly assumed
to clobber extended state. (And if anything out there assumes that
ymm state is preserved, I'd be amazed.)


--Andy


\
 
 \ /
  Last update: 2011-07-24 23:09    [W:0.072 / U:0.148 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site