Messages in this thread | | | Date | Mon, 29 Jan 2018 18:13:19 +0100 | From | Peter Zijlstra <> | Subject | Re: [PATCH v2 1/2] x86/retpoline: Simplify vmexit_fill_RSB() |
| |
On Fri, Jan 26, 2018 at 09:07:25PM +0100, Borislav Petkov wrote: > +.macro FILL_RETURN_BUFFER nr:req ftr:req > #ifdef CONFIG_RETPOLINE > + ALTERNATIVE "", "call __clear_rsb", \ftr > #endif > .endm > > @@ -206,15 +174,10 @@ extern char __indirect_thunk_end[]; > static inline void vmexit_fill_RSB(void) > { > #ifdef CONFIG_RETPOLINE > + alternative_input("", > + "call __fill_rsb", > + X86_FEATURE_RETPOLINE, > + ASM_NO_INPUT_CLOBBER("memory")); > #endif > } >
> @@ -19,6 +20,37 @@ ENDPROC(__x86_indirect_thunk_\reg) > .endm > > /* > + * Google experimented with loop-unrolling and this turned out to be > + * the optimal version — two calls, each with their own speculation > + * trap should their return address end up getting used, in a loop. > + */ > +.macro BOINK_RSB nr:req sp:req > + push %_ASM_AX > + mov $(\nr / 2), %_ASM_AX > + .align 16 > +771: > + call 772f > +773: /* speculation trap */ > + pause > + lfence > + jmp 773b > + .align 16 > +772: > + call 774f > +775: /* speculation trap */ > + pause > + lfence > + jmp 775b > + .align 16 > +774: > + dec %_ASM_AX > + jnz 771b > + add $((BITS_PER_LONG/8) * \nr), \sp > + pop %_ASM_AX > +.endm > + > + > +/* > * Despite being an assembler file we can't just use .irp here > * because __KSYM_DEPS__ only uses the C preprocessor and would > * only see one instance of "__x86_indirect_thunk_\reg" rather > @@ -46,3 +78,15 @@ GENERATE_THUNK(r13) > GENERATE_THUNK(r14) > GENERATE_THUNK(r15) > #endif > + > +ENTRY(__fill_rsb) > + BOINK_RSB RSB_FILL_LOOPS, %_ASM_SP > + ret > +END(__fill_rsb) > +EXPORT_SYMBOL_GPL(__fill_rsb) > + > +ENTRY(__clear_rsb) > + BOINK_RSB RSB_CLEAR_LOOPS, %_ASM_SP > + ret > +END(__clear_rsb) > +EXPORT_SYMBOL_GPL(__clear_rsb)
One thing I feel this ought to mention (in the Changelog probably) is that it looses one RET for SKL+. That is, where we used to have 16 'safe' RETs before this, we now have 15.
| |