Messages in this thread Patch in this message | | | Date | Wed, 25 Feb 2015 14:25:05 +0100 | From | Denys Vlasenko <> | Subject | Re: [PATCH 2/4] x86: get rid of KERNEL_STACK_OFFSET |
| |
On 02/25/2015 01:48 PM, Denys Vlasenko wrote: > On Wed, Feb 25, 2015 at 9:53 AM, Ingo Molnar <mingo@kernel.org> wrote: >> But the fix should be to not touch RSP in SAVE_ARGS, to >> keep percpu::kernel_stack as an optimized entry point - >> with KERNEL_STACK_OFFSET pointing to. >> >> So NAK - this should be fixed for real. > > IOW, the proposal is to set KERNEL_STACK_OFFSET > to SIZEOF_PTREGS. I can do that. > > However. > > There is an ortogonal idea we were discussing: to save > registers and construct iret frame using PUSH insns, not MOVs. > IIRC Andy and Linus liked it. I am ambivalent: the code will be smaller, > but might get slower (at least on some CPUs). > If we go that way, we will require KERNEL_STACK_OFFSET = 0 > (IOW: the current patch). > > The decision on how exactly we should fix KERNEL_STACK_OFFSET > (set it to SIZEOF_PTREGS or to zero) depends on whether > we switch to using PUSHes, or not. What do you think?
A data point. I implemented push-based creation of pt_regs and benchmarked it. The patch is on top of all my latest patches sent to ML.
On SandyBridge CPU, it does not get slower: seems to be 1 cycle faster per syscall.
We lose a number of large insns there:
text data bss dec hex filename - 9863 0 0 9863 2687 entry_64.o + 9671 0 0 9671 25c7 entry_64.o
diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S index f505cb6..d97bd92 100644 --- a/arch/x86/kernel/entry_64.S +++ b/arch/x86/kernel/entry_64.S @@ -128,8 +128,6 @@ ENDPROC(native_usergs_sysret64) * manipulation. */ .macro FIXUP_TOP_OF_STACK tmp offset=0 - movq $__USER_DS,SS+\offset(%rsp) - movq $__USER_CS,CS+\offset(%rsp) movq RIP+\offset(%rsp),\tmp /* get rip */ movq \tmp,RCX+\offset(%rsp) /* copy it to rcx as sysret would do */ movq EFLAGS+\offset(%rsp),\tmp /* ditto for rflags->r11 */ @@ -245,14 +243,22 @@ GLOBAL(system_call_after_swapgs) * and short: */ ENABLE_INTERRUPTS(CLBR_NONE) - ALLOC_PT_GPREGS_ON_STACK 6*8 /* 6*8: space for orig_ax and iret frame */ - movq %rcx,RIP(%rsp) - movq %r11,EFLAGS(%rsp) - movq PER_CPU_VAR(old_rsp),%rcx - movq %rcx,RSP(%rsp) - movq_cfi rax,ORIG_RAX - SAVE_C_REGS_EXCEPT_RAX_RCX_R11 - movq $-ENOSYS,RAX(%rsp) + /* Construct struct pt_regs on stack */ + pushq $__USER_DS /* pt_regs->ss */ + pushq PER_CPU_VAR(old_rsp) /* pt_regs->sp */ + pushq %r11 /* pt_regs->flags */ + pushq $__USER_CS /* pt_regs->cs */ + pushq %rcx /* pt_regs->ip */ + pushq %rax /* pt_regs->orig_ax */ + pushq %rdi /* pt_regs->di */ + pushq %rsi /* pt_regs->si */ + pushq %rdx /* pt_regs->dx */ + pushq %rcx /* pt_regs->cx */ + pushq $-ENOSYS /* pt_regs->ax */ + pushq %r8 /* pt_regs->r8 */ + pushq %r9 /* pt_regs->r9 */ + pushq %r10 /* pt_regs->r10 */ + sub $(7*8),%rsp /* pt_regs->r11,bp,bx,r12-15 */ CFI_REL_OFFSET rip,RIP testl $_TIF_WORK_SYSCALL_ENTRY,TI_flags+THREAD_INFO(%rsp,SIZEOF_PTREGS) jnz tracesys
| |