lkml.org 
[lkml]   [2011]   [May]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH v4 3/6] x86-64: Don't generate cmov in vread_tsc
    Date
    vread_tsc checks whether rdtsc returns something less than
    cycle_last, which is an extremely predictable branch. GCC likes
    to generate a cmov anyway, which is several cycles slower than
    a predicted branch. This saves a couple of nanoseconds.

    Signed-off-by: Andy Lutomirski <luto@mit.edu>
    ---
    arch/x86/kernel/tsc.c | 18 ++++++++++++++++--
    1 files changed, 16 insertions(+), 2 deletions(-)
    diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
    index 7cabdae..db4c6e6 100644
    --- a/arch/x86/kernel/tsc.c
    +++ b/arch/x86/kernel/tsc.c
    @@ -767,6 +767,7 @@ static cycle_t read_tsc(struct clocksource *cs)
    static cycle_t __vsyscall_fn vread_tsc(void)
    {
    cycle_t ret;
    + u64 last;

    /*
    * Empirically, a fence (of type that depends on the CPU)
    @@ -778,8 +779,21 @@ static cycle_t __vsyscall_fn vread_tsc(void)
    rdtsc_barrier();
    ret = (cycle_t)vget_cycles();

    - return ret >= VVAR(vsyscall_gtod_data).clock.cycle_last ?
    - ret : VVAR(vsyscall_gtod_data).clock.cycle_last;
    + last = VVAR(vsyscall_gtod_data).clock.cycle_last;
    +
    + if (likely(ret >= last))
    + return ret;
    +
    + /*
    + * GCC likes to generate cmov here, but this branch is extremely
    + * predictable (it's just a funciton of time and the likely is
    + * very likely) and there's a data dependence, so force GCC
    + * to generate a branch instead. I don't barrier() because
    + * we don't actually need a barrier, and if this function
    + * ever gets inlined it will generate worse code.
    + */
    + asm volatile ("");
    + return last;
    }
    #endif

    --
    1.7.5.1


    \
     
     \ /
      Last update: 2011-05-16 18:07    [W:0.441 / U:0.844 seconds]
    ©2003-2014 Jasper Spaans. Advertise on this site