lkml.org 
[lkml]   [2019]   [Jun]   [8]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    Patch in this message
    /
    From
    Subject[PATCH AUTOSEL 5.1 22/70] perf/ring_buffer: Add ordering to rb->nest increment
    Date
    From: Peter Zijlstra <peterz@infradead.org>

    [ Upstream commit 3f9fbe9bd86c534eba2faf5d840fd44c6049f50e ]

    Similar to how decrementing rb->next too early can cause data_head to
    (temporarily) be observed to go backward, so too can this happen when
    we increment too late.

    This barrier() ensures the rb->head load happens after the increment,
    both the one in the 'goto again' path, as the one from
    perf_output_get_handle() -- albeit very unlikely to matter for the
    latter.

    Suggested-by: Yabin Cui <yabinc@google.com>
    Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
    Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
    Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
    Cc: Jiri Olsa <jolsa@redhat.com>
    Cc: Linus Torvalds <torvalds@linux-foundation.org>
    Cc: Peter Zijlstra <peterz@infradead.org>
    Cc: Stephane Eranian <eranian@google.com>
    Cc: Thomas Gleixner <tglx@linutronix.de>
    Cc: Vince Weaver <vincent.weaver@maine.edu>
    Cc: acme@kernel.org
    Cc: mark.rutland@arm.com
    Cc: namhyung@kernel.org
    Fixes: ef60777c9abd ("perf: Optimize the perf_output() path by removing IRQ-disables")
    Link: http://lkml.kernel.org/r/20190517115418.309516009@infradead.org
    Signed-off-by: Ingo Molnar <mingo@kernel.org>
    Signed-off-by: Sasha Levin <sashal@kernel.org>
    ---
    kernel/events/ring_buffer.c | 9 +++++++++
    1 file changed, 9 insertions(+)

    diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c
    index 009467a60578..4b5f8d932400 100644
    --- a/kernel/events/ring_buffer.c
    +++ b/kernel/events/ring_buffer.c
    @@ -48,6 +48,15 @@ static void perf_output_put_handle(struct perf_output_handle *handle)
    unsigned long head;

    again:
    + /*
    + * In order to avoid publishing a head value that goes backwards,
    + * we must ensure the load of @rb->head happens after we've
    + * incremented @rb->nest.
    + *
    + * Otherwise we can observe a @rb->head value before one published
    + * by an IRQ/NMI happening between the load and the increment.
    + */
    + barrier();
    head = local_read(&rb->head);

    /*
    --
    2.20.1
    \
     
     \ /
      Last update: 2019-06-08 13:55    [W:4.089 / U:0.552 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site