Messages in this thread Patch in this message | | | Date | Fri, 24 Mar 2023 15:23:09 -0400 | From | Steven Rostedt <> | Subject | Re: [PATCH] ring-buffer: Fix race while reader and writer are on the same page |
| |
On Fri, 24 Mar 2023 20:50:37 +0800 Zheng Yejian <zhengyejian1@huawei.com> wrote:
> Fixes: 77ae365eca89 ("ring-buffer: make lockless") > Signed-off-by: Zheng Yejian <zhengyejian1@huawei.com> > --- > kernel/trace/ring_buffer.c | 14 +++++++++++++- > 1 file changed, 13 insertions(+), 1 deletion(-) > > diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c > index c6f47b6cfd5f..79fd5e10ee05 100644 > --- a/kernel/trace/ring_buffer.c > +++ b/kernel/trace/ring_buffer.c > @@ -2942,6 +2942,13 @@ rb_update_event(struct ring_buffer_per_cpu *cpu_buffer, > event->array[0] = length; > } else > event->type_len = DIV_ROUND_UP(length, RB_ALIGNMENT); > + > + /* > + * The 'event' may be reserved from the page which is reading > + * by reader, make sure 'event' is completely updated before > + * reader_page->page->commit being set. > + */ > + smp_wmb();
This isn't the place to put this. We only care before the commit is updated, not at *ever* update to the event (this can be called several times before a commit).
If we need to add a smp_wmb() it's best to be in rb_set_commit_to_write()
> } > > static unsigned rb_calculate_event_length(unsigned length) > @@ -4684,7 +4691,12 @@ rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer) > > /* > * Make sure we see any padding after the write update > - * (see rb_reset_tail()) > + * (see rb_reset_tail()). > + * > + * In addition, writer may be writing on the reader page > + * if the page has not been fully filled, so the read barrier > + * is also needed to make sure we see the completely updated > + * event that reserved by writer (see rb_update_event()). > */ > smp_rmb(); >
I think we want this instead:
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 2d5c3caff32d..22d05cd04a3a 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -3092,6 +3092,10 @@ rb_set_commit_to_write(struct ring_buffer_per_cpu *cpu_buffer) if (RB_WARN_ON(cpu_buffer, rb_is_reader_page(cpu_buffer->tail_page))) return; + /* + * No need for a memory barrier here, as the update + * of the tail_page did it for this page. + */ local_set(&cpu_buffer->commit_page->page->commit, rb_page_write(cpu_buffer->commit_page)); rb_inc_page(&cpu_buffer->commit_page); @@ -3101,6 +3105,8 @@ rb_set_commit_to_write(struct ring_buffer_per_cpu *cpu_buffer) while (rb_commit_index(cpu_buffer) != rb_page_write(cpu_buffer->commit_page)) { + /* Make sure the readers see the content of what is committed. */ + smp_wmb(); local_set(&cpu_buffer->commit_page->page->commit, rb_page_write(cpu_buffer->commit_page)); RB_WARN_ON(cpu_buffer, @@ -4676,7 +4682,12 @@ rb_get_reader_page(struct ring_buffer_per_cpu *cpu_buffer) /* * Make sure we see any padding after the write update - * (see rb_reset_tail()) + * (see rb_reset_tail()). + * + * In addition, a writer may be writing on the reader page + * if the page has not been fully filled, so the read barrier + * is also needed to make sure we see the completely updated + * event reserved by the writer (see rb_tail_page_update()). */ smp_rmb();
| |