Messages in this thread | | | From | Jiri Olsa <> | Date | Sun, 19 Mar 2023 16:29:22 +0100 | Subject | Re: [PATCH 5/7] ftrace: Store direct called addresses in their ops |
| |
On Thu, Mar 16, 2023 at 06:38:09PM +0100, Florent Revest wrote: > All direct calls are now registered using the register_ftrace_direct API > so each ops can jump to only one direct-called trampoline. > > By storing the direct called trampoline address directly in the ops we > can save one hashmap lookup in the direct call ops and implement arm64 > direct calls on top of call ops. > > Signed-off-by: Florent Revest <revest@chromium.org> > --- > include/linux/ftrace.h | 3 +++ > kernel/trace/ftrace.c | 7 +++++-- > 2 files changed, 8 insertions(+), 2 deletions(-) > > diff --git a/include/linux/ftrace.h b/include/linux/ftrace.h > index abee60865fc7..6a532dd6789e 100644 > --- a/include/linux/ftrace.h > +++ b/include/linux/ftrace.h > @@ -321,6 +321,9 @@ struct ftrace_ops { > unsigned long trampoline_size; > struct list_head list; > ftrace_ops_func_t ops_func; > +#ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS > + unsigned long direct_call; > +#endif > #endif > }; > > diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c > index 33530198d1ca..66c91fa4b6ab 100644 > --- a/kernel/trace/ftrace.c > +++ b/kernel/trace/ftrace.c > @@ -2582,9 +2582,8 @@ ftrace_add_rec_direct(unsigned long ip, unsigned long addr, > static void call_direct_funcs(unsigned long ip, unsigned long pip, > struct ftrace_ops *ops, struct ftrace_regs *fregs) > { > - unsigned long addr; > + unsigned long addr = ops->direct_call;
nice, should it be read with READ_ONCE ?
jirka
> > - addr = ftrace_find_rec_direct(ip); > if (!addr) > return; > > @@ -5380,6 +5379,7 @@ int register_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) > ops->func = call_direct_funcs; > ops->flags = MULTI_FLAGS; > ops->trampoline = FTRACE_REGS_ADDR; > + ops->direct_call = addr; > > err = register_ftrace_function_nolock(ops); > > @@ -5454,6 +5454,7 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) > /* Enable the tmp_ops to have the same functions as the direct ops */ > ftrace_ops_init(&tmp_ops); > tmp_ops.func_hash = ops->func_hash; > + tmp_ops.direct_call = addr; > > err = register_ftrace_function_nolock(&tmp_ops); > if (err) > @@ -5475,6 +5476,8 @@ __modify_ftrace_direct(struct ftrace_ops *ops, unsigned long addr) > entry->direct = addr; > } > } > + /* Prevent store tearing if a trampoline concurrently accesses the value */ > + WRITE_ONCE(ops->direct_call, addr); > > mutex_unlock(&ftrace_lock); > > -- > 2.40.0.rc2.332.ga46443480c-goog >
| |