[lkml]   [2008]   [Sep]   [18]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    Patch in this message
    Subject[RFC PATCH] rcu: introduce kfree_rcu()

    sometimes a rcu callback is just calling kfree() to free a struct's memory
    (we say this callback is a trivial callback.).
    this patch introduce kfree_rcu() to do these things directly, easily.

    There are 4 reasons that we need kfree_rcu():

    1) unloadable modules:
    a module(rcu callback is defined in this module) using rcu must
    call rcu_barrier() when unload. rcu_barrier() will increase
    the system's overhead(the more cpus the worse) and
    rcu_barrier() is very time-consuming. if all rcu callback defined
    in this module are trivial callback, we can just call kfree_rcu()
    instead, save a rcu_barrier() when unload.

    2) duplicate code:
    all trivial callback are duplicate code though the structs to be freed
    are different. it's just a container_of() and a kfree().
    There are about 50% callbacks are trivial callbacks for call_rcu() in
    current kernel code.

    3) cache:
    the instructions of trivial callback is not in the cache supposedly.
    calling a trivial callback will let to cache missing very likely.
    the more trivial callback the more cache missing. OK, this is
    not a problem now or in a few days: Only less than 1% trivial callback
    are called in running kernel.

    4) future:
    the number of user of rcu is increasing. new code for rcu is
    trivial callback very likely. it means more modules using rcu
    and more duplicate code(may come to 90% of callbacks is trivial
    callbacks) and more cache missing.

    there were a lot of ideas came out when i implemented kfree_rcu().
    I chose the simplest one as this patch shows. but these implementation
    may cannot be used for to free a struct larger than 16KBytes.

    kfree_rcu_bh()? kfree_rcu_sched()?
    these two are not need current. call_rcu_bh() & call_rcu_sched()
    are hardly be called(and hardly be called for trivial callback).

    No, vfree() is not atomic function, will not be called in softirq.

    Signed-off-by: Lai Jiangshan <>
    diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h
    index e8b4039..04c654f 100644
    --- a/include/linux/rcupdate.h
    +++ b/include/linux/rcupdate.h
    @@ -253,4 +253,25 @@ extern void rcu_barrier_sched(void);
    extern void rcu_init(void);
    extern int rcu_needs_cpu(int cpu);

    +#define __KFREE_RCU_MAX_OFFSET 4095
    +#define KFREE_RCU_MAX_OFFSET (sizeof(void *) * __KFREE_RCU_MAX_OFFSET)
    +#define __rcu_reclaim(head) \
    +do { \
    + unsigned long __offset = (unsigned long)head->func; \
    + if (__offset <= __KFREE_RCU_MAX_OFFSET) \
    + kfree((void *)head - sizeof(void *) * __offset); \
    + else \
    + head->func(head); \
    +} while(0)
    + * kfree_rcu - free previously allocated memory after a grace period.
    + * @ptr: pointer returned by kmalloc.
    + * @head: structure to be used for queueing the RCU updates. This structure
    + * is a part of previously allocated memory @ptr.
    + */
    +extern void kfree_rcu(const void *ptr, struct rcu_head *head);
    #endif /* __LINUX_RCUPDATE_H */
    diff --git a/kernel/rcuclassic.c b/kernel/rcuclassic.c
    index aad93cd..5a14190 100644
    --- a/kernel/rcuclassic.c
    +++ b/kernel/rcuclassic.c
    @@ -232,7 +232,7 @@ static void rcu_do_batch(struct rcu_data *rdp)
    while (list) {
    next = list->next;
    - list->func(list);
    + __rcu_reclaim(list);
    list = next;
    if (++count >= rdp->blimit)
    diff --git a/kernel/rcupdate.c b/kernel/rcupdate.c
    index 467d594..aa9b56a 100644
    --- a/kernel/rcupdate.c
    +++ b/kernel/rcupdate.c
    @@ -162,6 +162,18 @@ void rcu_barrier_sched(void)

    +void kfree_rcu(const void *ptr, struct rcu_head *head)
    + unsigned long offset;
    + typedef void (*rcu_callback)(struct rcu_head *);
    + offset = (void *)head - (void *)ptr;
    + BUG_ON(offset > KFREE_RCU_MAX_OFFSET);
    + call_rcu(head, (rcu_callback)(offset / sizeof(void *)));
    void __init rcu_init(void)
    diff --git a/kernel/rcupreempt.c b/kernel/rcupreempt.c
    index 2782793..62a9e54 100644
    --- a/kernel/rcupreempt.c
    +++ b/kernel/rcupreempt.c
    @@ -1108,7 +1108,7 @@ static void rcu_process_callbacks(struct softirq_action *unused)
    spin_unlock_irqrestore(&rdp->lock, flags);
    while (list) {
    next = list->next;
    - list->func(list);
    + __rcu_reclaim(list);
    list = next;

     \ /
      Last update: 2008-09-18 06:23    [W:0.050 / U:98.548 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site