Messages in this thread | | | Date | Fri, 25 Feb 2011 13:23:53 -0500 | From | Mathieu Desnoyers <> | Subject | Re: [cpuops cmpxchg double V3 2/5] slub: Get rid of slab_free_hook_irq() |
| |
* Christoph Lameter (cl@linux.com) wrote: > The following patch will make the fastpaths lockless and will no longer > require interrupts to be disabled. Calling the free hook with irq disabled > will no longer be possible. > > Move the slab_free_hook_irq() logic into slab_free_hook. Only disable > interrupts if the features are selected that require callbacks with > interrupts off and reenable after calls have been made. > > Signed-off-by: Christoph Lameter <cl@linux.com> > > --- > mm/slub.c | 29 +++++++++++++++++------------ > 1 file changed, 17 insertions(+), 12 deletions(-) > > Index: linux-2.6/mm/slub.c > =================================================================== > --- linux-2.6.orig/mm/slub.c 2011-01-10 12:06:58.000000000 -0600 > +++ linux-2.6/mm/slub.c 2011-01-10 12:07:11.000000000 -0600 > @@ -807,14 +807,24 @@ static inline void slab_post_alloc_hook( > static inline void slab_free_hook(struct kmem_cache *s, void *x) > { > kmemleak_free_recursive(x, s->flags); > -} > > -static inline void slab_free_hook_irq(struct kmem_cache *s, void *object) > -{ > - kmemcheck_slab_free(s, object, s->objsize); > - debug_check_no_locks_freed(object, s->objsize); > - if (!(s->flags & SLAB_DEBUG_OBJECTS)) > - debug_check_no_obj_freed(object, s->objsize); > + /* > + * Trouble is that we may no longer disable interupts in the fast path
interrupts
/nitpick ;)
Mathieu
> + * So in order to make the debug calls that expect irqs to be > + * disabled we need to disable interrupts temporarily. > + */ > +#if defined(CONFIG_KMEMCHECK) || defined(CONFIG_LOCKDEP) > + { > + unsigned long flags; > + > + local_irq_save(flags); > + kmemcheck_slab_free(s, x, s->objsize); > + debug_check_no_locks_freed(x, s->objsize); > + if (!(s->flags & SLAB_DEBUG_OBJECTS)) > + debug_check_no_obj_freed(x, s->objsize); > + local_irq_restore(flags); > + } > +#endif > } > > /* > @@ -1101,9 +1111,6 @@ static inline void slab_post_alloc_hook( > > static inline void slab_free_hook(struct kmem_cache *s, void *x) {} > > -static inline void slab_free_hook_irq(struct kmem_cache *s, > - void *object) {} > - > #endif /* CONFIG_SLUB_DEBUG */ > > /* > @@ -1909,8 +1916,6 @@ static __always_inline void slab_free(st > local_irq_save(flags); > c = __this_cpu_ptr(s->cpu_slab); > > - slab_free_hook_irq(s, x); > - > if (likely(page == c->page && c->node != NUMA_NO_NODE)) { > set_freepointer(s, object, c->freelist); > c->freelist = object; >
-- Mathieu Desnoyers Operating System Efficiency R&D Consultant EfficiOS Inc. http://www.efficios.com
| |