Messages in this thread Patch in this message | | | Date | Wed, 19 Mar 2014 16:05:29 +0100 (CET) | From | Thomas Gleixner <> | Subject | Re: rb tree hrtimer lockup bug (found by perf_fuzzer) |
| |
On Wed, 19 Mar 2014, Vince Weaver wrote: > On Wed, 19 Mar 2014, Thomas Gleixner wrote: > > > On Wed, 19 Mar 2014, Vince Weaver wrote: > > > On Tue, 18 Mar 2014, Thomas Gleixner wrote: > > > your patch didn't seem to print anything additional the first time throug. > > > > > > I then tried the trace command you suggested, but I'm getting an empty > > > ftrace buffer which possibly means I don't have enough ftrace kernel > > > options enabled. > > > > > > Here's the most recent boot crash. > > > > > > [ 5.367069] ODEBUG: Info active (active state 0) object type: timer_list hint: (null) > > > > Stupid me. We get the hint from the wreckaged object .... > > > > A hopefully better approach is the delta patch below. > > with that applied on top: > > [ 5.342681] Invalid timer base: tmr ffff880117740150 tmr->base (null) base ffff880118618000 > [ 5.352786] ------------[ cut here ]------------ > [ 5.357911] WARNING: CPU: 4 PID: 0 at lib/debugobjects.c:260 debug_print_object+0x8c/0xb0() > [ 5.367023] ODEBUG: Info active (active state 0) object type: timer_list hint: (null) delayed_work_timer_fn+0x0/0x20
Good, we have at least a point where we can gather useful information. The timer belongs to a delayed work. The work itself is tracked via debug objects as well. So we can get that as well. Delta patch below.
Thanks,
tglx ------
Index: linux-2.6/lib/debugobjects.c =================================================================== --- linux-2.6.orig/lib/debugobjects.c +++ linux-2.6/lib/debugobjects.c @@ -14,6 +14,7 @@ #include <linux/debugfs.h> #include <linux/slab.h> #include <linux/hash.h> +#include <linux/workqueue.h> #define ODEBUG_HASH_BITS 14 #define ODEBUG_HASH_SIZE (1 << ODEBUG_HASH_BITS) @@ -400,6 +401,20 @@ void debug_object_info(void *addr, struc debug_print_object(obj, "Info"); raw_spin_unlock_irqrestore(&db->lock, flags); + + addr = container_of(addr, struct delayed_work, timer); + + db = get_bucket((unsigned long) addr); + + raw_spin_lock_irqsave(&db->lock, flags); + + obj = lookup_object(addr, db); + if (!obj) + pr_err("Object unknown %p\n", addr); + else + debug_print_object(obj, "Info"); + + raw_spin_unlock_irqrestore(&db->lock, flags); } /** @@ -452,6 +467,7 @@ int debug_object_activate(void *addr, st } if (!obj->hint) obj->hint = descr->debug_hint ? descr->debug_hint(addr) : NULL; + raw_spin_unlock_irqrestore(&db->lock, flags); return ret; }
| |