Messages in this thread | | | Date | Fri, 25 Feb 2011 15:41:47 +0100 | From | Dominik Klein <> | Subject | Re: Is it a workqueue related issue in 2.6.37 (Was: Re: [libvirt] blkio cgroup [solved]) |
| |
On 02/25/2011 02:18 PM, Tejun Heo wrote: > Hello, > > On Fri, Feb 25, 2011 at 12:46:16PM +0100, Dominik Klein wrote: >> With 2.6.37 (also tried .1 and .2) it does not work but end up like I >> documented. With 2.6.38-rc1, it does work. With deadline scheduler, it >> also works in 2.6.37. > > Okay, here's the problematic part. > > <idle>-0 [013] 1640.975562: workqueue_queue_work: work struct=ffff88080f14f270 function=blk_throtl_work workqueue=ffff88102c8fc700 req_cpu=13 cpu=13 > <idle>-0 [013] 1640.975564: workqueue_activate_work: work struct ffff88080f14f270 > <...>-477 [013] 1640.975574: workqueue_execute_start: work struct ffff88080f14f270: function blk_throtl_work > <idle>-0 [013] 1641.087450: workqueue_queue_work: work struct=ffff88080f14f270 function=blk_throtl_work workqueue=ffff88102c8fc700 req_cpu=13 cpu=13 > > The workqueue is per-cpu, so we only need to follow cpu=13 cases. > @1640, blk_throtl_work() is queued, activated and starts executing but > never finishes. The same work item is never executed more than once > at the same on the same CPU, so when the next work item is queued, it > doesn't get activated until the previous execution is complete. > > The next thing to do would be finding out why blk_throtl_work() isn't > finishing. sysrq-t or /proc/PID/stack should show us where it's > stalled. > > Thanks. >
See attached logs of another run.
sysctl -w kernel.sysrq=1
echo blk > /sys/kernel/debug/tracing/current_tracer echo 1 > /sys/block/sdb/trace/enable echo workqueue_queue_work >> /sys/kernel/debug/tracing/set_event echo workqueue_activate_work >> /sys/kernel/debug/tracing/set_event echo workqueue_execute_start >> /sys/kernel/debug/tracing/set_event echo workqueue_execute_end >> /sys/kernel/debug/tracing/set_event
That makes attachment trace_pipe5.gz
echo 8 > /proc/sysrq-trigger echo t > /proc/sysrq-trigger
That makes attachment console.gz
hth Dominik [unhandled content-type:application/gzip][unhandled content-type:application/gzip] | |