Messages in this thread | | | Date | Wed, 9 Aug 2017 18:42:50 +0200 | From | Peter Zijlstra <> | Subject | Re: [PATCH v3] scheduler: enhancement to show_state_filter and SysRq |
| |
On Wed, Aug 09, 2017 at 05:26:14PM +0800, Yafang Shao wrote: > 2017-08-09 17:09 GMT+08:00 Peter Zijlstra <peterz@infradead.org>: > > On Wed, Aug 09, 2017 at 04:01:49PM +0800, Yafang Shao wrote: > >> 2017-08-09 15:43 GMT+08:00 Peter Zijlstra <peterz@infradead.org>: > >> > On Wed, Aug 09, 2017 at 06:31:28PM +0800, Yafang Shao wrote: > >> >> Sometimes we want to get tasks in TASK_RUNNING sepcifically, > >> >> instead of dump all tasks. > >> >> > >> >> For example, when the loadavg are high, we want to dump > >> >> tasks in TASK_RUNNING and TASK_UNINTERRUPTIBLE, which contribute > >> >> to system load. But mostly there're lots of tasks in Sleep state, > >> >> which occupies almost all of the kernel log buffer, even overflows > >> >> it, that causes the useful messages get lost. Although we can > >> >> enlarge the kernel log buffer, but that's not a good idea. > >> > > >> > That's what you have serial consoles for... > >> > > >> mostly we don't even have one console because we alwayas login the > >> servers via ssh. And manage the servers with console is not so convenient. > > > > I find IPMI SOL very useful. Serial console (esp. earlyprintk) keeps on > > working long after most other things have died. > > > > In any case, you can easily dump the printk output into your ssh session > > if you want, use something like: > > > > cat /dev/kmsg | tee logfile & echo t > /proc/sysrq-trigger > > that's what I'm doing it currently :) > Then I thought deeply why not do it more smartly? > Introducing a new key(here I just modified the key 'w') only dump > tasks in running and blocked should be more smarter.
Since you're strictly ssh based, you could maybe do a sysctl that allows changing the 'default' filter of sysctl-t, dunno if that makes sense though.
Also, since you're not actually debugging a dead machine, maybe you can do a custom kernel module / systemtap / ebpf thing that collects precisely the information you want.
sysrq is typically a last ditch debug mostly dead machine thing, which you're very much not having.
> > > > I really see no problem here. Then you can run a bit of awk or whatever > > your favourite tool is to filter out the stuff you don't want. > > > Another question, if we could filter with scritpts in userland, why > did we introduced the key 'w' to dump only blocked state > as we already have a key 't' to dump all tasks ?
No idea that is long before my time, I expect because 'w' (blocked) is typically a small number of tasks.
| |