lkml.org 
[lkml]   [2014]   [Dec]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: frequent lockups in 3.18rc4


On Mon, Dec 1, 2014 at 6:25 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Mon, Dec 1, 2014 at 3:08 PM, Chris Mason <clm@fb.com> wrote:
>> I'm not sure if this is related, but running trinity here, I
>> noticed it
>> was stuck at 100% system time on every CPU. perf report tells me
>> we are
>> spending all of our time in spin_lock under the sync system call.
>>
>> I think it's coming from contention in the bdi_queue_work() call
>> from
>> inside sync_inodes_sb, which is spin_lock_bh().
>
> Please do a perf run with -g to get the call chain to make sure..

The call chain goes something like this:

--- _raw_spin_lock
|
|--99.72%-- sync_inodes_sb
| sync_inodes_one_sb
| iterate_supers
| sys_sync
| |
| |--79.66%-- system_call_fastpath
| | syscall
| |
| --20.34%-- ia32_sysret
| __do_syscall
--0.28%-- [...]

(the 64bit call variation is similar) Adding -v doesn't really help,
because it isn't giving me the address inside sync_inodes_sb()

I first read this and guessed it must be leaving out the call to
bdi_queue_work, hoping the spin_lock_bh and lock debugging were teaming
up to stall the box.

But looking harder it's probably inside wait_sb_inodes:

spin_lock(&inode_sb_list_lock);

Which is a little harder to blame. Maaaaaybe with lock debugging, but
its enough of a stretch that I wouldn't have emailed at all if I hadn't
fixated on the bdi code.

-chris





\
 
 \ /
  Last update: 2014-12-02 01:21    [W:0.395 / U:0.888 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site