lkml.org 
[lkml]   [2016]   [Aug]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: kernel BUG at net/unix/garbage.c:149!"
From
Date


On 08/30/2016 12:18 PM, Miklos Szeredi wrote:
> On Tue, Aug 30, 2016 at 12:37 AM, Miklos Szeredi <mszeredi@redhat.com> wrote:
>> On Sat, Aug 27, 2016 at 11:55 AM, Miklos Szeredi <mszeredi@redhat.com> wrote:
>
>> crash> list -H gc_inflight_list unix_sock.link -s unix_sock.inflight |
>> grep counter | cut -d= -f2 | awk '{s+=$1} END {print s}'
>> 130
>> crash> p unix_tot_inflight
>> unix_tot_inflight = $2 = 135
>>
>> We've lost track of a total of five inflight sockets, so it's not a
>> one-off thing. Really weird... Now off to sleep, maybe I'll dream of
>> the solution.
>
> Okay, found one bug: gc assumes that in-flight sockets that don't have
> an external ref can't gain one while unix_gc_lock is held. That is
> true because unix_notinflight() will be called before detaching fds,
> which takes unix_gc_lock. Only MSG_PEEK was somehow overlooked. That
> one also clones the fds, also keeping them in the skb. But through
> MSG_PEEK an external reference can definitely be gained without ever
> touching unix_gc_lock.
>
> Not sure whether the reported bug can be explained by this. Can you
> confirm the MSG_PEEK was used in the setup?
>
> Does someone want to write a stress test for SCM_RIGHTS + MSG_PEEK?
>
> Anyway, attaching a fix that works by acquiring unix_gc_lock in case
> of MSG_PEEK also. It is trivially correct, but I haven't tested it.

I have no way of being 100% sure but looking through nginx's source code
it seems they do utilize MSG_PEEK on several occasions. This issue has
been apparently very hard to reproduce since I have 100s of servers
running a lot of NGINX processes and this has been triggered only once.

On a different note - if I inspect a live node without this patch should
the discrepancy between the gc_inflight_list and the unix_tot_inflight
be present VS with this patch applied?

>
> Thanks,
> Miklos
>

\
 
 \ /
  Last update: 2016-09-17 09:58    [W:0.072 / U:1.508 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site