lkml.org 
[lkml]   [2016]   [Apr]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [BUG linux-next] Kernel panic found with linux-next-20160414
From
Date
On 4/27/2016 1:14 AM, Hugh Dickins wrote:
> On Wed, 20 Apr 2016, Shi, Yang wrote:
>> On 4/20/2016 1:01 AM, Hugh Dickins wrote:
>>> On Tue, 19 Apr 2016, Shi, Yang wrote:
>>>> Hi folks,
>>>>
>>>> When I ran ltp on linux-next-20160414 on my ARM64 machine, I got the
>>>> below
>>>> kernel panic:
>>>>
>>>> Unable to handle kernel paging request at virtual address
>>>> ffffffc007846000
>>>> pgd = ffffffc01e21d000
>>>> [ffffffc007846000] *pgd=0000000000000000, *pud=0000000000000000
>>>> Internal error: Oops: 96000047 [#11] PREEMPT SMP
>>>> Modules linked in: loop
>>>> CPU: 7 PID: 274 Comm: systemd-journal Tainted: G D
>>>> 4.6.0-rc3-next-20160414-WR8.0.0.0_standard+ #9
>>>> Hardware name: Freescale Layerscape 2085a RDB Board (DT)
>>>> task: ffffffc01e3fcf80 ti: ffffffc01ea8c000 task.ti: ffffffc01ea8c000
>>>> PC is at copy_page+0x38/0x120
>>>> LR is at migrate_page_copy+0x604/0x1660
>>>> pc : [<ffffff9008ff2318>] lr : [<ffffff900867cdac>] pstate: 20000145
>>>> sp : ffffffc01ea8ecd0
>>>> x29: ffffffc01ea8ecd0 x28: 0000000000000000
>>>> x27: 1ffffff7b80240f8 x26: ffffffc018196f20
>>>> x25: ffffffbdc01e1180 x24: ffffffbdc01e1180
>>>> x23: 0000000000000000 x22: ffffffc01e3fcf80
>>>> x21: ffffffc00481f000 x20: ffffff900a31d000
>>>> x19: ffffffbdc01207c0 x18: 0000000000000f00
>>>> x17: 0000000000000000 x16: 0000000000000000
>>>> x15: 0000000000000000 x14: 0000000000000000
>>>> x13: 0000000000000000 x12: 0000000000000000
>>>> x11: 0000000000000000 x10: 0000000000000000
>>>> x9 : 0000000000000000 x8 : 0000000000000000
>>>> x7 : 0000000000000000 x6 : 0000000000000000
>>>> x5 : 0000000000000000 x4 : 0000000000000000
>>>> x3 : 0000000000000000 x2 : 0000000000000000
>>>> x1 : ffffffc00481f080 x0 : ffffffc007846000
>>>>
>>>> Call trace:
>>>> Exception stack(0xffffffc021fc2ed0 to 0xffffffc021fc2ff0)
>>>> 2ec0: ffffffbdc00887c0 ffffff900a31d000
>>>> 2ee0: ffffffc021fc30f0 ffffff9008ff2318 0000000020000145 0000000000000025
>>>> 2f00: ffffffbdc025a280 ffffffc020adc4c0 0000000041b58ab3 ffffff900a085fd0
>>>> 2f20: ffffff9008200658 0000000000000000 0000000000000000 ffffffbdc00887c0
>>>> 2f40: ffffff900b0f1320 ffffffc021fc3078 0000000041b58ab3 ffffff900a0864f8
>>>> 2f60: ffffff9008210010 ffffffc021fb8960 ffffff900867bacc 1ffffff8043f712d
>>>> 2f80: ffffffc021fc2fb0 ffffff9008210564 ffffffc021fc3070 ffffffc021fb8940
>>>> 2fa0: 0000000008221f78 ffffff900862f9c8 ffffffc021fc2fe0 ffffff9008215dc8
>>>> 2fc0: 1ffffff8043f8602 ffffffc021fc0000 ffffffc00968a000 ffffffc00221f080
>>>> 2fe0: f9407e11d00001f0 d61f02209103e210
>>>> [<ffffff9008ff2318>] copy_page+0x38/0x120
>>>> [<ffffff900867de7c>] migrate_page+0x74/0x98
>>>> [<ffffff90089ba418>] nfs_migrate_page+0x58/0x80
>>>> [<ffffff900867dffc>] move_to_new_page+0x15c/0x4d8
>>>> [<ffffff900867eec8>] migrate_pages+0x7c8/0x11f0
>>>> [<ffffff90085f8724>] compact_zone+0xdfc/0x2570
>>>> [<ffffff90085f9f78>] compact_zone_order+0xe0/0x170
>>>> [<ffffff90085fb688>] try_to_compact_pages+0x2e8/0x8f8
>>>> [<ffffff90085913a0>] __alloc_pages_direct_compact+0x100/0x540
>>>> [<ffffff9008592420>] __alloc_pages_nodemask+0xc40/0x1c58
>>>> [<ffffff90086887e8>] khugepaged+0x468/0x19c8
>>>> [<ffffff9008301700>] kthread+0x248/0x2c0
>>>> [<ffffff9008206610>] ret_from_fork+0x10/0x40
>>>> Code: d281f012 91020021 f1020252 d503201f (a8000c02)
>>>>
>>>>
>>>> I did some initial investigation and found it is caused by
>>>> DEBUG_PAGEALLOC
>>>> and CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT. And, mainline 4.6-rc3 works
>>>> well.
>>>>
>>>> It should be not arch specific although I got it caught on ARM64. I
>>>> suspect
>>>> this might be caused by Hugh's huge tmpfs patches.
>>>
>>> Thanks for testing. It might be caused by my patches, but I don't think
>>> that's very likely. This is page migraton for compaction, in the service
>>> of anon THP's khugepaged; and I wonder if you were even exercising huge
>>> tmpfs when running LTP here (it certainly can be done: I like to mount a
>>> huge tmpfs on /opt/ltp and install there, with shmem_huge 2 so any other
>>> tmpfs mounts are also huge).
>>
>> Some further investigation shows I got the panic even though I don't have
>> tmpfs mounted with huge=1 or set shmem_huge to 2.
>>
>>>
>>> There are compaction changes in linux-next too, but I don't see any
>>> reason why they'd cause this. I don't know arm64 traces enough to know
>>> whether it's the source page or the destination page for the copy, but
>>> it looks as if it has been freed (and DEBUG_PAGEALLOC unmapped) before
>>> reaching migration's copy.
>>
>> The fault address is passed by x0, which is dest in the implementation of
>> copy_page, so it is the destination page.
>>
>>>
>>> Needs more debugging, I'm afraid: is it reproducible?
>>
>> Yes, as long as I enable those two PAGEALLOC debug options, I can get the
>> panic once I run ltp. But, it is not caused any specific ltp test case
>> directly, the panic happens randomly during ltp is running.
>
> Your ping on the crash in release_freepages() reminded me to take another
> look at this one. And found that I only needed to enable DEBUG_PAGEALLOC
> and run LTP to get it on x86_64 too, as you suspected.
>
> It's another of those compaction errors, in mmotm and linux-next of a
> week or two ago, whose patch has since been withdrawn (but huge tmpfs
> has also been withdrawn for now, so you're right to stick with the
> older linux-next for testing it).

Yes, I saw the discussion on LSFMM 2016 and the patches have gone in my
latest update from linux-next. I will stick to 20160420 for the huge
tmpfs testing.

>
> I believe the patch below fixes it; but I've not done full diligence
> on it - if I had more time, I'd want to check that all of the things
> that need doing are now being done on this path, and that it's also
> okay if the release undoes them even when they didn't get to be done.
> But not worth that diligence if the patch is withdrawn already.
>
> It's rather horrible that compaction.c uses functions in page_alloc.c
> which skip doing some of the things we expect to be done: the non-debug
> preparation tends to get noticed, but the debug options overlooked.
> We can expect more problems of this kind in future: someone will add
> yet another debug prep line in page_alloc.c, and at first nobody will
> notice that it's also needed in compaction.c.
>
> I am hopeful, since the missed map_pages() does KASAN initialization too,
> that this might also fix your KASAN use-after-free in nfs_do_filldir(),
> which you also reported on April 20th.
>
> But with this patch in, I do get a more interesting crash in
> remap_team_by_ptes() from LTP's mmapstress10: there appears to be an
> anon THP in a huge tmpfs vma. Maybe I've got the test at the head of
> __split_huge_pmd() wrong, but I don't recall seeing this before
> rebuilding with DEBUG_PAGEALLOC. Can't spend longer on it now,
> will return to it tomorrow.

Thanks for the patch and the patch for another problem.

Regards,
Yang

>
> Hugh
> ---
> mm/compaction.c | 1 +
> 1 file changed, 1 insertion(+)
>
> --- 4.6-rc2-mm1/mm/compaction.c 2016-04-11 11:35:08.000000000 -0700
> +++ linux/mm/compaction.c 2016-04-26 22:15:10.954455303 -0700
> @@ -1113,6 +1113,7 @@ static void isolate_freepages_direct(str
> }
>
> spin_unlock_irqrestore(&cc->zone->lock, flags);
> + map_pages(&cc->freepages);
> }
>
> /*
>

\
 
 \ /
  Last update: 2016-04-28 20:41    [W:0.082 / U:0.180 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site