lkml.org 
[lkml]   [2020]   [Feb]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [PATCH v7 0/4] support reserving crashkernel above 4G on arm64 kdump
From
Date


> On Feb 12, 2020, at 7:20 AM, John Donnelly <John.P.Donnelly@Oracle.com> wrote:
>
> On 12/23/19 9:23 AM, Chen Zhou wrote:
>> This patch series enable reserving crashkernel above 4G in arm64.
>> There are following issues in arm64 kdump:
>> 1. We use crashkernel=X to reserve crashkernel below 4G, which will fail
>> when there is no enough low memory.
>> 2. Currently, crashkernel=Y@X can be used to reserve crashkernel above 4G,
>> in this case, if swiotlb or DMA buffers are required, crash dump kernel
>> will boot failure because there is no low memory available for allocation.
>> To solve these issues, introduce crashkernel=X,low to reserve specified
>> size low memory.
>> Crashkernel=X tries to reserve memory for the crash dump kernel under
>> 4G. If crashkernel=Y,low is specified simultaneously, reserve spcified
>> size low memory for crash kdump kernel devices firstly and then reserve
>> memory above 4G.
>
>
> Hi Chen,
>
>
> I've applied your V7 patches to 5.4.17 test kernel and I am still seeing
> failures when I use a kdump kernel .
>
>
> On the kernel boot I see:
>
> Reserving 250MB of low memory at 3618MB for crashkernel (System low RAM: 2029MB)
> crashkernel reserved: 0x00000008c0000000 - 0x0000000940000000 (2048 MB)
>
> # cat /proc/iomem | grep -i cras
> e2200000-f1bfffff : Crash kernel (low)
> 8c0000000-93fffffff : Crash kernel
>
>
> When kdump kernel is started I see what appears to be DMA initialized :
>
> NUMA: NODE_DATA(1) on node 0
> Zone ranges:
> DMA32 [mem 0x00000000802f0000-0x00000000ffffffff]
> Normal [mem 0x0000000100000000-0x000000093fffffff]
>
> But the sas driver still fails :
>
>
> [ 12.092769] CPU: 0 PID: 149 Comm: kworker/0:13 Not tainted 5.4.17-4-uek6m_ol8-jpdonnel+ #2
> [ 12.101019] Hardware name: To be filled by O.E.M. Saber/Saber, BIOS 0ACKL028 09/09/2019
> [ 12.109019] Workqueue: events work_for_cpu_fn
> [ 12.113363] Call trace:
> [ 12.115803] dump_backtrace+0x0/0x19c
> [ 12.119453] show_stack+0x24/0x2c
> [ 12.122769] dump_stack+0xcc/0xf8
> [ 12.126078] warn_alloc+0x108/0x11c
> [ 12.129554] __alloc_pages_slowpath+0x8fc/0xa10
> [ 12.134071] __alloc_pages_nodemask+0x2ec/0x334
> [ 12.138597] __dma_direct_alloc_pages+0x19c/0x238
> [ 12.143288] dma_direct_alloc_pages+0x48/0xf8
> [ 12.147632] dma_direct_alloc+0x4c/0x6c
> [ 12.151455] dma_alloc_attrs+0x88/0xf4
> [ 12.155196] dma_pool_alloc+0x11c/0x2ec
> [ 12.159053] _base_allocate_memory_pools+0x2ec/0x1078 [mpt3sas]
> [ 12.164978] mpt3sas_base_attach+0x444/0x748 [mpt3sas]
> [ 12.170121] _scsih_probe+0x554/0x848 [mpt3sas]
> [ 12.174648] local_pci_probe+0x4c/0x98
>
> And the kdump fails to find local storage:
>
>
> mpt3sas_cm0: reply_post_free pool: dma_pool_alloc failed
> mpt3sas_cm0: failure at ../drivers/scsi/mpt3sas/mpt3sas_scsih.c:10626/_scsih_probe()!

Hi Chen,


I was able to unit test these series of kernel patches applied to a 5.4.17 test kernel along with the kexec CLI change :

0001-arm64-kdump-add-another-DT-property-to-crash-dump-ke.patch

Applied to :

kexec-tools-2.0.19-12.0.4.el8.src.rpm

And obtained a vmcore using this cmdline :

BOOT_IMAGE=(hd6,gpt2)/vmlinuz-5.4.17-4-uek6m_ol8-jpdonnel+ root=/dev/mapper/ol01-root ro crashkernel=2048M@35G crashkernel=250M,low rd.lvm.lv=ol01/root rd.lvm.lv=ol01/swap console=ttyS4 loglevel=7

Can you add :

Tested-by: John Donnelly <John.p.donnelly@oracle.com>


How can we get these changes included into an rc kernel release ?

Thanks,

John.






>
>
>
>
>> When crashkernel is reserved above 4G in memory, that is, crashkernel=X,low
>> is specified simultaneously, kernel should reserve specified size low memory
>> for crash dump kernel devices. So there may be two crash kernel regions, one
>> is below 4G, the other is above 4G.
>> In order to distinct from the high region and make no effect to the use of
>> kexec-tools, rename the low region as "Crash kernel (low)", and add DT property
>> "linux,low-memory-range" to crash dump kernel's dtb to pass the low region.
>> Besides, we need to modify kexec-tools:
>> arm64: kdump: add another DT property to crash dump kernel's dtb(see [1])
>
>
> Can you explain what needs done to kexec tools in more detail ?
>
> I'd like to understand why the Arm kdump boot images are so large ( 1024M+ ) as compared to x86 that can take a vmcore using a 512M kdump image .
>
>
>
> ======= <clipped>=======


I was able to unit test these series of patches along with the kexec CLI change :

0001-arm64-kdump-add-another-DT-property-to-crash-dump-ke.patch

Applied to :

kexec-tools-2.0.19-12.0.4.el8.src.rpm


And was able to get a vmcore dump

>
>
> _______________________________________________
> kexec mailing list
> kexec@lists.infradead.org
> https://urldefense.com/v3/__http://lists.infradead.org/mailman/listinfo/kexec__;!!GqivPVa7Brio!PTJ8J3z7crIzNbPXZr99_vgRkany0mRuHvQqzUIK_4QoWqLEcdWLXfjsdyw3vIntYsG7$

\
 
 \ /
  Last update: 2020-02-24 16:19    [W:0.139 / U:0.788 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site