Messages in this thread | | | Date | Fri, 16 Aug 2019 12:32:08 -0700 | From | Guenter Roeck <> | Subject | Re: sched: Unexpected reschedule of offline CPU#2! |
| |
On Fri, Aug 16, 2019 at 12:22:22PM +0200, Thomas Gleixner wrote: > On Mon, 29 Jul 2019, Guenter Roeck wrote: > > On Mon, Jul 29, 2019 at 12:47:45PM +0200, Peter Zijlstra wrote: > > > On Mon, Jul 29, 2019 at 12:38:30PM +0200, Thomas Gleixner wrote: > > > > Reboot has two modes: > > > > > > > > - Regular reboot initiated from user space > > > > > > > > - Panic reboot > > > > > > > > For the regular reboot we can make it go through proper hotplug, > > > > > > That seems sensible. > > > > > > > for the panic case not so much. > > > > > > It's panic, shit has already hit fan, one or two more pieces shouldn't > > > something anybody cares about. > > > > > > > Some more digging shows that this happens a lot with Google GCE intances, > > typically after a panic. The problem with that, if I understand correctly, > > is that it may prevent coredumps from being written. So, while of course > > the panic is what needs to be fixed, it is still quite annoying, and it > > would help if this can be fixed for panic handling as well. > > > > How about the patch suggested by Hillf Danton ? Would that help for the > > panic case ? > > I have no idea how that patch looks like, but the quick hack is below. > > Thanks, > > tglx > > 8<--------------- > diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c > index 75fea0d48c0e..625627b1457c 100644 > --- a/arch/x86/kernel/process.c > +++ b/arch/x86/kernel/process.c > @@ -601,6 +601,7 @@ void stop_this_cpu(void *dummy) > /* > * Remove this CPU: > */ > + set_cpu_active(smp_processor_id(), false); > set_cpu_online(smp_processor_id(), false); > disable_local_APIC(); > mcheck_cpu_clear(this_cpu_ptr(&cpu_info)); > No luck. The problem is still seen with this patch applied on top of the mainline kernel (commit a69e90512d9def6).
Guenter
--- [ 22.315834] e1000e: EEE TX LPI TIMER: 00000000 [ 22.323624] reboot: Restarting system [ 22.324260] reboot: machine restart [ 22.325885] ------------[ cut here ]------------ [ 22.330425] sched: Unexpected reschedule of offline CPU#3! ILLOPC: ffffffffb524403f: 0f 0b [ 22.330926] WARNING: CPU: 1 PID: 0 at arch/x86/kernel/smp.c:126 native_smp_send_reschedule+0x2f/0x40 [ 22.331238] Modules linked in: [ 22.331427] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.3.0-rc4+ #1 [ 22.331626] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014 [ 22.331971] RIP: 0010:native_smp_send_reschedule+0x2f/0x40 [ 22.332164] Code: 05 de 81 95 01 73 15 48 8b 05 bd fa 61 01 be fd 00 00 00 48 8b 40 30 e9 6f d0 fb 00 89 fe 48 c7 c7 88 da 74 b6 e8 7f 6c 02 00 <0f> 0b c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 55 53 48 83 ec [ 22.332705] RSP: 0018:ffffa457800d0d68 EFLAGS: 00000086 [ 22.332884] RAX: 0000000000000000 RBX: ffff9a8cbb9ba000 RCX: 0000000000000103 [ 22.333109] RDX: 0000000080000103 RSI: 0000000000000000 RDI: 00000000ffffffff [ 22.333327] RBP: ffffa457800d0e90 R08: 0000000000000000 R09: 0000000000000000 [ 22.333546] R10: 0000000000000000 R11: ffffa457800d0c10 R12: 000000000000a1b9 [ 22.333767] R13: ffff9a8cbae26030 R14: ffff9a8cbae25f80 R15: ffff9a8cbb83a000 [ 22.334045] FS: 0000000000000000(0000) GS:ffff9a8cbb880000(0000) knlGS:0000000000000000 [ 22.334321] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 22.334520] CR2: 00007fba66a35010 CR3: 0000000176cd6000 CR4: 00000000007406e0 [ 22.334794] PKRU: 55555554 [ 22.334915] Call Trace: [ 22.335062] <IRQ> [ 22.335148] check_preempt_curr+0x7f/0xc0 [ 22.335295] load_balance+0x589/0xc50 [ 22.335513] rebalance_domains+0x30d/0x410 [ 22.335684] _nohz_idle_balance+0x1bd/0x200 [ 22.335854] __do_softirq+0xe5/0x478 [ 22.336023] irq_exit+0xa9/0xc0 [ 22.336163] reschedule_interrupt+0xf/0x20 [ 22.336317] </IRQ> [ 22.336409] RIP: 0010:default_idle+0x23/0x180 [ 22.336561] Code: ff 90 90 90 90 90 90 41 55 41 54 55 53 e8 45 75 7c ff 0f 1f 44 00 00 e8 0b aa 40 ff e9 07 00 00 00 0f 00 2d 31 94 4a 00 fb f4 <e8> 28 75 7c ff 89 c5 0f 1f 44 00 00 5b 5d 41 5c 41 5d c3 65 8b 05 [ 22.337102] RSP: 0018:ffffa4578006bec0 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff02 [ 22.337342] RAX: ffff9a8cbae23fc0 RBX: 0000000000000001 RCX: 0000000000000001 [ 22.337561] RDX: 0000000000000046 RSI: 0000000000000006 RDI: ffffffffb6852dd6 [ 22.337780] RBP: ffffffffb6b9c1f8 R08: 0000000000000001 R09: 0000000000000000 [ 22.337996] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 [ 22.338229] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 [ 22.338501] do_idle+0x1df/0x260 [ 22.338588] ? _raw_spin_unlock_irqrestore+0x4c/0x60 [ 22.338706] cpu_startup_entry+0x14/0x20 [ 22.338793] start_secondary+0x151/0x180 [ 22.338885] secondary_startup_64+0xa4/0xb0 [ 22.339060] irq event stamp: 61631 [ 22.339176] hardirqs last enabled at (61630): [<ffffffffb5f5c6dc>] _raw_spin_unlock_irqrestore+0x4c/0x60 [ 22.339373] hardirqs last disabled at (61631): [<ffffffffb5f5c46d>] _raw_spin_lock_irqsave+0xd/0x50 [ 22.339568] softirqs last enabled at (61626): [<ffffffffb5272bc8>] irq_enter+0x58/0x60 [ 22.339726] softirqs last disabled at (61627): [<ffffffffb5272c79>] irq_exit+0xa9/0xc0 [ 22.339897] ---[ end trace 8ad53445879058cc ]--- [ 22.340384] ACPI MEMORY or I/O RESET_REG.
| |