Hi everyone.

We areexperiencing  100% reproducable kernel lockups (4 servers crashes 2-3 times a day each) on a highload DNS servers (30-40kpps udp requests).
Hardware is pretty different, I can provide lspci if needed. We are using Intel ethernet adapters with Jumbo Frames enabled and we are using no swap at all. 

e1000e driver:
03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
08:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06)

igb driver:
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)

All hardware affected. 


My system is Ubuntu-12.04 Server LTS. But this problem occures on many kernel versions:
linux-3.2 ubuntu lts
linux-3.5 quantal ubuntu lts
linux-3.4.* kernel.org LTS
linux-3.8.* kernel.org LTS

I've attached crash I've catched with ipmitool. If there is anything I can do to help to debug this problem, please, let me know!
Thanks.
[110804.419268] INFO: rcu_sched self-detected stall on CPU
[110804.424588] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[110804.430547] 	 (t=15000 jiffies)
[110804.434541] INFO: rcu_sched detected stalls on CPUs/tasks:
[110804.434544] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[110804.434545] 	(detected by 1, t=15003 jiffies)
[110804.434547] INFO: Stall ended before state dump start
[110805.078303] INFO: rcu_bh detected stalls on CPUs/tasks:
[110805.083710] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[110805.089671] 	(detected by 6, t=15005 jiffies)
[110805.094207] INFO: Stall ended before state dump start
[110950.191379] INFO: task md1_raid10:361 blocked for more than 120 seconds.
[110950.198225] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[110950.206394] INFO: task jbd2/md1-8:377 blocked for more than 120 seconds.
[110950.213220] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[110950.221457] INFO: task debian-sa1:23250 blocked for more than 120 seconds.
[110950.228495] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[110950.236650] INFO: task date:23252 blocked for more than 120 seconds.
[110950.243135] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[110950.251230] INFO: task ifconfig:23253 blocked for more than 120 seconds.
[110950.258152] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[110950.266291] INFO: task job_converter.p:23257 blocked for more than 120 seconds.
[110950.273778] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[110984.167042] INFO: rcu_sched self-detected stall on CPU
[110984.172386] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[110984.178372] 	 (t=60006 jiffies)
[110984.182481] INFO: rcu_sched detected stalls on CPUs/tasks:
[110984.182489] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[110984.182491] 	(detected by 6, t=60009 jiffies)
[110984.182495] INFO: Stall ended before state dump start
[110984.822148] INFO: rcu_bh detected stalls on CPUs/tasks:
[110984.827589] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[110984.833576] 	(detected by 7, t=60010 jiffies)
[110984.838134] INFO: Stall ended before state dump start
[111070.095039] INFO: task md1_raid10:361 blocked for more than 120 seconds.
[111070.101925] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[111070.110252] INFO: task jbd2/md1-8:377 blocked for more than 120 seconds.
[111070.117156] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[111070.125637] INFO: task rs:main Q:Reg:1052 blocked for more than 120 seconds.
[111070.132937] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[111070.141070] INFO: task flush-9:1:4108 blocked for more than 120 seconds.
[111070.147952] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[111163.914422] INFO: rcu_sched self-detected stall on CPU
[111163.919767] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[111163.925753] 	 (t=105012 jiffies)
[111163.929853] INFO: rcu_sched detected stalls on CPUs/tasks: linear
[111163.936177] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[111163.942160] 	(detected by 1, t=105019 jiffies)
[111163.946806] INFO: Stall ended before state dump start
[111164.565547] INFO: rcu_bh detected stalls on CPUs/tasks:
[111164.570981] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[111164.576968] 	(detected by 6, t=105015 jiffies)
[111164.581615] INFO: Stall ended before state dump start
[111343.662005] INFO: rcu_sched self-detected stall on CPU
[111343.667349] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[111343.673334] 	 (t=150018 jiffies)
[111343.677418] INFO: rcu_sched detected stalls on CPUs/tasks: async_tx raid1 raid0 multipath linear
[111343.686460] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[111343.692442] 	(detected by 1, t=150025 jiffies)
[111343.697089] INFO: Stall ended before state dump start
[111344.309009] INFO: rcu_bh detected stalls on CPUs/tasks:
[111344.314444] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[111344.320431] 	(detected by 2, t=150020 jiffies)
[111344.325077] INFO: Stall ended before state dump start
[111523.409590] INFO: rcu_sched self-detected stall on CPU
[111523.414938] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[111523.420924] 	 (t=195024 jiffies)
[111523.425297] INFO: rcu_sched detected stalls on CPUs/tasks:
[111523.430966] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[111523.436950] 	(detected by 2, t=195031 jiffies)
[111523.441598] INFO: Stall ended before state dump start
[111524.052678] INFO: rcu_bh detected stalls on CPUs/tasks:
[111524.058114] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[111524.064101] 	(detected by 6, t=195025 jiffies)
[111524.068746] INFO: Stall ended before state dump start
[111703.157255] INFO: rcu_sched self-detected stall on CPU
[111703.162597] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[111703.168583] 	 (t=240030 jiffies)
[111703.172947] INFO: rcu_sched detected stalls on CPUs/tasks:
[111703.172950] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[111703.172952] 	(detected by 2, t=240033 jiffies)
[111703.172954] INFO: Stall ended before state dump start
[111703.796252] INFO: rcu_bh detected stalls on CPUs/tasks:
[111703.801687] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[111703.807674] 	(detected by 2, t=240030 jiffies)
[111703.812321] INFO: Stall ended before state dump start
[111882.904766] INFO: rcu_sched self-detected stall on CPU
[111882.910107] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[111882.916086] 	 (t=285036 jiffies)
[111882.920173] INFO: rcu_sched detected stalls on CPUs/tasks: multipath linear
[111882.927367] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[111882.933351] 	(detected by 1, t=285043 jiffies)
[111882.937996] INFO: Stall ended before state dump start
[111883.539885] INFO: rcu_bh detected stalls on CPUs/tasks:
[111883.545321] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[111883.551310] 	(detected by 6, t=285035 jiffies)
[111883.555955] INFO: Stall ended before state dump start
[112062.652392] INFO: rcu_sched self-detected stall on CPU
[112062.657742] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[112062.663727] 	 (t=330042 jiffies)
[112062.667856] INFO: rcu_sched detected stalls on CPUs/tasks:
[112062.667859] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[112062.667861] 	(detected by 1, t=330045 jiffies)
[112062.667862] INFO: Stall ended before state dump start
[112063.283472] INFO: rcu_bh detected stalls on CPUs/tasks:
[112063.288906] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[112063.294894] 	(detected by 6, t=330040 jiffies)
[112063.299538] INFO: Stall ended before state dump start
[112242.399956] INFO: rcu_sched self-detected stall on CPU
[112242.405305] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[112242.411290] 	 (t=375048 jiffies)
[112242.415393] INFO: rcu_sched detected stalls on CPUs/tasks: multipath linear
[112242.422593] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[112242.428574] 	(detected by 1, t=375055 jiffies)
[112242.433220] INFO: Stall ended before state dump start
[112243.027095] INFO: rcu_bh detected stalls on CPUs/tasks:
[112243.032533] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[112243.038519] 	(detected by 7, t=375045 jiffies)
[112243.043165] INFO: Stall ended before state dump start
[112422.147588] INFO: rcu_sched self-detected stall on CPU
[112422.152930] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[112422.158915] 	 (t=420054 jiffies)
[112422.163011] INFO: rcu_sched detected stalls on CPUs/tasks: raid0 multipath linear
[112422.170758] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[112422.176742] 	(detected by 1, t=420061 jiffies)
[112422.181389] INFO: Stall ended before state dump start
[112422.770722] INFO: rcu_bh detected stalls on CPUs/tasks:
[112422.776158] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[112422.782146] 	(detected by 6, t=420050 jiffies)
[112422.786792] INFO: Stall ended before state dump start
[112601.895159] INFO: rcu_sched self-detected stall on CPU
[112601.900502] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[112601.906488] 	 (t=465060 jiffies)
[112601.910589] INFO: rcu_sched detected stalls on CPUs/tasks:
[112601.910592] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[112601.910594] 	(detected by 1, t=465063 jiffies)
[112601.910595] INFO: Stall ended before state dump start
[112602.514229] INFO: rcu_bh detected stalls on CPUs/tasks:
[112602.519663] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[112602.525650] 	(detected by 6, t=465055 jiffies)
[112602.530298] INFO: Stall ended before state dump start
[112781.642947] INFO: rcu_sched self-detected stall on CPU
[112781.648290] 	0: (1 GPs behind) idle=ccf/2/0 drain=0 . timer=-1
[112781.654278] 	 (t=510066 jiffies)
[112781.658643] INFO: rcu_sched detected stalls on CPUs/tasks: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112781.687911] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[112781.693894] 	(detected by 2, t=510079 jiffies)
[112781.698539] INFO: Stall ended before state dump start
[112782.257865] INFO: rcu_bh detected stalls on CPUs/tasks:
[112782.263300] 	0: (1 GPs behind) idle=ccf/1/0 drain=0 . timer=-1
[112782.269287] 	(detected by 1, t=510060 jiffies)
[112782.273933] INFO: Stall ended before state dump start
[112899.146651] SysRq : Show backtrace of all active CPUs
[112912.006981] SysRq : Trigger a crash
[112912.010673] BUG: unable to handle kernel NULL pointer dereference at           (null)
[112912.018789] IP: [<ffffffff8132c3f1>] sysrq_handle_crash+0x11/0x20
[112912.025087] PGD 232ebc067 PUD 232083067 PMD 0 
[112912.029840] Oops: 0002 [#1] SMP 
[112912.033318] CPU 0 
[112912.035281] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112912.063864] 
[112912.065508] Pid: 0, comm: swapper/0 Not tainted 3.4.39-21 #8 System manufacturer System Product Name/Z8NR-D12
[112912.075755] RIP: 0010:[<ffffffff8132c3f1>]  [<ffffffff8132c3f1>] sysrq_handle_crash+0x11/0x20
[112912.084512] RSP: 0018:ffff880237c03360  EFLAGS: 00010086
[112912.089972] RAX: 0000000000000010 RBX: 0000000000000063 RCX: 0000000000002928
[112912.097284] RDX: 0000000000000000 RSI: 0000000000000086 RDI: 0000000000000063
[112912.104597] RBP: ffff880237c03360 R08: 0000000000000000 R09: 0000000000000080
[112912.111909] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff818411c0
[112912.119219] R13: 0000000000000086 R14: 0000000000000004 R15: 0000000000000001
[112912.126533] FS:  0000000000000000(0000) GS:ffff880237c00000(0000) knlGS:0000000000000000
[112912.134797] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[112912.140689] CR2: 0000000000000000 CR3: 0000000233bcb000 CR4: 00000000000007f0
[112912.148928] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[112912.156241] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[112912.163551] Process swapper/0 (pid: 0, threadinfo ffffffff81800000, task ffffffff81813020)
[112912.171988] Stack:
[112912.174153]  ffff880237c033a0 ffffffff8132ca99 ffffea0008671700 ffff880231cb59a0
[112912.181931]  0000000000000001 0000000000000001 0000000000000001 000000000000002e
[112912.189721]  ffff880237c033e0 ffffffff8132cbe6 ffff880200000001 ffffffff0000002e
[112912.197499] Call Trace:
[112912.200096]  <IRQ> 
[112912.202398]  [<ffffffff8132ca99>] __handle_sysrq+0x119/0x180
[112912.208204]  [<ffffffff8132cbe6>] sysrq_filter+0x96/0x1a0
[112912.213752]  [<ffffffff81402c76>] input_pass_event+0x76/0xd0
[112912.219568]  [<ffffffff8140433f>] input_handle_event+0x8f/0x4a0
[112912.225642]  [<ffffffff81404884>] input_event+0x84/0xa0
[112912.231017]  [<ffffffffa00dc39d>] hidinput_hid_event+0x27d/0x3e0 [hid]
[112912.237697]  [<ffffffffa00d7292>] hid_process_event+0x112/0x140 [hid]
[112912.244291]  [<ffffffffa00d7a9d>] hid_report_raw_event+0x36d/0x3e0 [hid]
[112912.251143]  [<ffffffffa00d7cf6>] hid_input_report+0x1e6/0x2b0 [hid]
[112912.257651]  [<ffffffffa0113105>] hid_irq_in+0x1a5/0x1f0 [usbhid]
[112912.263899]  [<ffffffff813cdd8f>] usb_hcd_giveback_urb+0x5f/0xe0
[112912.270059]  [<ffffffff813e1968>] ehci_urb_done+0xd8/0x120
[112912.275694]  [<ffffffff813e1d29>] qh_completions+0x379/0x460
[112912.281507]  [<ffffffff813e2bde>] ehci_work+0x35e/0xbb0
[112912.286881]  [<ffffffff813e3c52>] ehci_irq+0x202/0x260
[112912.292169]  [<ffffffff810682b4>] ? scheduler_tick+0xf4/0x130
[112912.298070]  [<ffffffff813cd397>] usb_hcd_irq+0x37/0x60
[112912.303445]  [<ffffffff810a7bd3>] handle_irq_event_percpu+0x43/0x160
[112912.309951]  [<ffffffff810a7d33>] handle_irq_event+0x43/0x70
[112912.315758]  [<ffffffff810aac55>] handle_fasteoi_irq+0x55/0xf0
[112912.321737]  [<ffffffff8100c19d>] handle_irq+0x1d/0x30
[112912.327025]  [<ffffffff8100be05>] do_IRQ+0x55/0xd0
[112912.331967]  [<ffffffff815632aa>] common_interrupt+0x6a/0x6a
[112912.337781]  [<ffffffff814bdbc4>] ? udp4_lib_lookup2.isra.34+0x54/0x250
[112912.344548]  [<ffffffff814bdfc7>] __udp4_lib_lookup+0x207/0x310
[112912.350622]  [<ffffffff814bed24>] __udp4_lib_rcv+0x1b4/0x6c0
[112912.356431]  [<ffffffff81491426>] ? ip_route_input_common+0xcd6/0xe70
[112912.363022]  [<ffffffff814bf245>] udp_rcv+0x15/0x20
[112912.368050]  [<ffffffff814935b5>] ip_local_deliver_finish+0xd5/0x270
[112912.374557]  [<ffffffff814938d5>] ip_local_deliver+0x45/0x80
[112912.380364]  [<ffffffff81493285>] ip_rcv_finish+0x105/0x360
[112912.386085]  [<ffffffff81493b01>] ip_rcv+0x1f1/0x2f0
[112912.391199]  [<ffffffff81102505>] ? init_object+0x45/0x80
[112912.396746]  [<ffffffff81463326>] __netif_receive_skb+0x506/0x5c0
[112912.402993]  [<ffffffff814633fe>] netif_receive_skb+0x1e/0x80
[112912.408886]  [<ffffffff81463831>] ? dev_gro_receive+0x1b1/0x2b0
[112912.414961]  [<ffffffff81463588>] napi_skb_finish+0x48/0x60
[112912.420680]  [<ffffffff81464735>] napi_gro_receive+0xf5/0x140
[112912.426575]  [<ffffffffa0059f16>] e1000_receive_skb+0x56/0x70 [e1000e]
[112912.433255]  [<ffffffffa005d629>] e1000_clean_rx_irq_ps+0x439/0x720 [e1000e]
[112912.440455]  [<ffffffffa005b32d>] e1000_clean+0x6d/0x2b0 [e1000e]
[112912.446703]  [<ffffffff81463c89>] net_rx_action+0x139/0x1d0
[112912.452430]  [<ffffffff81045da8>] __do_softirq+0x98/0x120
[112912.457976]  [<ffffffff81562f39>] ? _raw_spin_lock+0x9/0x10
[112912.463699]  [<ffffffff81564e9c>] call_softirq+0x1c/0x30
[112912.469157]  [<ffffffff8100c215>] do_softirq+0x65/0xa0
[112912.474444]  [<ffffffff810460ae>] irq_exit+0x8e/0xb0
[112912.479558]  [<ffffffff8100be0e>] do_IRQ+0x5e/0xd0
[112912.484501]  [<ffffffff815632aa>] common_interrupt+0x6a/0x6a
[112912.490313]  <EOI> 
[112912.492616]  [<ffffffff812d8165>] ? intel_idle+0xe5/0x150
[112912.498164]  [<ffffffff812d8147>] ? intel_idle+0xc7/0x150
[112912.503711]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112912.509256]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112912.515237]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112912.520438]  [<ffffffff8153a179>] rest_init+0x6d/0x74
[112912.525638]  [<ffffffff818a0c18>] start_kernel+0x3c0/0x3cd
[112912.531271]  [<ffffffff818a0658>] ? repair_env_string+0x5a/0x5a
[112912.537338]  [<ffffffff818a0346>] x86_64_start_reservations+0x131/0x135
[112912.544105]  [<ffffffff818a0140>] ? early_idt_handlers+0x140/0x140
[112912.550438]  [<ffffffff818a044c>] x86_64_start_kernel+0x102/0x111
[112912.556687] Code: 8a 45 01 f4 45 39 65 54 75 e5 4c 89 ef e8 28 f7 ff ff eb db 90 90 90 90 90 90 55 c7 05 c1 25 63 00 01 00 00 00 48 89 e5 0f ae f8 <c6> 04 25 00 00 00 00 01 5d c3 0f 1f 44 00 00 55 31 c0 48 89 e5 
[112912.579063] RIP  [<ffffffff8132c3f1>] sysrq_handle_crash+0x11/0x20
[112912.585450]  RSP <ffff880237c03360>
[112912.589085] CR2: 0000000000000000
[112912.592555] ---[ end trace 6552bded36af03a9 ]---
[112912.597322] Kernel panic - not syncing: Fatal exception in interrupt
[112912.603911] ------------[ cut here ]------------
[112912.608765] WARNING: at net/sched/sch_generic.c:256 dev_watchdog+0x25f/0x270()
[112912.616268] Hardware name: System Product Name
[112912.620939] NETDEV WATCHDOG: eth2 (e1000e): transmit queue 0 timed out
[112912.627723] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112912.656807] Pid: 7, comm: watchdog/0 Tainted: G      D      3.4.39-21 #8
[112912.663761] Call Trace:
[112912.666469]  [<ffffffff8103f8da>] warn_slowpath_common+0x7a/0xb0
[112912.672726]  [<ffffffff8103f9b1>] warn_slowpath_fmt+0x41/0x50
[112912.678700]  [<ffffffff8128aaad>] ? do_raw_spin_unlock+0x5d/0xb0
[112912.684971]  [<ffffffff8147e6cf>] dev_watchdog+0x25f/0x270
[112912.690715]  [<ffffffff8104b56f>] run_timer_softirq+0xff/0x230
[112912.696798]  [<ffffffff81011819>] ? read_tsc+0x9/0x20
[112912.702106]  [<ffffffff8147e470>] ? pfifo_fast_init+0x90/0x90
[112912.708106]  [<ffffffff810276e8>] ? lapic_next_event+0x18/0x20
[112912.714198]  [<ffffffff81045da8>] __do_softirq+0x98/0x120
[112912.719861]  [<ffffffff8108171f>] ? tick_program_event+0x1f/0x30
[112912.726119]  [<ffffffff81564e9c>] call_softirq+0x1c/0x30
[112912.731684]  [<ffffffff8100c215>] do_softirq+0x65/0xa0
[112912.737070]  [<ffffffff810460ae>] irq_exit+0x8e/0xb0
[112912.742294]  [<ffffffff81027b79>] smp_apic_timer_interrupt+0x69/0xa0
[112912.748908]  [<ffffffff8156454a>] apic_timer_interrupt+0x6a/0x70
[112912.755171]  [<ffffffff81562eed>] ? _raw_spin_unlock_irq+0xd/0x20
[112912.761522]  [<ffffffff810647a7>] finish_task_switch+0x57/0xe0
[112912.767615]  [<ffffffff8100d9fb>] ? register_nmi_handler+0x4b/0x180
[112912.774140]  [<ffffffff81561cbb>] __schedule+0x3ab/0x720
[112912.779711]  [<ffffffff810a6e60>] ? watchdog_enable+0x240/0x240
[112912.785891]  [<ffffffff815622f4>] schedule+0x24/0x70
[112912.791112]  [<ffffffff810a6eda>] watchdog+0x7a/0xd0
[112912.796333]  [<ffffffff8105bd2e>] kthread+0x8e/0xa0
[112912.801467]  [<ffffffff81564da4>] kernel_thread_helper+0x4/0x10
[112912.807644]  [<ffffffff8105bca0>] ? flush_kthread_work+0x150/0x150
[112912.814086]  [<ffffffff81564da0>] ? gs_change+0x13/0x13
[112912.819567] ---[ end trace 6552bded36af03aa ]---
[112912.824447] Clocksource tsc unstable (delta = 1000253115943 ns)
[112935.209819] ------------[ cut here ]------------
[112935.214592] WARNING: at kernel/watchdog.c:241 watchdog_overflow_callback+0x9a/0xc0()
[112935.222508] Hardware name: System Product Name
[112935.227104] Watchdog detected hard LOCKUP on cpu 0
[112935.231844] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112935.260436] Pid: 7, comm: watchdog/0 Tainted: G      D W    3.4.39-21 #8
[112935.267291] Call Trace:
[112935.269886]  <NMI>  [<ffffffff8103f8da>] warn_slowpath_common+0x7a/0xb0
[112935.276706]  [<ffffffff8103f9b1>] warn_slowpath_fmt+0x41/0x50
[112935.282607]  [<ffffffff810a70ca>] watchdog_overflow_callback+0x9a/0xc0
[112935.289289]  [<ffffffff810ba2bc>] __perf_event_overflow+0x9c/0x220
[112935.295622]  [<ffffffff8101a38a>] ? x86_perf_event_set_period+0xda/0x150
[112935.302476]  [<ffffffff810babb4>] perf_event_overflow+0x14/0x20
[112935.308551]  [<ffffffff8101f3fc>] intel_pmu_handle_irq+0x17c/0x2f0
[112935.314885]  [<ffffffff81305ec0>] ? ghes_read_estatus+0x90/0x180
[112935.321047]  [<ffffffff8128aaad>] ? do_raw_spin_unlock+0x5d/0xb0
[112935.327208]  [<ffffffff81019141>] perf_event_nmi_handler+0x21/0x30
[112935.333542]  [<ffffffff8100dc4a>] do_nmi+0x11a/0x3a0
[112935.338656]  [<ffffffff815638bc>] end_repeat_nmi+0x1a/0x1e
[112935.344289]  [<ffffffff81012496>] ? native_read_tsc+0x6/0x20
[112935.350096]  [<ffffffff81012496>] ? native_read_tsc+0x6/0x20
[112935.355902]  [<ffffffff81012496>] ? native_read_tsc+0x6/0x20
[112935.361707]  <<EOE>>  [<ffffffff8128362a>] delay_tsc+0x4a/0x80
[112935.367748]  [<ffffffff8128354a>] __delay+0xa/0x10
[112935.372690]  [<ffffffff8128a9bc>] do_raw_spin_lock+0xcc/0x120
[112935.378592]  [<ffffffff815630ef>] _raw_spin_lock_irqsave+0x1f/0x30
[112935.384926]  [<ffffffff81404126>] input_repeat_key+0x26/0xd0
[112935.390742]  [<ffffffff8104b56f>] run_timer_softirq+0xff/0x230
[112935.396730]  [<ffffffff81011819>] ? read_tsc+0x9/0x20
[112935.401930]  [<ffffffff81404100>] ? input_dev_resume+0x20/0x20
[112935.407918]  [<ffffffff810276e8>] ? lapic_next_event+0x18/0x20
[112935.413907]  [<ffffffff81045da8>] __do_softirq+0x98/0x120
[112935.419452]  [<ffffffff8108171f>] ? tick_program_event+0x1f/0x30
[112935.425616]  [<ffffffff81564e9c>] call_softirq+0x1c/0x30
[112935.431074]  [<ffffffff8100c215>] do_softirq+0x65/0xa0
[112935.436361]  [<ffffffff810460ae>] irq_exit+0x8e/0xb0
[112935.441477]  [<ffffffff81027b79>] smp_apic_timer_interrupt+0x69/0xa0
[112935.447983]  [<ffffffff8156454a>] apic_timer_interrupt+0x6a/0x70
[112935.454144]  [<ffffffff81562eed>] ? _raw_spin_unlock_irq+0xd/0x20
[112935.460391]  [<ffffffff810647a7>] finish_task_switch+0x57/0xe0
[112935.466371]  [<ffffffff8100d9fb>] ? register_nmi_handler+0x4b/0x180
[112935.472793]  [<ffffffff81561cbb>] __schedule+0x3ab/0x720
[112935.478254]  [<ffffffff810a6e60>] ? watchdog_enable+0x240/0x240
[112935.484326]  [<ffffffff815622f4>] schedule+0x24/0x70
[112935.489443]  [<ffffffff810a6eda>] watchdog+0x7a/0xd0
[112935.494557]  [<ffffffff8105bd2e>] kthread+0x8e/0xa0
[112935.499583]  [<ffffffff81564da4>] kernel_thread_helper+0x4/0x10
[112935.505650]  [<ffffffff8105bca0>] ? flush_kthread_work+0x150/0x150
[112935.511983]  [<ffffffff81564da0>] ? gs_change+0x13/0x13
[112935.517356] ---[ end trace 6552bded36af03ab ]---
[112992.222406] BUG: spinlock lockup on CPU#0, watchdog/0/7
[112992.227780]  lock: ffff8804346d5d20, .magic: dead4ead, .owner: swapper/0/0, .owner_cpu: 0
[112992.237075] Pid: 7, comm: watchdog/0 Tainted: G      D W    3.4.39-21 #8
[112992.243928] Call Trace:
[112992.246526]  [<ffffffff8155cf14>] spin_dump+0x8a/0x8f
[112992.251727]  [<ffffffff8128a9d8>] do_raw_spin_lock+0xe8/0x120
[112992.257627]  [<ffffffff815630ef>] _raw_spin_lock_irqsave+0x1f/0x30
[112992.263962]  [<ffffffff81404126>] input_repeat_key+0x26/0xd0
[112992.269770]  [<ffffffff8104b56f>] run_timer_softirq+0xff/0x230
[112992.275757]  [<ffffffff81011819>] ? read_tsc+0x9/0x20
[112992.280957]  [<ffffffff81404100>] ? input_dev_resume+0x20/0x20
[112992.286946]  [<ffffffff810276e8>] ? lapic_next_event+0x18/0x20
[112992.292933]  [<ffffffff81045da8>] __do_softirq+0x98/0x120
[112992.298481]  [<ffffffff8108171f>] ? tick_program_event+0x1f/0x30
[112992.304642]  [<ffffffff81564e9c>] call_softirq+0x1c/0x30
[112992.310103]  [<ffffffff8100c215>] do_softirq+0x65/0xa0
[112992.315390]  [<ffffffff810460ae>] irq_exit+0x8e/0xb0
[112992.320504]  [<ffffffff81027b79>] smp_apic_timer_interrupt+0x69/0xa0
[112992.327012]  [<ffffffff8156454a>] apic_timer_interrupt+0x6a/0x70
[112992.333173]  [<ffffffff81562eed>] ? _raw_spin_unlock_irq+0xd/0x20
[112992.339421]  [<ffffffff810647a7>] finish_task_switch+0x57/0xe0
[112992.345408]  [<ffffffff8100d9fb>] ? register_nmi_handler+0x4b/0x180
[112992.351830]  [<ffffffff81561cbb>] __schedule+0x3ab/0x720
[112992.357289]  [<ffffffff810a6e60>] ? watchdog_enable+0x240/0x240
[112992.363365]  [<ffffffff815622f4>] schedule+0x24/0x70
[112992.368479]  [<ffffffff810a6eda>] watchdog+0x7a/0xd0
[112992.373593]  [<ffffffff8105bd2e>] kthread+0x8e/0xa0
[112992.378620]  [<ffffffff81564da4>] kernel_thread_helper+0x4/0x10
[112992.384694]  [<ffffffff8105bca0>] ? flush_kthread_work+0x150/0x150
[112992.391029]  [<ffffffff81564da0>] ? gs_change+0x13/0x13
[112992.396403] sending NMI to all CPUs:
[112992.400133] NMI backtrace for cpu 0
[112992.403775] CPU 0 
[112992.405739] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112992.434321] 
[112992.435965] Pid: 7, comm: watchdog/0 Tainted: G      D W    3.4.39-21 #8 System manufacturer System Product Name/Z8NR-D12
[112992.447259] RIP: 0010:[<ffffffff81012496>]  [<ffffffff81012496>] native_read_tsc+0x6/0x20
[112992.455670] RSP: 0018:ffff8802348bfaa8  EFLAGS: 00000046
[112992.461130] RAX: 00000000fb5c0731 RBX: 0000000000000000 RCX: 00000000fb5c0707
[112992.468441] RDX: 000000000000f72f RSI: 0000000000000080 RDI: 000000000003a1fe
[112992.475754] RBP: ffff8802348bfaa8 R08: ffffffff81885f10 R09: 0000000000000000
[112992.483066] R10: 0000000000000000 R11: 0000000000000000 R12: 00000000fb5c068c
[112992.490379] R13: 0000000000000000 R14: 000000000003a1fe R15: 0000000000000096
[112992.497690] FS:  0000000000000000(0000) GS:ffff880237c00000(0000) knlGS:0000000000000000
[112992.505954] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[112992.511846] CR2: 0000000000000000 CR3: 0000000233bcb000 CR4: 00000000000007f0
[112992.519159] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[112992.526473] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[112992.533783] Process watchdog/0 (pid: 7, threadinfo ffff8802348be000, task ffff880234899890)
[112992.542307] Stack:
[112992.544470]  ffff8802348bfad8 ffffffff8128362a 00000000000003e9 0000000000001000
[112992.552258]  ffffffff81885f10 0000000000000400 ffff8802348bfae8 ffffffff8128357c
[112992.560046]  ffff8802348bfb08 ffffffff81027a32 0000000000000002 000000000000dc42
[112992.567835] Call Trace:
[112992.570433]  [<ffffffff8128362a>] delay_tsc+0x4a/0x80
[112992.575634]  [<ffffffff8128357c>] __const_udelay+0x2c/0x30
[112992.581276]  [<ffffffff81027a32>] native_safe_apic_wait_icr_idle+0x22/0x50
[112992.588302]  [<ffffffff81028bdf>] default_send_IPI_mask_sequence_phys+0xcf/0xe0
[112992.595788]  [<ffffffff8102cd02>] physflat_send_IPI_all+0x12/0x20
[112992.602034]  [<ffffffff81028df0>] arch_trigger_all_cpu_backtrace+0x60/0xa0
[112992.609060]  [<ffffffff8128a9dd>] do_raw_spin_lock+0xed/0x120
[112992.614953]  [<ffffffff815630ef>] _raw_spin_lock_irqsave+0x1f/0x30
[112992.621288]  [<ffffffff81404126>] input_repeat_key+0x26/0xd0
[112992.627095]  [<ffffffff8104b56f>] run_timer_softirq+0xff/0x230
[112992.633082]  [<ffffffff81011819>] ? read_tsc+0x9/0x20
[112992.638283]  [<ffffffff81404100>] ? input_dev_resume+0x20/0x20
[112992.644264]  [<ffffffff810276e8>] ? lapic_next_event+0x18/0x20
[112992.650251]  [<ffffffff81045da8>] __do_softirq+0x98/0x120
[112992.655797]  [<ffffffff8108171f>] ? tick_program_event+0x1f/0x30
[112992.661951]  [<ffffffff81564e9c>] call_softirq+0x1c/0x30
[112992.667412]  [<ffffffff8100c215>] do_softirq+0x65/0xa0
[112992.672699]  [<ffffffff810460ae>] irq_exit+0x8e/0xb0
[112992.677813]  [<ffffffff81027b79>] smp_apic_timer_interrupt+0x69/0xa0
[112992.684319]  [<ffffffff8156454a>] apic_timer_interrupt+0x6a/0x70
[112992.690480]  [<ffffffff81562eed>] ? _raw_spin_unlock_irq+0xd/0x20
[112992.696728]  [<ffffffff810647a7>] finish_task_switch+0x57/0xe0
[112992.702709]  [<ffffffff8100d9fb>] ? register_nmi_handler+0x4b/0x180
[112992.709131]  [<ffffffff81561cbb>] __schedule+0x3ab/0x720
[112992.714589]  [<ffffffff810a6e60>] ? watchdog_enable+0x240/0x240
[112992.720664]  [<ffffffff815622f4>] schedule+0x24/0x70
[112992.725780]  [<ffffffff810a6eda>] watchdog+0x7a/0xd0
[112992.730893]  [<ffffffff8105bd2e>] kthread+0x8e/0xa0
[112992.735920]  [<ffffffff81564da4>] kernel_thread_helper+0x4/0x10
[112992.741996]  [<ffffffff8105bca0>] ? flush_kthread_work+0x150/0x150
[112992.748329]  [<ffffffff81564da0>] ? gs_change+0x13/0x13
[112992.753703] Code: c3 90 90 90 90 55 89 f8 48 89 e5 e6 70 e4 71 5d c3 0f 1f 40 00 55 89 f0 48 89 e5 e6 70 89 f8 e6 71 5d c3 66 90 55 48 89 e5 0f 31 <89> c1 48 89 d0 48 c1 e0 20 48 09 c8 5d c3 66 66 66 2e 0f 1f 84 
[112992.776079] Call Trace:
[112992.778678]  [<ffffffff8128362a>] delay_tsc+0x4a/0x80
[112992.783878]  [<ffffffff8128357c>] __const_udelay+0x2c/0x30
[112992.789511]  [<ffffffff81027a32>] native_safe_apic_wait_icr_idle+0x22/0x50
[112992.796537]  [<ffffffff81028bdf>] default_send_IPI_mask_sequence_phys+0xcf/0xe0
[112992.804024]  [<ffffffff8102cd02>] physflat_send_IPI_all+0x12/0x20
[112992.810272]  [<ffffffff81028df0>] arch_trigger_all_cpu_backtrace+0x60/0xa0
[112992.817297]  [<ffffffff8128a9dd>] do_raw_spin_lock+0xed/0x120
[112992.823198]  [<ffffffff815630ef>] _raw_spin_lock_irqsave+0x1f/0x30
[112992.829532]  [<ffffffff81404126>] input_repeat_key+0x26/0xd0
[112992.835339]  [<ffffffff8104b56f>] run_timer_softirq+0xff/0x230
[112992.841319]  [<ffffffff81011819>] ? read_tsc+0x9/0x20
[112992.846520]  [<ffffffff81404100>] ? input_dev_resume+0x20/0x20
[112992.852509]  [<ffffffff810276e8>] ? lapic_next_event+0x18/0x20
[112992.858496]  [<ffffffff81045da8>] __do_softirq+0x98/0x120
[112992.864044]  [<ffffffff8108171f>] ? tick_program_event+0x1f/0x30
[112992.870205]  [<ffffffff81564e9c>] call_softirq+0x1c/0x30
[112992.875664]  [<ffffffff8100c215>] do_softirq+0x65/0xa0
[112992.880951]  [<ffffffff810460ae>] irq_exit+0x8e/0xb0
[112992.886065]  [<ffffffff81027b79>] smp_apic_timer_interrupt+0x69/0xa0
[112992.892574]  [<ffffffff8156454a>] apic_timer_interrupt+0x6a/0x70
[112992.898736]  [<ffffffff81562eed>] ? _raw_spin_unlock_irq+0xd/0x20
[112992.904983]  [<ffffffff810647a7>] finish_task_switch+0x57/0xe0
[112992.910971]  [<ffffffff8100d9fb>] ? register_nmi_handler+0x4b/0x180
[112992.917393]  [<ffffffff81561cbb>] __schedule+0x3ab/0x720
[112992.922851]  [<ffffffff810a6e60>] ? watchdog_enable+0x240/0x240
[112992.928927]  [<ffffffff815622f4>] schedule+0x24/0x70
[112992.934042]  [<ffffffff810a6eda>] watchdog+0x7a/0xd0
[112992.939155]  [<ffffffff8105bd2e>] kthread+0x8e/0xa0
[112992.944182]  [<ffffffff81564da4>] kernel_thread_helper+0x4/0x10
[112992.950257]  [<ffffffff8105bca0>] ? flush_kthread_work+0x150/0x150
[112992.956591]  [<ffffffff81564da0>] ? gs_change+0x13/0x13
[112992.962023] NMI backtrace for cpu 1
[112992.965669] CPU 1 
[112992.967633] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112992.996215] 
[112992.997858] Pid: 0, comm: swapper/1 Tainted: G      D W    3.4.39-21 #8 System manufacturer System Product Name/Z8NR-D12
[112993.009066] RIP: 0010:[<ffffffff812d8137>]  [<ffffffff812d8137>] intel_idle+0xb7/0x150
[112993.017217] RSP: 0018:ffff88023499de48  EFLAGS: 00000046
[112993.022677] RAX: 0000000000000020 RBX: 0000000000000008 RCX: 0000000000000001
[112993.029990] RDX: 0000000000000000 RSI: ffff88023499dfd8 RDI: ffffffff81815880
[112993.037300] RBP: ffff88023499dea8 R08: 0000000000000000 R09: 000000000000020f
[112993.044612] R10: 0000000000000002 R11: 0000000000000000 R12: 0000000000000003
[112993.051925] R13: ffff880237c39470 R14: 0000000000000020 R15: 12f57b20d8479d09
[112993.059237] FS:  0000000000000000(0000) GS:ffff880237c20000(0000) knlGS:0000000000000000
[112993.067501] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[112993.073395] CR2: 00007f8d804d2000 CR3: 000000000180b000 CR4: 00000000000007e0
[112993.080708] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[112993.088018] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[112993.095331] Process swapper/1 (pid: 0, threadinfo ffff88023499c000, task ffff88023489c9b0)
[112993.103768] Stack:
[112993.105930]  0000000000000000 000000000011bf06 ffff88023499deb8 ffffffff814375b0
[112993.113718]  0000000000000000 000000010011bf06 0000000000000000 ffff880237c39470
[112993.121507]  ffffffff8183abc0 00000000fffffff0 0000000000000000 0000000000000000
[112993.129295] Call Trace:
[112993.132812]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112993.138452]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112993.143999]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112993.149987]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112993.155188]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112993.161089] Code: 28 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 f0 0f 01 c9 <e8> d4 26 da ff 4c 29 f8 48 89 c7 e8 59 cf d6 ff 4c 69 f0 40 42 
[112993.183467] Call Trace:
[112993.186063]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112993.191696]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112993.197244]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112993.203231]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112993.208433]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112993.214335] NMI backtrace for cpu 2
[112993.215370] INFO: rcu_sched detected stalls on CPUs/tasks:
[112993.215374] 	0: (1 GPs behind) idle=ccf/3/0 drain=0 . timer=-1
[112993.215376] 	(detected by 1, t=563040 jiffies)
[112993.215377] INFO: Stall ended before state dump start
[112993.215379] INFO: rcu_bh detected stalls on CPUs/tasks:
[112993.215381] 	0: (1 GPs behind) idle=ccf/3/0 drain=0 . timer=-1
[112993.215383] 	(detected by 1, t=562877 jiffies)
[112993.215384] INFO: Stall ended before state dump start
[112993.260534] CPU 2 
[112993.262498] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112993.291080] 
[112993.292725] Pid: 0, comm: swapper/2 Tainted: G      D W    3.4.39-21 #8 System manufacturer System Product Name/Z8NR-D12
[112993.303932] RIP: 0010:[<ffffffff812d8137>]  [<ffffffff812d8137>] intel_idle+0xb7/0x150
[112993.312083] RSP: 0018:ffff8802349b5e48  EFLAGS: 00000046
[112993.317544] RAX: 0000000000000010 RBX: 0000000000000004 RCX: 0000000000000001
[112993.324854] RDX: 0000000000000000 RSI: ffff8802349b5fd8 RDI: ffffffff81815880
[112993.332166] RBP: ffff8802349b5ea8 R08: 0000000000000000 R09: 0000000000000199
[112993.339479] R10: 0000000000000002 R11: 0000000000000000 R12: 0000000000000002
[112993.346792] R13: ffff880237c59470 R14: 0000000000000010 R15: 12f57b20d846a366
[112993.354155] FS:  0000000000000000(0000) GS:ffff880237c40000(0000) knlGS:0000000000000000
[112993.362419] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[112993.368311] CR2: 00007f6c0453ebd0 CR3: 000000000180b000 CR4: 00000000000007e0
[112993.375625] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[112993.382938] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[112993.390248] Process swapper/2 (pid: 0, threadinfo ffff8802349b4000, task ffff8802349bf2a0)
[112993.398685] Stack:
[112993.400848]  0000000000000000 000000000010000c ffff8802349b5eb8 ffffffff814375b0
[112993.408636]  0000000000000000 000000020010000c 0000000000000000 ffff880237c59470
[112993.416426]  ffffffff8183abc0 00000000fffffff0 0000000000000000 0000000000000000
[112993.424214] Call Trace:
[112993.426811]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112993.432443]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112993.437991]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112993.443979]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112993.449180]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112993.455079] Code: 28 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 f0 0f 01 c9 <e8> d4 26 da ff 4c 29 f8 48 89 c7 e8 59 cf d6 ff 4c 69 f0 40 42 
[112993.477457] Call Trace:
[112993.480055]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112993.485697]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112993.491245]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112993.497233]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112993.502433]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112993.508336] NMI backtrace for cpu 3
[112993.511976] CPU 3 
[112993.513940] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112993.542523] 
[112993.544168] Pid: 0, comm: swapper/3 Tainted: G      D W    3.4.39-21 #8 System manufacturer System Product Name/Z8NR-D12
[112993.555374] RIP: 0010:[<ffffffff812d8137>]  [<ffffffff812d8137>] intel_idle+0xb7/0x150
[112993.563525] RSP: 0018:ffff8802349d7e48  EFLAGS: 00000046
[112993.568984] RAX: 0000000000000010 RBX: 0000000000000004 RCX: 0000000000000001
[112993.576297] RDX: 0000000000000000 RSI: ffff8802349d7fd8 RDI: ffffffff81815880
[112993.583609] RBP: ffff8802349d7ea8 R08: 0000000000000000 R09: 0000000000000682
[112993.590922] R10: 0000000000000002 R11: 0000000000000000 R12: 0000000000000002
[112993.598235] R13: ffff880237c79470 R14: 0000000000000010 R15: 12f57b20d840513c
[112993.605546] FS:  0000000000000000(0000) GS:ffff880237c60000(0000) knlGS:0000000000000000
[112993.613810] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[112993.619704] CR2: 00007f8390134098 CR3: 000000000180b000 CR4: 00000000000007e0
[112993.627015] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[112993.634328] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[112993.641640] Process swapper/3 (pid: 0, threadinfo ffff8802349d6000, task ffff8802349b9890)
[112993.650085] Stack:
[112993.652248]  0000000000000000 0000000000190ba0 ffff8802349d7eb8 ffffffff814375b0
[112993.660027]  0000000000000000 0000000300190ba0 0000000000000000 ffff880237c79470
[112993.667816]  ffffffff8183abc0 00000000fffffff0 0000000000000000 0000000000000000
[112993.675604] Call Trace:
[112993.678202]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112993.683844]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112993.689391]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112993.695378]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112993.700579]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112993.706471] Code: 28 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 f0 0f 01 c9 <e8> d4 26 da ff 4c 29 f8 48 89 c7 e8 59 cf d6 ff 4c 69 f0 40 42 
[112993.728849] Call Trace:
[112993.731445]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112993.737079]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112993.742626]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112993.748615]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112993.753815]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112993.759717] NMI backtrace for cpu 4
[112993.763358] CPU 4 
[112993.765323] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112993.793914] 
[112993.795559] Pid: 0, comm: swapper/4 Tainted: G      D W    3.4.39-21 #8 System manufacturer System Product Name/Z8NR-D12
[112993.806763] RIP: 0010:[<ffffffff812d8137>]  [<ffffffff812d8137>] intel_idle+0xb7/0x150
[112993.814915] RSP: 0018:ffff8802349dbe48  EFLAGS: 00000046
[112993.820377] RAX: 0000000000000020 RBX: 0000000000000008 RCX: 0000000000000001
[112993.827689] RDX: 0000000000000000 RSI: ffff8802349dbfd8 RDI: ffffffff81815880
[112993.835001] RBP: ffff8802349dbea8 R08: 0000000000000000 R09: 0000000000000000
[112993.842311] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000003
[112993.849625] R13: ffff88043fc19470 R14: 0000000000000020 R15: 12f57b20d83d416f
[112993.856938] FS:  0000000000000000(0000) GS:ffff88043fc00000(0000) knlGS:0000000000000000
[112993.865209] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[112993.871104] CR2: 00007f8390133028 CR3: 000000000180b000 CR4: 00000000000007e0
[112993.878414] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[112993.885728] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[112993.893039] Process swapper/4 (pid: 0, threadinfo ffff8802349da000, task ffff8802349ba0c0)
[112993.901476] Stack:
[112993.903639]  0000000000000000 000000000ee4bff3 ffff8802349dbeb8 ffffffff814375b0
[112993.911427]  0000000000000000 000000040ee4bff3 0000000000000000 ffff88043fc19470
[112993.919216]  ffffffff8183abc0 00000000fffffff0 0000000000000000 0000000000000000
[112993.927004] Call Trace:
[112993.929601]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112993.935233]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112993.940781]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112993.946771]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112993.951971]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112993.957870] Code: 28 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 f0 0f 01 c9 <e8> d4 26 da ff 4c 29 f8 48 89 c7 e8 59 cf d6 ff 4c 69 f0 40 42 
[112993.981182] Call Trace:
[112993.983780]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112993.989413]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112993.994960]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112994.000948]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112994.006149]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112994.012052] NMI backtrace for cpu 5
[112994.015693] CPU 5 
[112994.017657] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112994.046257] 
[112994.047902] Pid: 0, comm: swapper/5 Tainted: G      D W    3.4.39-21 #8 System manufacturer System Product Name/Z8NR-D12
[112994.059098] RIP: 0010:[<ffffffff812d8137>]  [<ffffffff812d8137>] intel_idle+0xb7/0x150
[112994.067249] RSP: 0018:ffff8802349dde48  EFLAGS: 00000046
[112994.072712] RAX: 0000000000000010 RBX: 0000000000000004 RCX: 0000000000000001
[112994.080022] RDX: 0000000000000000 RSI: ffff8802349ddfd8 RDI: ffffffff81815880
[112994.087335] RBP: ffff8802349ddea8 R08: 0000000000000000 R09: 00000000001a76e6
[112994.094648] R10: 0000000000000002 R11: 0000000000000000 R12: 0000000000000002
[112994.101959] R13: ffff88043fc39470 R14: 0000000000000010 R15: 12f57b20d8445d94
[112994.109273] FS:  0000000000000000(0000) GS:ffff88043fc20000(0000) knlGS:0000000000000000
[112994.117543] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[112994.123436] CR2: 00007f838c2c3058 CR3: 000000000180b000 CR4: 00000000000007e0
[112994.130748] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[112994.138061] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[112994.145374] Process swapper/5 (pid: 0, threadinfo ffff8802349dc000, task ffff8802349bd1e0)
[112994.153818] Stack:
[112994.155981]  0000000000000000 000000000015014a ffff8802349ddeb8 ffffffff814375b0
[112994.163770]  0000000000000000 000000050015014a 0000000000000000 ffff88043fc39470
[112994.171559]  ffffffff8183abc0 00000000fffffff0 0000000000000000 0000000000000000
[112994.179347] Call Trace:
[112994.181945]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112994.187586]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112994.193133]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112994.199122]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112994.204323]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112994.210222] Code: 28 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 f0 0f 01 c9 <e8> d4 26 da ff 4c 29 f8 48 89 c7 e8 59 cf d6 ff 4c 69 f0 40 42 
[112994.232600] Call Trace:
[112994.235196]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112994.240831]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112994.246376]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112994.252356]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112994.257559]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112994.263459] NMI backtrace for cpu 6
[112994.267100] CPU 6 
[112994.269065] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112994.297648] 
[112994.299292] Pid: 0, comm: swapper/6 Tainted: G      D W    3.4.39-21 #8 System manufacturer System Product Name/Z8NR-D12
[112994.310490] RIP: 0010:[<ffffffff812d8137>]  [<ffffffff812d8137>] intel_idle+0xb7/0x150
[112994.318640] RSP: 0018:ffff8802349f1e48  EFLAGS: 00000046
[112994.324102] RAX: 0000000000000020 RBX: 0000000000000008 RCX: 0000000000000001
[112994.331412] RDX: 0000000000000000 RSI: ffff8802349f1fd8 RDI: ffffffff81815880
[112994.338725] RBP: ffff8802349f1ea8 R08: 0000000000000000 R09: 0000000000000000
[112994.346037] R10: 0000000000000002 R11: 0000000000000000 R12: 0000000000000003
[112994.353349] R13: ffff88043fc59470 R14: 0000000000000020 R15: 12f57b20d83d8930
[112994.360663] FS:  0000000000000000(0000) GS:ffff88043fc40000(0000) knlGS:0000000000000000
[112994.368927] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[112994.374820] CR2: 00007f0aceea5bd0 CR3: 000000000180b000 CR4: 00000000000007e0
[112994.382131] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[112994.389444] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[112994.396757] Process swapper/6 (pid: 0, threadinfo ffff8802349f0000, task ffff8802349ba8f0)
[112994.405201] Stack:
[112994.407364]  0000000000000000 00000000001bd26d ffff8802349f1eb8 ffffffff814375b0
[112994.415153]  0000000000000000 00000006001bd26d 0000000000000000 ffff88043fc59470
[112994.422941]  ffffffff8183abc0 00000000fffffff0 0000000000000000 0000000000000000
[112994.430729] Call Trace:
[112994.433326]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112994.438959]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112994.444506]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112994.450495]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112994.455695]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112994.461595] Code: 28 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 f0 0f 01 c9 <e8> d4 26 da ff 4c 29 f8 48 89 c7 e8 59 cf d6 ff 4c 69 f0 40 42 
[112994.483972] Call Trace:
[112994.486571]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112994.492213]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112994.497760]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112994.503749]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112994.508949]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112994.514850] NMI backtrace for cpu 7
[112994.518492] CPU 7 
[112994.520456] Modules linked in: coretemp psmouse ioatdma i7core_edac edac_core microcode dca serio_raw tcp_htcp raid10 raid456 async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid ahci libahci mptsas mptscsih mptbase scsi_transport_sas e1000e raid6_pq async_tx raid1 raid0 multipath linear
[112994.549047] 
[112994.550692] Pid: 0, comm: swapper/7 Tainted: G      D W    3.4.39-21 #8 System manufacturer System Product Name/Z8NR-D12
[112994.561889] RIP: 0010:[<ffffffff812d8137>]  [<ffffffff812d8137>] intel_idle+0xb7/0x150
[112994.570040] RSP: 0018:ffff8802349f5e48  EFLAGS: 00000046
[112994.575502] RAX: 0000000000000020 RBX: 0000000000000008 RCX: 0000000000000001
[112994.582812] RDX: 0000000000000000 RSI: ffff8802349f5fd8 RDI: ffffffff81815880
[112994.590125] RBP: ffff8802349f5ea8 R08: 0000000000000000 R09: 0000000000000000
[112994.597438] R10: 0000000000000001 R11: 0000000000000ffe R12: 0000000000000003
[112994.604748] R13: ffff88043fc79470 R14: 0000000000000020 R15: 12f57b20d83d4e60
[112994.612061] FS:  0000000000000000(0000) GS:ffff88043fc60000(0000) knlGS:0000000000000000
[112994.620325] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[112994.626218] CR2: 00007f8d804d2000 CR3: 000000000180b000 CR4: 00000000000007e0
[112994.633530] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[112994.640843] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[112994.648156] Process swapper/7 (pid: 0, threadinfo ffff8802349f4000, task ffff8802349bc9b0)
[112994.656600] Stack:
[112994.658763]  0000000000000000 000000002c61ee04 ffff8802349f5eb8 ffffffff814375b0
[112994.666552]  0000000000000000 000000072c61ee04 0000000000000000 ffff88043fc79470
[112994.674340]  ffffffff8183abc0 00000000fffffff0 0000000000000000 0000000000000000
[112994.682127] Call Trace:
[112994.684726]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112994.690358]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112994.695905]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112994.701893]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112994.707096]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd
[112994.712995] Code: 28 e0 ff ff 83 e2 08 75 22 31 d2 48 83 c0 10 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 b1 01 4c 89 f0 0f 01 c9 <e8> d4 26 da ff 4c 29 f8 48 89 c7 e8 59 cf d6 ff 4c 69 f0 40 42 
[112994.735373] Call Trace:
[112994.737970]  [<ffffffff814375b0>] ? menu_select+0xf0/0x3a0
[112994.743603]  [<ffffffff81435dd9>] cpuidle_enter+0x19/0x20
[112994.749151]  [<ffffffff8143639b>] cpuidle_idle_call+0x8b/0x100
[112994.755138]  [<ffffffff81013535>] cpu_idle+0xc5/0x120
[112994.760339]  [<ffffffff8154fb47>] start_secondary+0x1f6/0x1fd