lkml.org 
[lkml]   [2004]   [Jun]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: processes hung in D (raid5/dm/ext3)
    From
    I just had another go at this with 2.6.7.  I tried some different things
    this time.

    First I tried taring the filesystem, and that went OK. Then I tried find
    -exec cat and that was OK, too. Next I did cat /dev/md0, and when that
    was OK I did cat /dev/vg0/home and that also worked. Interestingly, cat
    /dev/md0 gets about 120-130MB/s throughput whereas cat /dev/vg0/home
    only gets 60-70MB/s.

    Fianlly I tried dump again and that hung right away, trace below. I
    also got an ls hung before I made the trace, but that doesn't seem to
    show anything new.

    A possibly interesting note about the serial console: when I triggered
    the trace, apcupsd lost communication with the UPS, which is using the
    other serial port (this is while the trace was printing to the console).
    Communication resumed when it was done printing. The serial ports don't
    share an interrupt, with each other or anything else.

    There was no ethernet grief when I rebooted.


    Bootdata ok (command line is BOOT_IMAGE=linux ro root=851 console=ttyS0,19200n8 single)
    Linux version 2.6.7 (root@carignan) (gcc version 3.3.4 (Debian)) #1 SMP Wed Jun 16 02:55:47 EDT 2004
    BIOS-provided physical RAM map:
    BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
    BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
    BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
    BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
    BIOS-e820: 000000007fff0000 - 000000007ffff000 (ACPI data)
    BIOS-e820: 000000007ffff000 - 0000000080000000 (ACPI NVS)
    BIOS-e820: 00000000ff7c0000 - 0000000100000000 (reserved)
    Scanning NUMA topology in Northbridge 24
    Number of nodes 2 (10010)
    Node 0 MemBase 0000000000000000 Limit 000000003fffffff
    Node 1 MemBase 0000000040000000 Limit 000000007fff0000
    Using node hash shift of 24
    Bootmem setup node 0 0000000000000000-000000003fffffff
    Bootmem setup node 1 0000000040000000-000000007fff0000
    No mptable found.
    On node 0 totalpages: 262143
    DMA zone: 4096 pages, LIFO batch:1
    Normal zone: 258047 pages, LIFO batch:16
    HighMem zone: 0 pages, LIFO batch:1
    On node 1 totalpages: 262128
    DMA zone: 0 pages, LIFO batch:1
    Normal zone: 262128 pages, LIFO batch:16
    HighMem zone: 0 pages, LIFO batch:1
    ACPI: RSDP (v002 ACPIAM ) @ 0x00000000000f4680
    ACPI: XSDT (v001 A M I OEMXSDT 0x06000318 MSFT 0x00000097) @ 0x000000007fff0100
    ACPI: FADT (v001 A M I OEMFACP 0x06000318 MSFT 0x00000097) @ 0x000000007fff0281
    ACPI: MADT (v001 A M I OEMAPIC 0x06000318 MSFT 0x00000097) @ 0x000000007fff0380
    ACPI: OEMB (v001 A M I OEMBIOS 0x06000318 MSFT 0x00000097) @ 0x000000007ffff040
    ACPI: ASF! (v001 AMIASF AMDSTRET 0x00000001 INTL 0x02002026) @ 0x000000007fff3530
    ACPI: DSDT (v001 0ABCF 0ABCF007 0x00000007 INTL 0x02002026) @ 0x0000000000000000
    ACPI: Local APIC address 0xfee00000
    ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
    Processor #0 15:5 APIC version 16
    ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
    Processor #1 15:5 APIC version 16
    ACPI: IOAPIC (id[0x02] address[0xfec00000] global_irq_base[0x0])
    IOAPIC[0]: Assigned apic_id 2
    IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
    ACPI: IOAPIC (id[0x03] address[0xfebfe000] global_irq_base[0x18])
    IOAPIC[1]: Assigned apic_id 3
    IOAPIC[1]: apic_id 3, version 17, address 0xfebfe000, GSI 24-27
    ACPI: IOAPIC (id[0x04] address[0xfebff000] global_irq_base[0x1c])
    IOAPIC[2]: Assigned apic_id 4
    IOAPIC[2]: apic_id 4, version 17, address 0xfebff000, GSI 28-31
    ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
    ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
    ACPI: IRQ0 used by override.
    ACPI: IRQ2 used by override.
    ACPI: IRQ9 used by override.
    Using ACPI (MADT) for SMP configuration information
    Checking aperture...
    CPU 0: aperture @ 1ee0000000 size 64 MB
    Aperture from northbridge cpu 0 beyond 4GB. Ignoring.
    No AGP bridge found
    Built 2 zonelists
    Kernel command line: BOOT_IMAGE=linux ro root=851 console=ttyS0,19200n8 single
    Initializing CPU#0
    PID hash table entries: 16 (order 4: 256 bytes)
    time.c: Using 1.193182 MHz PIT timer.
    time.c: Detected 1394.599 MHz processor.
    Console: colour VGA+ 80x25
    Memory: 2062532k/2097088k available (2541k kernel code, 0k reserved, 1205k data, 180k init)
    Calibrating delay loop... 2736.12 BogoMIPS
    Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
    Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
    Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
    CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
    CPU: L2 Cache: 1024K (64 bytes/line)
    Using local APIC NMI watchdog using perfctr0
    CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
    CPU: L2 Cache: 1024K (64 bytes/line)
    CPU0: AMD Opteron(tm) Processor 240 stepping 01
    per-CPU timeslice cutoff: 1023.75 usecs.
    task migration cache decay timeout: 2 msecs.
    Booting processor 1/1 rip 6000 rsp 10040e3ff58
    Initializing CPU#1
    Calibrating delay loop... 2785.28 BogoMIPS
    CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
    CPU: L2 Cache: 1024K (64 bytes/line)
    AMD Opteron(tm) Processor 240 stepping 01
    Total of 2 processors activated (5521.40 BogoMIPS).
    ENABLING IO-APIC IRQs
    init IO_APIC IRQs
    IO-APIC (apicid-pin) 2-0, 2-16, 2-17, 2-18, 2-19, 2-20, 2-21, 2-22, 2-23, 3-0, 3-1, 3-2, 3-3, 4-0, 4-1, 4-2, 4-3 not connected.
    ..TIMER: vector=0x31 pin1=2 pin2=-1
    Using local APIC timer interrupts.
    Detected 12.451 MHz APIC timer.
    checking TSC synchronization across 2 CPUs: passed.
    time.c: Using PIT/TSC based timekeeping.
    Brought up 2 CPUs
    CPU0: online
    domain 0: span 01
    groups: 01
    domain 1: span 03
    groups: 01 02
    CPU1: online
    domain 0: span 02
    groups: 02
    domain 1: span 03
    groups: 02 01
    NET: Registered protocol family 16
    PCI: Using configuration type 1
    mtrr: v2.0 (20020519)
    ACPI: Subsystem revision 20040326
    ACPI: Interpreter enabled
    ACPI: Using IOAPIC for interrupt routing
    ACPI: PCI Root Bridge [PCI0] (00:00)
    PCI: Probing PCI hardware (bus 00)
    ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
    ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PCI1._PRT]
    ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.GOLA._PRT]
    ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.GOLB._PRT]
    ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
    ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
    ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
    ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 *9 10 11 12 14 15)
    SCSI subsystem initialized
    usbcore: registered new driver usbfs
    usbcore: registered new driver hub
    IOAPIC[0]: Set PCI routing entry (2-16 -> 0xa9 -> IRQ 16 Mode:1 Active:1)
    00:00:07[A] -> 2-16 -> IRQ 16
    IOAPIC[0]: Set PCI routing entry (2-17 -> 0xb1 -> IRQ 17 Mode:1 Active:1)
    00:00:07[B] -> 2-17 -> IRQ 17
    IOAPIC[0]: Set PCI routing entry (2-18 -> 0xb9 -> IRQ 18 Mode:1 Active:1)
    00:00:07[C] -> 2-18 -> IRQ 18
    IOAPIC[0]: Set PCI routing entry (2-19 -> 0xc1 -> IRQ 19 Mode:1 Active:1)
    00:00:07[D] -> 2-19 -> IRQ 19
    IOAPIC[1]: Set PCI routing entry (3-3 -> 0xc9 -> IRQ 27 Mode:1 Active:1)
    00:02:08[A] -> 3-3 -> IRQ 27
    IOAPIC[1]: Set PCI routing entry (3-0 -> 0xd1 -> IRQ 24 Mode:1 Active:1)
    00:02:08[B] -> 3-0 -> IRQ 24
    IOAPIC[1]: Set PCI routing entry (3-1 -> 0xd9 -> IRQ 25 Mode:1 Active:1)
    00:02:08[C] -> 3-1 -> IRQ 25
    IOAPIC[1]: Set PCI routing entry (3-2 -> 0xe1 -> IRQ 26 Mode:1 Active:1)
    00:02:08[D] -> 3-2 -> IRQ 26
    IOAPIC[2]: Set PCI routing entry (4-0 -> 0xe9 -> IRQ 28 Mode:1 Active:1)
    00:01:03[A] -> 4-0 -> IRQ 28
    IOAPIC[2]: Set PCI routing entry (4-1 -> 0x32 -> IRQ 29 Mode:1 Active:1)
    00:01:03[B] -> 4-1 -> IRQ 29
    IOAPIC[2]: Set PCI routing entry (4-2 -> 0x3a -> IRQ 30 Mode:1 Active:1)
    00:01:03[C] -> 4-2 -> IRQ 30
    IOAPIC[2]: Set PCI routing entry (4-3 -> 0x42 -> IRQ 31 Mode:1 Active:1)
    00:01:03[D] -> 4-3 -> IRQ 31
    number of MP IRQ sources: 16.
    number of IO-APIC #2 registers: 24.
    number of IO-APIC #3 registers: 4.
    number of IO-APIC #4 registers: 4.
    testing the IO APIC.......................

    IO APIC #2......
    .... register #00: 02000000
    ....... : physical APIC id: 02
    .... register #01: 00170011
    ....... : max redirection entries: 0017
    ....... : PRQ implemented: 0
    ....... : IO APIC version: 0011
    .... register #02: 02000000
    ....... : arbitration: 02
    .... IRQ redirection table:
    NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
    00 000 00 1 0 0 0 0 0 0 00
    01 001 01 0 0 0 0 0 1 1 39
    02 001 01 0 0 0 0 0 1 1 31
    03 001 01 0 0 0 0 0 1 1 41
    04 001 01 0 0 0 0 0 1 1 49
    05 001 01 0 0 0 0 0 1 1 51
    06 001 01 0 0 0 0 0 1 1 59
    07 001 01 0 0 0 0 0 1 1 61
    08 001 01 0 0 0 0 0 1 1 69
    09 001 01 0 1 0 1 0 1 1 71
    0a 001 01 0 0 0 0 0 1 1 79
    0b 001 01 0 0 0 0 0 1 1 81
    0c 001 01 0 0 0 0 0 1 1 89
    0d 001 01 0 0 0 0 0 1 1 91
    0e 001 01 0 0 0 0 0 1 1 99
    0f 001 01 0 0 0 0 0 1 1 A1
    10 001 01 1 1 0 1 0 1 1 A9
    11 001 01 1 1 0 1 0 1 1 B1
    12 001 01 1 1 0 1 0 1 1 B9
    13 001 01 1 1 0 1 0 1 1 C1
    14 000 00 1 0 0 0 0 0 0 00
    15 000 00 1 0 0 0 0 0 0 00
    16 000 00 1 0 0 0 0 0 0 00
    17 000 00 1 0 0 0 0 0 0 00

    IO APIC #3......
    .... register #00: 03000000
    ....... : physical APIC id: 03
    .... register #01: 00030011
    ....... : max redirection entries: 0003
    ....... : PRQ implemented: 0
    ....... : IO APIC version: 0011
    .... register #02: 00000000
    ....... : arbitration: 00
    .... IRQ redirection table:
    NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
    00 001 01 1 1 0 1 0 1 1 D1
    01 001 01 1 1 0 1 0 1 1 D9
    02 001 01 1 1 0 1 0 1 1 E1
    03 001 01 1 1 0 1 0 1 1 C9

    IO APIC #4......
    .... register #00: 04000000
    ....... : physical APIC id: 04
    .... register #01: 00030011
    ....... : max redirection entries: 0003
    ....... : PRQ implemented: 0
    ....... : IO APIC version: 0011
    .... register #02: 00000000
    ....... : arbitration: 00
    .... IRQ redirection table:
    NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
    00 001 01 1 1 0 1 0 1 1 E9
    01 001 01 1 1 0 1 0 1 1 32
    02 001 01 1 1 0 1 0 1 1 3A
    03 001 01 1 1 0 1 0 1 1 42
    IRQ to pin mappings:
    IRQ0 -> 0:2
    IRQ1 -> 0:1
    IRQ3 -> 0:3
    IRQ4 -> 0:4
    IRQ5 -> 0:5
    IRQ6 -> 0:6
    IRQ7 -> 0:7
    IRQ8 -> 0:8
    IRQ9 -> 0:9
    IRQ10 -> 0:10
    IRQ11 -> 0:11
    IRQ12 -> 0:12
    IRQ13 -> 0:13
    IRQ14 -> 0:14
    IRQ15 -> 0:15
    IRQ16 -> 0:16
    IRQ17 -> 0:17
    IRQ18 -> 0:18
    IRQ19 -> 0:19
    IRQ24 -> 1:0
    IRQ25 -> 1:1
    IRQ26 -> 1:2
    IRQ27 -> 1:3
    IRQ28 -> 2:0
    IRQ29 -> 2:1
    IRQ30 -> 2:2
    IRQ31 -> 2:3
    .................................... done.
    PCI: Using ACPI for IRQ routing
    PCI-DMA: Disabling IOMMU.
    IA-32 Microcode Update Driver: v1.13 <tigran@veritas.com>
    IA32 emulation $Id: sys_ia32.c,v 1.32 2002/03/24 13:02:28 ak Exp $
    Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
    ACPI: Power Button (FF) [PWRF]
    ACPI: Processor [CPU1] (supports C1, 8 throttling states)
    ACPI: Processor [CPU2] (supports C1)
    lp: driver loaded but no devices found
    Real Time Clock Driver v1.12
    hw_random: AMD768 system management I/O registers at 0x5000.
    hw_random hardware driver 1.0.0 loaded
    Software Watchdog Timer: 0.07 initialized. soft_noboot=0 soft_margin=60 sec (nowayout= 0)
    Linux agpgart interface v0.100 (c) Dave Jones
    Hangcheck: starting hangcheck timer 0.5.0 (tick is 180 seconds, margin is 60 seconds).
    Serial: 8250/16550 driver $Revision: 1.90 $ 8 ports, IRQ sharing disabled
    ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
    ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
    parport0: PC-style at 0x378 [PCSPP(,...)]
    lp0: using parport0 (polling).
    Using anticipatory io scheduler
    Floppy drive(s): fd0 is 1.44M
    FDC 0 is a post-1991 82077
    RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
    loop: loaded (max 8 devices)
    tg3.c:v3.6 (June 12, 2004)
    eth0: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:c9
    eth0: HostTXDS[1] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
    eth1: Tigon3 [partno(BCM95704A7) rev 2003 PHY(5704)] (PCIX:100MHz:64-bit) 10/100/1000BaseT Ethernet 00:e0:81:51:d6:ca
    eth1: HostTXDS[1] RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1]
    sym0: <875> rev 0x26 at pci 0000:03:04.0 irq 16
    sym0: Symbios NVRAM, ID 7, Fast-20, SE, parity checking
    sym0: open drain IRQ line driver, using on-chip SRAM
    sym0: using LOAD/STORE-based firmware.
    sym0: SCSI BUS has been reset.
    scsi0 : sym-2.1.18j
    Vendor: SONY Model: SDX-400C Rev: 0702
    Type: Sequential-Access ANSI SCSI revision: 02
    scsi(0:0:4:0): Beginning Domain Validation
    sym0:4: wide asynchronous.
    sym0:4: FAST-20 WIDE SCSI 40.0 MB/s ST (50.0 ns, offset 15)
    scsi(0:0:4:0): Domain Validation skipping write tests
    scsi(0:0:4:0): Ending Domain Validation
    Vendor: OVERLAND Model: LIBRARYPRO Rev: 0417
    Type: Medium Changer ANSI SCSI revision: 02
    scsi(0:0:6:0): Beginning Domain Validation
    sym0:6: wide asynchronous.
    sym0:6: FAST-10 WIDE SCSI 20.0 MB/s ST (100.0 ns, offset 15)
    scsi(0:0:6:0): Domain Validation skipping write tests
    scsi(0:0:6:0): Ending Domain Validation
    st: Version 20040403, fixed bufsize 32768, s/g segs 256
    Attached scsi tape st0 at scsi0, channel 0, id 4, lun 0
    st0: try direct i/o: yes (alignment 512 B), max page reachable by HBA 1048575
    Attached scsi generic sg0 at scsi0, channel 0, id 4, lun 0, type 1
    Attached scsi generic sg1 at scsi0, channel 0, id 6, lun 0, type 8
    Fusion MPT base driver 3.01.06
    Copyright (c) 1999-2004 LSI Logic Corporation
    mptbase: Initiating ioc0 bringup
    ioc0: 53C1030: Capabilities={Initiator}
    mptbase: Initiating ioc1 bringup
    ioc1: 53C1030: Capabilities={Initiator}
    Fusion MPT SCSI Host driver 3.01.06
    scsi1 : ioc0: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=24
    Vendor: SUPER Model: GEM318 Rev: 0
    Type: Processor ANSI SCSI revision: 02
    Attached scsi generic sg2 at scsi1, channel 0, id 8, lun 0, type 3
    Vendor: SEAGATE Model: ST3146807LC Rev: 0006
    Type: Direct-Access ANSI SCSI revision: 03
    SCSI device sda: 286749488 512-byte hdwr sectors (146816 MB)
    SCSI device sda: drive cache: write back
    sda: sda1
    Attached scsi disk sda at scsi1, channel 0, id 9, lun 0
    Attached scsi generic sg3 at scsi1, channel 0, id 9, lun 0, type 0
    Vendor: SEAGATE Model: ST3146807LC Rev: 0006
    Type: Direct-Access ANSI SCSI revision: 03
    SCSI device sdb: 286749488 512-byte hdwr sectors (146816 MB)
    SCSI device sdb: drive cache: write back
    sdb: sdb1
    Attached scsi disk sdb at scsi1, channel 0, id 10, lun 0
    Attached scsi generic sg4 at scsi1, channel 0, id 10, lun 0, type 0
    Vendor: SEAGATE Model: ST3146807LC Rev: 0006
    Type: Direct-Access ANSI SCSI revision: 03
    SCSI device sdc: 286749488 512-byte hdwr sectors (146816 MB)
    SCSI device sdc: drive cache: write back
    sdc: sdc1
    Attached scsi disk sdc at scsi1, channel 0, id 11, lun 0
    Attached scsi generic sg5 at scsi1, channel 0, id 11, lun 0, type 0
    Vendor: SEAGATE Model: ST3146807LC Rev: 0007
    Type: Direct-Access ANSI SCSI revision: 03
    SCSI device sdd: 286749488 512-byte hdwr sectors (146816 MB)
    SCSI device sdd: drive cache: write back
    sdd: sdd1
    Attached scsi disk sdd at scsi1, channel 0, id 12, lun 0
    Attached scsi generic sg6 at scsi1, channel 0, id 12, lun 0, type 0
    Vendor: SEAGATE Model: ST3146807LC Rev: 0007
    Type: Direct-Access ANSI SCSI revision: 03
    SCSI device sde: 286749488 512-byte hdwr sectors (146816 MB)
    SCSI device sde: drive cache: write back
    sde: sde1
    Attached scsi disk sde at scsi1, channel 0, id 13, lun 0
    Attached scsi generic sg7 at scsi1, channel 0, id 13, lun 0, type 0
    scsi2 : ioc1: LSI53C1030, FwRev=01030600h, Ports=1, MaxQ=255, IRQ=25
    Vendor: SEAGATE Model: ST336607LW Rev: 0007
    Type: Direct-Access ANSI SCSI revision: 03
    SCSI device sdf: 71687372 512-byte hdwr sectors (36704 MB)
    SCSI device sdf: drive cache: write back
    sdf: sdf1 sdf2 sdf3 sdf4
    Attached scsi disk sdf at scsi2, channel 0, id 0, lun 0
    Attached scsi generic sg8 at scsi2, channel 0, id 0, lun 0, type 0
    USB Universal Host Controller Interface driver v2.2
    Initializing USB Mass Storage driver...
    usbcore: registered new driver usb-storage
    USB Mass Storage support registered.
    mice: PS/2 mouse device common for all mice
    serio: i8042 AUX port at 0x60,0x64 irq 12
    serio: i8042 KBD port at 0x60,0x64 irq 1
    i2c /dev entries driver
    md: raid1 personality registered as nr 3
    md: raid5 personality registered as nr 4
    raid5: measuring checksumming speed
    generic_sse: 4256.000 MB/sec
    raid5: using function: generic_sse (4256.000 MB/sec)
    md: md driver 0.90.0 MAX_MD_DEVS=256, MD_SB_DISKS=27
    device-mapper: 4.1.0-ioctl (2003-12-10) initialised: dm@uk.sistina.com
    NET: Registered protocol family 2
    IP: routing cache hash table of 8192 buckets, 128Kbytes
    TCP: Hash tables configured (established 262144 bind 65536)
    NET: Registered protocol family 1
    NET: Registered protocol family 17
    md: Autodetecting RAID arrays.
    md: autorun ...
    md: considering sde1 ...
    md: adding sde1 ...
    md: adding sdd1 ...
    md: adding sdc1 ...
    md: adding sdb1 ...
    md: adding sda1 ...
    md: created md0
    md: bind<sda1>
    md: bind<sdb1>
    md: bind<sdc1>
    md: bind<sdd1>
    md: bind<sde1>
    md: running: <sde1><sdd1><sdc1><sdb1><sda1>
    raid5: device sde1 operational as raid disk 4
    raid5: device sdd1 operational as raid disk 3
    raid5: device sdc1 operational as raid disk 2
    raid5: device sdb1 operational as raid disk 1
    raid5: device sda1 operational as raid disk 0
    raid5: allocated 5300kB for md0
    raid5: raid level 5 set md0 active with 5 out of 5 devices, algorithm 2
    RAID5 conf printout:
    --- rd:5 wd:5 fd:0
    disk 0, o:1, dev:sda1
    disk 1, o:1, dev:sdb1
    disk 2, o:1, dev:sdc1
    disk 3, o:1, dev:sdd1
    disk 4, o:1, dev:sde1
    md: ... autorun DONE.
    kjournald starting. Commit interval 5 seconds
    EXT3-fs: mounted filesystem with ordered data mode.
    VFS: Mounted root (ext3 filesystem) readonly.
    Freeing unused kernel memory: 180k freed
    Adding 2097136k swap on /dev/sdf2. Priority:-1 extents:1
    EXT3 FS on sdf1, internal journal
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on sdf3, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-32, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-34, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-33, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-35, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-0, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-2, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-3, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-1, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-4, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-5, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-6, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-7, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-8, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-9, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-10, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-11, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-12, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-13, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-14, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-15, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-16, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-17, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-18, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-19, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-20, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-21, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-22, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-23, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-24, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-25, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-26, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-27, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-28, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-29, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-30, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    kjournald starting. Commit interval 5 seconds
    EXT3 FS on dm-31, internal journal
    EXT3-fs: mounted filesystem with ordered data mode.
    Ethernet Channel Bonding Driver: v2.6.0 (January 14, 2004)
    bonding: MII link monitoring set to 100 ms
    bonding: bond0: enslaving eth0 as an active interface with a down link.
    bonding: bond0: enslaving eth1 as an active interface with a down link.
    tg3: eth0: Link is up at 1000 Mbps, full duplex.
    tg3: eth0: Flow control is off for TX and off for RX.
    tg3: eth1: Link is up at 1000 Mbps, full duplex.
    tg3: eth1: Flow control is off for TX and off for RX.
    bonding: bond0: link status definitely up for interface eth0.
    bonding: bond0: link status definitely up for interface eth1.
    process `syslogd' is using obsolete setsockopt SO_BSDCOMPAT
    SCSI-3 Opcodes & ASC/ASCQ Strings 3.01.06
    mptbase: English readable SCSI-3 strings enabled:-)
    isense: Registered SCSI-3 Opcodes & ASC/ASCQ Strings
    st0: Block limits 2 - 16777215 bytes.
    rpc.mountd[752]: segfault at 000000004b87ad6e rip 00000000080503e9 rsp 00000000ffffd684 error 4
    nfsd: last server has exited
    nfsd: unexporting all filesystems
    rpc.mountd[3959]: segfault at 000000004b87ad6e rip 00000000080503e9 rsp 00000000ffffd494 error 4
    nfsd: last server has exited
    nfsd: unexporting all filesystems
    rpc.mountd[3996]: segfault at 000000004b87ad6e rip 00000000080503e9 rsp 00000000ffffd494 error 4
    nfsd: last server has exited
    nfsd: unexporting all filesystems
    rpciod: active tasks at shutdown?!
    RPC: error 5 connecting to server localhost
    RPC: failed to contact portmap (errno -5).
    SysRq : Show State

    sibling
    task PC pid father child younger older
    init S 00003e2596edc7f2 0 1 0 2 (NOTLB)
    000001003ff8fd88 0000000000000006 0000007800000002 000001007ff9b070
    0000000000000626 ffffffff803e23e0 000001007ff9b388 0000000000000246
    0000000000000216 ffffffff8015a512
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377ef4>{schedule_timeout+180}
    <ffffffff80141b20>{process_timeout+0} <ffffffff801918ca>{do_select+906}
    <ffffffff80186c64>{vfs_stat+68} <ffffffff80191380>{__pollwait+0}
    <ffffffff80186bcb>{vfs_getattr+59} <ffffffff801ab8a1>{compat_sys_select+1105}
    <ffffffff80123f81>{ia32_sysret+0}
    migration/0 S 0000000000000000 0 2 1 3 (L-TLB)
    0000010040e73ed8 0000000000000046 00000074349d5cc8 000001007ff9a030
    00000000000001ec 000001003e29e8d0 000001007ff9a348 00000100349d5cb8
    00000100349d5cc0 0000000000000000
    Call Trace:<ffffffff8013487c>{migration_thread+236} <ffffffff80134790>{migration_thread+0}
    <ffffffff80134790>{migration_thread+0} <ffffffff8014e502>{kthread+146}
    <ffffffff801115f7>{child_rip+8} <ffffffff80134790>{migration_thread+0}
    <ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}

    ksoftirqd/0 S 0000000000000000 0 3 1 4 2 (L-TLB)
    0000010040e71f08 0000000000000046 0000007d40e71e98 0000010001e6d0b0
    00000000000000b7 000001003e61b370 0000010001e6d3c8 0000010040e71e98
    0000010001e12b20 0000000000000000
    Call Trace:<ffffffff8013d0ff>{tasklet_action+111} <ffffffff8013d2a0>{ksoftirqd+0}
    <ffffffff8013d2df>{ksoftirqd+63} <ffffffff8014e502>{kthread+146}
    <ffffffff801115f7>{child_rip+8} <ffffffff8013d2a0>{ksoftirqd+0}
    <ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}

    migration/1 S 0000000000000000 0 4 1 5 3 (L-TLB)
    0000010001e6bed8 0000000000000046 000000795435fcc8 0000010001e6c890
    00000000000000fc 000001007ed104b0 0000010001e6cba8 000001005435fcb8
    000001005435fcc0 0000000000000001
    Call Trace:<ffffffff8013487c>{migration_thread+236} <ffffffff80134790>{migration_thread+0}
    <ffffffff80134790>{migration_thread+0} <ffffffff8014e502>{kthread+146}
    <ffffffff801115f7>{child_rip+8} <ffffffff80134790>{migration_thread+0}
    <ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}

    ksoftirqd/1 S 00003e1fba66ad5c 0 5 1 6 4 (L-TLB)
    0000010001e69f08 0000000000000046 0000010001e69e98 0000010001e6c070
    000000000000010f 000001007ff9a850 0000010001e6c388 0000010001e69e98
    0000010040e1e9a0 0000000000000000
    Call Trace:<ffffffff8013d0ff>{tasklet_action+111} <ffffffff8013d2a0>{ksoftirqd+0}
    <ffffffff8013d2df>{ksoftirqd+63} <ffffffff8014e502>{kthread+146}
    <ffffffff801115f7>{child_rip+8} <ffffffff8013d2a0>{ksoftirqd+0}
    <ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}

    events/0 R running task 0 6 1 8 7 5 (L-TLB)
    events/1 S 00003e267f613016 0 7 1 42 6 (L-TLB)
    000001007ff7fe58 0000000000000046 0000007300000212 0000010040e408d0
    000000000000039f 000001007ff9a850 0000010040e40be8 0000000000000001
    0000000000000003 0000000000000000
    Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
    <ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
    <ffffffff80149cf0>{worker_thread+0} <ffffffff8014e470>{kthread+0}
    <ffffffff801115ef>{child_rip+0}
    khelper S 000000067c85eea7 0 8 6 9 (L-TLB)
    0000010001e4fe58 0000000000000046 000001007f459d38 0000010040e400b0
    000000000000024d 000001007ff9a850 0000010040e403c8 0000000000000001
    0000000000000003 0000000000000000
    Call Trace:<ffffffff801115ef>{child_rip+0} <ffffffff80149df5>{worker_thread+261}
    <ffffffff803774bc>{thread_return+41} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80149cf0>{worker_thread+0}
    <ffffffff80149cf0>{worker_thread+0} <ffffffff8014e502>{kthread+146}
    <ffffffff801115f7>{child_rip+8} <ffffffff8014e540>{keventd_create_kthread+0}
    <ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}

    kacpid S 00000000087a440f 0 9 6 40 8 (L-TLB)
    000001007fefde58 0000000000000046 000001007fefde18 0000010001e8d130
    0000000000002db2 000001007ff9a850 0000010001e8d448 0000000000000001
    0000000000010000 0000010001e8d130
    Call Trace:<ffffffff80146273>{do_sigaction+659} <ffffffff80145361>{sigprocmask+241}
    <ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
    <ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
    <ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
    <ffffffff801115ef>{child_rip+0}
    kblockd/0 S 00003e224ad18831 0 40 6 41 9 (L-TLB)
    0000010001fb9e58 0000000000000046 0000010040e59800 000001007fec29d0
    000000000000010b ffffffff803e23e0 000001007fec2ce8 0000000000000001
    0000000000000003 0000000000000000
    Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
    <ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
    <ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
    <ffffffff801115ef>{child_rip+0}
    kblockd/1 S 00003e266f08347a 0 41 6 54 40 (L-TLB)
    000001007fe95e58 0000000000000046 0000007300000202 000001007fec21b0
    000000000000004f 0000010001fe8a50 000001007fec24c8 0000000000000001
    0000000000000003 0000000000000000
    Call Trace:<ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
    <ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
    <ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
    <ffffffff801115ef>{child_rip+0}
    khubd S 000000001a81861e 0 42 1 56 7 (L-TLB)
    000001007fe63f08 0000000000000046 0000000000000000 000001007febf230
    0000000000002cc2 000001007ff9a850 000001007febf548 0000000000000000
    0000000000000000 0000000000000216
    Call Trace:<ffffffff802d18b1>{hub_thread+161} <ffffffff80133440>{default_wake_function+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff802d1810>{hub_thread+0}
    <ffffffff801115ef>{child_rip+0}
    pdflush S 00000000242315c5 0 54 6 55 41 (L-TLB)
    0000010001f77e98 0000000000000046 0000006a01e0f760 0000010001fce2f0
    0000000000000622 0000010040e410f0 0000010001fce608 0000010040e410f0
    0000010001e0f760 0000010001e100a0
    Call Trace:<ffffffff8015cae0>{pdflush+0} <ffffffff8015c8c5>{__pdflush+261}
    <ffffffff8015cae0>{pdflush+0} <ffffffff8015cafc>{pdflush+28}
    <ffffffff8015cae0>{pdflush+0} <ffffffff8014e502>{kthread+146}
    <ffffffff801115f7>{child_rip+8} <ffffffff8014e540>{keventd_create_kthread+0}
    <ffffffff8014e470>{kthread+0} <ffffffff801115ef>{child_rip+0}

    pdflush S 00003e266dbb5896 0 55 6 58 54 (L-TLB)
    000001007fe5fe98 0000000000000046 0000007300000000 0000010001fceb10
    000000000000055d 0000010001fe8a50 0000010001fcee28 0000000000000000
    000001007fe5fe18 00000000000003cb
    Call Trace:<ffffffff8015cae0>{pdflush+0} <ffffffff8015c8c5>{__pdflush+261}
    <ffffffff8015cae0>{pdflush+0} <ffffffff8015cafc>{pdflush+28}
    <ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
    <ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
    <ffffffff801115ef>{child_rip+0}
    kswapd1 S 00003e1d6861ae1a 0 56 1 57 42 (L-TLB)
    0000010001f75eb8 0000000000000046 0000000000000000 0000010001fcf330
    0000000000022d38 000001007ff9a850 0000010001fcf648 0000000000000202
    00000000000000ef 0000000000000000
    Call Trace:<ffffffff80164af2>{balance_pgdat+2} <ffffffff80164e6f>{kswapd+239}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff80131321>{schedule_tail+17} <ffffffff801115f7>{child_rip+8}
    <ffffffff80164d80>{kswapd+0} <ffffffff801115ef>{child_rip+0}

    kswapd0 S 0000000000000000 0 57 1 170 56 (L-TLB)
    000001007fe5deb8 0000000000000046 0000007500000000 000001007feba2b0
    00000000000227e7 0000010001e8c910 000001007feba5c8 0000000000000202
    00000000000000eb 0000000000000100
    Call Trace:<ffffffff80164e6f>{kswapd+239} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff80131321>{schedule_tail+17}
    <ffffffff801115f7>{child_rip+8} <ffffffff80164d80>{kswapd+0}
    <ffffffff801115ef>{child_rip+0}
    aio/0 S 0000000000000000 0 58 6 59 55 (L-TLB)
    000001007fe4be58 0000000000000046 0000006b7fe4be18 000001007febaad0
    000000000000044b 0000010040e410f0 000001007febade8 0000000000000001
    0000000000010000 000001007febaad0
    Call Trace:<ffffffff80146273>{do_sigaction+659} <ffffffff80145361>{sigprocmask+241}
    <ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
    <ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
    <ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
    <ffffffff801115ef>{child_rip+0}
    aio/1 S 0000000024252125 0 59 6 58 (L-TLB)
    0000010001f73e58 0000000000000046 0000010001f73e18 000001007febb2f0
    000000000000147f 000001007ff9a850 000001007febb608 0000000000000001
    0000000000010000 000001007febb2f0
    Call Trace:<ffffffff80146273>{do_sigaction+659} <ffffffff80145361>{sigprocmask+241}
    <ffffffff80149df5>{worker_thread+261} <ffffffff803774bc>{thread_return+41}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80149cf0>{worker_thread+0} <ffffffff80149cf0>{worker_thread+0}
    <ffffffff8014e502>{kthread+146} <ffffffff801115f7>{child_rip+8}
    <ffffffff8014e540>{keventd_create_kthread+0} <ffffffff8014e470>{kthread+0}
    <ffffffff801115ef>{child_rip+0}
    scsi_eh_0 S 000000005fdbff82 0 170 1 188 57 (L-TLB)
    000001003fdcde28 0000000000000046 0000000000000246 000001007fec71b0
    0000000000003031 000001007ff9a850 000001007fec74c8 0000000035c1a774
    0000000000000000 0000000000000000
    Call Trace:<ffffffff80376d2d>{__down_interruptible+301} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80133493>{__wake_up_common+67} <ffffffff8037806b>{__down_failed_interruptible+53}
    <ffffffff80298e53>{.text.lock.scsi_error+213} <ffffffff801115f7>{child_rip+8}
    <ffffffff802989b0>{scsi_error_handler+0} <ffffffff801115ef>{child_rip+0}

    scsi_eh_1 S 00000002446c8a8a 0 188 1 218 170 (L-TLB)
    000001003fdade28 0000000000000046 0000000000000246 000001007feea950
    0000000000002664 000001007ff9a850 000001007feeac68 000000003cab9668
    0000000000000000 0000000000000000
    Call Trace:<ffffffff80376d2d>{__down_interruptible+301} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80133493>{__wake_up_common+67} <ffffffff8037806b>{__down_failed_interruptible+53}
    <ffffffff80298e53>{.text.lock.scsi_error+213} <ffffffff801115f7>{child_rip+8}
    <ffffffff802989b0>{scsi_error_handler+0} <ffffffff801115ef>{child_rip+0}

    scsi_eh_2 S 0000000316deda1b 0 218 1 230 188 (L-TLB)
    0000010001fd3e28 0000000000000046 0000000000000246 000001007feeb170
    0000000000002588 000001007ff9a850 000001007feeb488 000000003ccbede4
    0000000000000000 0000000000000000
    Call Trace:<ffffffff80376d2d>{__down_interruptible+301} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80133493>{__wake_up_common+67} <ffffffff8037806b>{__down_failed_interruptible+53}
    <ffffffff80298e53>{.text.lock.scsi_error+213} <ffffffff801115f7>{child_rip+8}
    <ffffffff802989b0>{scsi_error_handler+0} <ffffffff801115ef>{child_rip+0}

    kseriod S 0000000416e9c96d 0 230 1 238 218 (L-TLB)
    000001007feb9ef8 0000000000000046 0000006900000000 000001007fec6170
    00000000178bebc3 0000010040e400b0 000001007fec6488 0000000000000001
    0000000000000000 0000000000000000
    Call Trace:<ffffffff802edc34>{serio_thread+244} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80131321>{schedule_tail+17} <ffffffff801115f7>{child_rip+8}
    <ffffffff802edb40>{serio_thread+0} <ffffffff801115ef>{child_rip+0}

    md0_raid5 S 00003e2670610809 0 238 1 239 230 (L-TLB)
    000001007f7fdee8 0000000000000046 0000000000000202 0000010001fe8a50
    0000000000000088 000001007ff9a850 0000010001fe8d68 000001003ffd5948
    000001003ffd5980 000001003ffd59f4
    Call Trace:<ffffffff802fdcaa>{raid5d+490} <ffffffff80303dc1>{md_thread+417}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80131321>{schedule_tail+17}
    <ffffffff802fdac0>{raid5d+0} <ffffffff801115f7>{child_rip+8}
    <ffffffff802fdac0>{raid5d+0} <ffffffff80303c20>{md_thread+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 00003e255708fbc8 0 239 1 560 238 (L-TLB)
    000001007f7dde38 0000000000000046 000000747f7ddde8 000001007feea130
    00000000000002d8 ffffffff803e23e0 000001007feea448 0000000000000001
    0000000000000003 000001003ffbdac8
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 000026122613b858 0 560 1 561 239 (L-TLB)
    000001003f309e38 0000000000000046 000001007eafbec0 000001007f7a8b50
    00000000000032f9 ffffffff803e23e0 000001007f7a8e68 0000000000000001
    0000000000000003 000001003ffaccc8
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 00002b440172e760 0 561 1 562 560 (L-TLB)
    000001003e7b7e38 0000000000000046 000000763e7b7de8 000001007f6c73f0
    0000000000001ffe 000001007ff9a850 000001007f6c7708 0000000000000001
    0000000000000003 000001003ffacac8
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 00002760eb1bb445 0 562 1 563 561 (L-TLB)
    000001003e271e38 0000000000000046 000001003e271de8 000001007f6c63b0
    00000000000021e0 000001007ff9a850 000001007f6c66c8 0000000000000001
    0000000000000003 000001003ffac8c8
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 563 1 564 562 (L-TLB)
    000001003e03fe38 0000000000000046 000000763e03fde8 000001007f63ac50
    00000000000016a2 000001007f6904b0 000001007f63af68 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 564 1 565 563 (L-TLB)
    000001003e7fde38 0000000000000046 000000763e7fdde8 000001007f64e3f0
    00000000000016b9 000001007f6904b0 000001007f64e708 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 00003e2557d33302 0 565 1 566 564 (L-TLB)
    000001003dff3e38 0000000000000046 0000010067094560 000001007f6334b0
    00000000000006d3 000001007ff9a850 000001007f6337c8 0000000000000001
    0000000000000003 000001003ffac2c8
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 566 1 567 565 (L-TLB)
    000001003dff1e38 0000000000000046 000000763dff1de8 000001007f6d4b90
    0000000000001813 000001007f6904b0 000001007f6d4ea8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 567 1 568 566 (L-TLB)
    000001003f43fe38 0000000000000046 000000763f43fde8 000001007f63b470
    000000000000164e 000001007f6904b0 000001007f63b788 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 00003e20834b0b52 0 568 1 569 567 (L-TLB)
    000001003f271e38 0000000000000046 000000763f271de8 000001007f632470
    0000000000000406 000001007ff9a850 000001007f632788 0000000000000001
    0000000000000003 000001007feb74c8
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 569 1 570 568 (L-TLB)
    000001003e583e38 0000000000000046 000000763e583de8 000001007f6914f0
    000000000000184a 000001007f6904b0 000001007f691808 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 570 1 571 569 (L-TLB)
    000001003e0fde38 0000000000000046 000000763e0fdde8 000001007f690cd0
    0000000000001662 000001007f6904b0 000001007f690fe8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 571 1 572 570 (L-TLB)
    000001003ff83e38 0000000000000046 000000763ff83de8 000001007f632c90
    0000000000001843 000001007f6904b0 000001007f632fa8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 572 1 573 571 (L-TLB)
    000001003ff81e38 0000000000000046 000000763ff81de8 000001007f641530
    0000000000001827 000001007f6904b0 000001007f641848 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 573 1 574 572 (L-TLB)
    000001003e69fe38 0000000000000046 000000763e69fde8 000001007f63a430
    0000000000001767 000001007f6904b0 000001007f63a748 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 574 1 575 573 (L-TLB)
    000001003e90de38 0000000000000046 000000763e90dde8 0000010001fe8230
    0000000000001658 000001007f6904b0 0000010001fe8548 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 575 1 576 574 (L-TLB)
    000001003e7bbe38 0000000000000046 000000763e7bbde8 000001007f7a9370
    000000000000176b 000001007f6904b0 000001007f7a9688 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 576 1 577 575 (L-TLB)
    000001003e389e38 0000000000000046 000000763e389de8 0000010001fe9270
    0000000000001945 000001007f6904b0 0000010001fe9588 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 00000005ebfdef22 0 577 1 578 576 (L-TLB)
    000001003e34fe38 0000000000000046 000001003e34fde8 000001007f7a8330
    0000000001bb1857 000001007ff9a850 000001007f7a8648 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 000000064d0606d4 0 578 1 579 577 (L-TLB)
    000001003e349e38 0000000000000046 000001003e349de8 000001003e34b570
    000000003ccbf700 000001007ff9a850 000001003e34b888 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 579 1 580 578 (L-TLB)
    000001003e64fe38 0000000000000046 000000763e64fde8 000001003e34ad50
    00000000000023f3 000001007f6904b0 000001003e34b068 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 580 1 581 579 (L-TLB)
    000001003dffde38 0000000000000046 000000763dffdde8 000001003e34a530
    0000000000001bf7 000001007f6904b0 000001003e34a848 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 581 1 582 580 (L-TLB)
    000001003f28fe38 0000000000000046 000000763f28fde8 000001003e0f15b0
    0000000000001b47 000001007f6904b0 000001003e0f18c8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 582 1 583 581 (L-TLB)
    000001003e7a5e38 0000000000000046 000000763e7a5de8 000001003e0f0d90
    0000000000001936 000001007f6904b0 000001003e0f10a8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 583 1 584 582 (L-TLB)
    000001003eaa3e38 0000000000000046 000000763eaa3de8 000001003e0f0570
    0000000000001929 000001007f6904b0 000001003e0f0888 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 584 1 585 583 (L-TLB)
    000001003e73de38 0000000000000046 000000763e73dde8 000001003e73f5f0
    0000000000001784 000001007f6904b0 000001003e73f908 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 585 1 586 584 (L-TLB)
    000001003e943e38 0000000000000046 000000763e943de8 000001003e73edd0
    0000000000001720 000001007f6904b0 000001003e73f0e8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 586 1 587 585 (L-TLB)
    000001003e1d9e38 0000000000000046 000000763e1d9de8 000001003e73e5b0
    00000000000018b2 000001007f6904b0 000001003e73e8c8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 587 1 588 586 (L-TLB)
    000001003e153e38 0000000000000046 000000763e153de8 000001003e155630
    00000000000018de 000001007f6904b0 000001003e155948 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 588 1 589 587 (L-TLB)
    000001003e2f1e38 0000000000000046 000000763e2f1de8 000001003e154e10
    000000000000194e 000001007f6904b0 000001003e155128 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 589 1 590 588 (L-TLB)
    000001003f1e7e38 0000000000000046 000000763f1e7de8 000001003e1545f0
    00000000000016eb 000001007f6904b0 000001003e154908 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 590 1 591 589 (L-TLB)
    000001003f1e1e38 0000000000000046 000000773f1e1de8 000001003f1e3670
    000000000000181b 000001007f6904b0 000001003f1e3988 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 591 1 592 590 (L-TLB)
    000001003e577e38 0000000000000046 000000773e577de8 000001003f1e2e50
    000000000000179c 000001007f6904b0 000001003f1e3168 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 592 1 593 591 (L-TLB)
    000001003e455e38 0000000000000046 000000773e455de8 000001003f1e2630
    00000000000016ba 000001007f6904b0 000001003f1e2948 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 593 1 594 592 (L-TLB)
    000001003e39fe38 0000000000000046 000000773e39fde8 000001003e4516b0
    00000000000017b9 000001007f6904b0 000001003e4519c8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 594 1 595 593 (L-TLB)
    000001003e1bde38 0000000000000046 000000773e1bdde8 000001003e450e90
    00000000000015fd 000001007f6904b0 000001003e4511a8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 00000006691aead7 0 595 1 596 594 (L-TLB)
    000001003e4dbe38 0000000000000046 000001003e4dbde8 000001003e450670
    000000000b3c589b 000001007ff9a850 000001003e450988 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    kjournald S 0000000000000000 0 596 1 690 595 (L-TLB)
    000001003e4d9e38 0000000000000046 000000773e4d9de8 000001007f6c6bd0
    0000000000001c3a 000001007f6904b0 000001007f6c6ee8 0000000000000001
    0000000000000003 0000000000000008
    Call Trace:<ffffffff801cf1ec>{kjournald+780} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801ceec0>{commit_timeout+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801ceee0>{kjournald+0}
    <ffffffff801115ef>{child_rip+0}
    syslogd S 0000000000000000 0 690 1 693 596 (NOTLB)
    000001007f687d88 0000000000000006 0000007400000000 000001003e4a8ed0
    0000000000000fdd 000001003e4a96f0 000001003e4a91e8 0000000000000216
    0000000000000216 ffffffff8015a512
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff80313e45>{datagram_poll+21} <ffffffff801918ca>{do_select+906}
    <ffffffff80191380>{__pollwait+0} <ffffffff801ab8a1>{compat_sys_select+1105}
    <ffffffff80123f81>{ia32_sysret+0}
    klogd R running task 0 693 1 696 690 (NOTLB)
    rpc.statd S 0000275d80f36f11 0 696 1 707 693 (NOTLB)
    000001003e3bfd88 0000000000000006 0000010034f5a3a0 000001003e931730
    0000000000001f26 ffffffff803e23e0 000001003e931a48 0000000000000216
    0000000000000216 ffffffff8015a512
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff80330341>{tcp_poll+33} <ffffffff801918ca>{do_select+906}
    <ffffffff80191380>{__pollwait+0} <ffffffff801ab8a1>{compat_sys_select+1105}
    <ffffffff80123f81>{ia32_sysret+0}
    exim S 00003c9d5e980681 0 707 1 714 696 (NOTLB)
    000001003e7c5d88 0000000000000006 000001000000d700 000001003e308f50
    0000000000007e6c ffffffff803e23e0 000001003e309268 ffffffff8015a512
    000001000000d700 0000000000000001
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff80330341>{tcp_poll+33} <ffffffff801918ca>{do_select+906}
    <ffffffff80191380>{__pollwait+0} <ffffffff80154c01>{compat_sys_wait4+65}
    <ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}

    inetd S 0000000000000000 0 714 1 720 707 (NOTLB)
    000001003e05bd88 0000000000000006 0000007800000000 000001003e49d070
    00000000002a0f7f 000001003e308730 000001003e49d388 0000000000000216
    0000000000000216 ffffffff8015a512
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff80330341>{tcp_poll+33} <ffffffff801918ca>{do_select+906}
    <ffffffff80191380>{__pollwait+0} <ffffffff8016c109>{unmap_region+409}
    <ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}

    rngd S 00003e2325ef73e8 0 720 1 730 714 (NOTLB)
    0000010040f27e68 0000000000000006 00000010ffffd36c 000001003e308730
    0000000000000d85 ffffffff803e23e0 000001003e308a48 0000000000000246
    ffffffff80443da0 0000000000000001
    Call Trace:<ffffffff8015a555>{__get_free_pages+37} <ffffffff80377ef4>{schedule_timeout+180}
    <ffffffff80141b20>{process_timeout+0} <ffffffff80192011>{do_poll+177}
    <ffffffff8019219e>{sys_poll+350} <ffffffff80191380>{__pollwait+0}
    <ffffffff80123f81>{ia32_sysret+0}
    lpd S 0000009c08291736 0 730 1 755 720 (NOTLB)
    000001003e63dd88 0000000000000006 000001000000d700 000001003e49c850
    0000000000002d11 ffffffff803e23e0 000001003e49cb68 ffffffff8015a512
    000001000000d700 0000000000000001
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff8018a812>{pipe_poll+50} <ffffffff801918ca>{do_select+906}
    <ffffffff80191380>{__pollwait+0} <ffffffff80154c01>{compat_sys_wait4+65}
    <ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}

    smartd S 00003c9e89326754 0 755 1 761 730 (NOTLB)
    000001003e11fee8 0000000000000006 000001003ffba600 000001003e1210b0
    00000000000035ab ffffffff803e23e0 000001003e1213c8 0000000000000246
    0000000000000009 ffffffff802761db
    Call Trace:<ffffffff802761db>{blkdev_ioctl+955} <ffffffff80190cf6>{sys_ioctl+678}
    <ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
    <ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}

    sshd S 00003e224ac5a392 0 761 1 1090 764 755 (NOTLB)
    000001003e80bd88 0000000000000006 000001000000d700 000001003e120890
    000000000000c315 ffffffff803e23e0 000001003e120ba8 ffffffff8015a512
    000001000000d700 0000000000000001
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff80330341>{tcp_poll+33} <ffffffff801918ca>{do_select+906}
    <ffffffff80191380>{__pollwait+0} <ffffffff80154c01>{compat_sys_wait4+65}
    <ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}

    ntpd S 00003e276959b563 0 764 1 768 761 (NOTLB)
    000001007ed8fd88 0000000000000006 0000007940fc0830 000001003e120070
    0000000000000421 000001007ff9a850 000001003e120388 0000000000000216
    0000000000000216 ffffffff8015a512
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff80313e45>{datagram_poll+21} <ffffffff801918ca>{do_select+906}
    <ffffffff80191380>{__pollwait+0} <ffffffff8011084f>{do_signal+175}
    <ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}

    atd S 00003afba35fb0da 0 768 1 771 764 (NOTLB)
    000001007f407ee8 0000000000000006 0000000000000000 000001007f6d4370
    0000000000004080 000001007ff9a850 000001007f6d4688 0000000000000246
    0000000000000000 ffffffff80186c64
    Call Trace:<ffffffff80186c64>{vfs_stat+68} <ffffffff80141011>{del_timer_sync+49}
    <ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
    <ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}

    cron S 00003e21579855b4 0 771 1 775 768 (NOTLB)
    000001007f405ee8 0000000000000006 0000000000000000 000001007f6404f0
    0000000000001261 ffffffff803e23e0 000001007f640808 0000000000000246
    0000000000000000 ffffffff80186c64
    Call Trace:<ffffffff80186c64>{vfs_stat+68} <ffffffff80141011>{del_timer_sync+49}
    <ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
    <ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}

    irqbalance S 00003e270edcc3d0 0 775 1 785 771 (NOTLB)
    000001003ea49ee8 0000000000000006 0000000000000212 000001003e29e0b0
    0000000000006564 000001007ff9a850 000001003e29e3c8 0000000000000246
    0000000000000216 0000010060208358
    Call Trace:<ffffffff8016a7c7>{remove_vm_struct+231} <ffffffff8016c109>{unmap_region+409}
    <ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
    <ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}

    zsh R running task 0 785 1 1797 775 (NOTLB)
    sshd S 0000000000000000 0 1090 761 1092 1101 (NOTLB)
    000001003ea33c18 0000000000000006 0000007500001000 000001003ea35130
    00000000000003a1 0000010001e8c0f0 000001003ea35448 ffffffff80382270
    000001003e180520 0000000000000374
    Call Trace:<ffffffff801c470d>{__ext3_journal_stop+45} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff801582b7>{generic_file_aio_write_nolock+2055}
    <ffffffff8035dc57>{unix_stream_data_wait+263} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff8035de31>{unix_stream_recvmsg+369}
    <ffffffff8030d5a4>{sock_aio_read+180} <ffffffff8017bb93>{do_sync_read+115}
    <ffffffff8017ae77>{dentry_open+263} <ffffffff8017ad52>{filp_open+66}
    <ffffffff8017bcba>{vfs_read+218} <ffffffff8017bf19>{sys_read+73}
    <ffffffff80123f81>{ia32_sysret+0}
    sshd S 0000000000000000 0 1092 1090 1093 (NOTLB)
    000001003e9edd88 0000000000000006 000000750000d700 0000010001e8c0f0
    0000000000000c00 000001007f640d10 0000010001e8c408 ffffffff8015a512
    0000000008098450 0000000000000001
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff8024e52b>{pty_write_room+43} <ffffffff8024d521>{normal_poll+305}
    <ffffffff801918ca>{do_select+906} <ffffffff80191380>{__pollwait+0}
    <ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}

    zsh S 00003d93b2c81863 0 1093 1092 1119 (NOTLB)
    000001003de79f18 0000000000000006 0000000000000216 000001003dece9d0
    0000000000001398 ffffffff803e23e0 000001003decece8 000001003e6b44c0
    000001003e6b44c0 ffffffff80145361
    Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff801958d1>{dput+33}
    <ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
    <ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}

    sshd S 0000000000000000 0 1101 761 1103 1090 (NOTLB)
    000001003de0fc18 0000000000000006 0000007600001000 000001003e29f0f0
    00000000000003f5 000001003decf1f0 000001003e29f408 ffffffff80382270
    000001003e180a00 0000000000000374
    Call Trace:<ffffffff801c470d>{__ext3_journal_stop+45} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff801582b7>{generic_file_aio_write_nolock+2055}
    <ffffffff8035dc57>{unix_stream_data_wait+263} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff8035de31>{unix_stream_recvmsg+369}
    <ffffffff8030d5a4>{sock_aio_read+180} <ffffffff8017bb93>{do_sync_read+115}
    <ffffffff8017ae77>{dentry_open+263} <ffffffff8017ad52>{filp_open+66}
    <ffffffff8017bcba>{vfs_read+218} <ffffffff8017bf19>{sys_read+73}
    <ffffffff80123f81>{ia32_sysret+0}
    sshd S 00003e242c914ec0 0 1103 1101 1104 (NOTLB)
    000001003dc05d88 0000000000000006 000000740000d700 000001003decf1f0
    0000000000000e87 000001003e29e8d0 000001003decf508 ffffffff8015a512
    0000000008098430 0000000000000001
    Call Trace:<ffffffff8015a512>{__alloc_pages+818} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff8024e52b>{pty_write_room+43} <ffffffff8024d521>{normal_poll+305}
    <ffffffff801918ca>{do_select+906} <ffffffff80191380>{__pollwait+0}
    <ffffffff801ab8a1>{compat_sys_select+1105} <ffffffff80123f81>{ia32_sysret+0}

    zsh S 00003e242c9c72d2 0 1104 1103 8825 (NOTLB)
    000001003dba3f18 0000000000000006 0000000000000216 000001003e29e8d0
    0000000000009ded ffffffff803e23e0 000001003e29ebe8 000001003e6b4b80
    000001003e6b4b80 ffffffff80145361
    Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff801958d1>{dput+33}
    <ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
    <ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}

    zsh S 00003e1fc58e116c 0 1119 1093 8824 8727 (NOTLB)
    000001003da01f18 0000000000000006 0000000000000216 000001003ea340f0
    000000000000179b ffffffff803e23e0 000001003ea34408 000001003e6b4940
    000001003e6b4940 ffffffff80145361
    Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff801958d1>{dput+33}
    <ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
    <ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}

    apcupsd S 00000a987aa07cb0 0 1797 1 1896 4010 785 (NOTLB)
    000001003d571de8 0000000000000006 000001003e2903d8 000001003e9306f0
    000000000000b624 ffffffff803e23e0 000001003e930a08 ffffffff801584eb
    00000000ffffdbdc 0000000000000038
    Call Trace:<ffffffff801584eb>{generic_file_aio_write+107} <ffffffff801bbbf3>{ext3_file_write+35}
    <ffffffff8013bdb2>{sys_wait4+642} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80154c01>{compat_sys_wait4+65}
    <ffffffff80146273>{do_sigaction+659} <ffffffff8017bea7>{vfs_write+247}
    <ffffffff8017bf99>{sys_write+73} <ffffffff80123f81>{ia32_sysret+0}

    apcupsd R running task 0 1896 1797 (NOTLB)
    portmap S 00003c7c39328cc8 0 4010 1 4021 1797 (NOTLB)
    000001007f2e1e68 0000000000000006 0000001000000216 0000010034d8a3b0
    0000000000001647 000001007ff9a850 0000010034d8a6c8 0000000000000246
    000001003e293730 0000000000000006
    Call Trace:<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
    <ffffffff80192011>{do_poll+177} <ffffffff8019219e>{sys_poll+350}
    <ffffffff80191380>{__pollwait+0} <ffffffff80123f81>{ia32_sysret+0}

    nfsd S 00003d33ef5e1212 0 4021 1 4022 4010 (L-TLB)
    0000010034c71d98 0000000000000046 0000007340a62e88 000001007f6904b0
    0000000000001dff 000001003e4a86b0 000001007f6907c8 0000000000000246
    0000000000000256 0000010034c71d58
    Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
    <ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}

    nfsd S 00003d33ef5e1d69 0 4022 1 4025 4021 (L-TLB)
    0000010034badd98 0000000000000046 0000007301b7b698 000001003e4a86b0
    0000000000000122 ffffffff803e23e0 000001003e4a89c8 0000000000000246
    000001003fd554a0 0000000000000216
    Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
    <ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}

    nfsd S 00003e0570804991 0 4025 1 4023 4022 (L-TLB)
    0000010034c09d98 0000000000000046 0000010001b79788 000001003e309770
    0000000000003076 000001007ff9a850 000001003e309a88 0000000000000246
    000001003fd434a0 0000000000000216
    Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
    <ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}

    nfsd S 00003e057080e3f4 0 4023 1 4027 4025 (L-TLB)
    000001003e08fd98 0000000000000046 00000100405be440 000001007febea10
    0000000000001712 ffffffff803e23e0 000001007febed28 0000000000000246
    000001007fe7c4a0 000001007f6b8cc0
    Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
    <ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}

    nfsd S 00003b3d1f1b4de6 0 4027 1 4024 4023 (L-TLB)
    000001003e04bd98 0000000000000046 0000007340db8990 000001007fec6990
    000000000000019c 000001007ff9a850 000001007fec6ca8 0000000000000246
    000001007ffb24a0 0000000000000216
    Call Trace:<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
    <ffffffff8036e0fb>{svc_recv+987} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80145361>{sigprocmask+241}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801f8f87>{nfsd+471}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801115f7>{child_rip+8}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801115ef>{child_rip+0}
    nfsd S 00003b90f03a78c3 0 4024 1 4026 4027 (L-TLB)
    0000010001e77d98 0000000000000046 0000007301b8b030 000001007fec31f0
    00000000000000ab ffffffff803e23e0 000001007fec3508 0000000000000246
    000001007fe814a0 0000000000000216
    Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
    <ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}

    nfsd S 00003b3d1f1b6fe4 0 4026 1 4028 4024 (L-TLB)
    000001003e6bfd98 0000000000000046 0000010001b7e610 000001003e49c030
    000000000000066f ffffffff803e23e0 000001003e49c348 0000000000000246
    000001007ffb44a0 000001007f6b8cc0
    Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
    <ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}

    nfsd S 00003b90f03a7214 0 4028 1 4030 4026 (L-TLB)
    000001003e3bdd98 0000000000000046 0000007340d8c188 0000010034d8b3f0
    0000000000000940 000001007fec31f0 0000010034d8b708 0000000000000246
    000001007fe734a0 000001007f6b8cc0
    Call Trace:<ffffffff8015a1c3>{buffered_rmqueue+515} <ffffffff80377ef4>{schedule_timeout+180}
    <ffffffff80141b20>{process_timeout+0} <ffffffff8036e0fb>{svc_recv+987}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80145361>{sigprocmask+241} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8f87>{nfsd+471} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff801f8db0>{nfsd+0}
    <ffffffff801f8db0>{nfsd+0} <ffffffff801115ef>{child_rip+0}

    lockd S 0000000000000000 0 4030 1 4031 4028 (L-TLB)
    000001007ed13de8 0000000000000046 000000763e61a330 000001003e61a330
    000000000000111d 000001003e61ab50 000001003e61a648 ffffffff803774bc
    0000007300000001 0000000000000212
    Call Trace:<ffffffff803774bc>{thread_return+41} <ffffffff80377e65>{schedule_timeout+37}
    <ffffffff8036c679>{svc_sock_release+569} <ffffffff8036e0fb>{svc_recv+987}
    <ffffffff80133440>{default_wake_function+0} <ffffffff80133440>{default_wake_function+0}
    <ffffffff80378031>{__down_failed+53} <ffffffff80133440>{default_wake_function+0}
    <ffffffff8020a6d9>{lockd+377} <ffffffff8020a560>{lockd+0}
    <ffffffff801115f7>{child_rip+8} <ffffffff8020a560>{lockd+0}
    <ffffffff8020a560>{lockd+0} <ffffffff801115ef>{child_rip+0}

    rpciod S 0000000000000000 0 4031 1 4033 4030 (L-TLB)
    000001007e893e88 0000000000000046 0000007440000000 000001007ed114f0
    00000000000012cf 000001007f6904b0 000001007ed11808 000001003e5e8668
    ffffffff8020a560 0000000000000008
    Call Trace:<ffffffff8020a560>{lockd+0} <ffffffff803780df>{__up_wakeup+53}
    <ffffffff8020a560>{lockd+0} <ffffffff8036952a>{rpciod+1050}
    <ffffffff801358f0>{autoremove_wake_function+0} <ffffffff801358f0>{autoremove_wake_function+0}
    <ffffffff80131321>{schedule_tail+17} <ffffffff801115f7>{child_rip+8}
    <ffffffff8020a560>{lockd+0} <ffffffff80369110>{rpciod+0}
    <ffffffff801115ef>{child_rip+0}
    rpc.mountd S 000027aa351b4f23 0 4033 1 4031 (NOTLB)
    000001003e4c5e68 0000000000000006 0000001080000000 000001007f64f430
    00000000000000ef ffffffff803e23e0 000001007f64f748 0000000000000246
    0000010040f5f8f0 0000000000000002
    Call Trace:<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
    <ffffffff80192011>{do_poll+177} <ffffffff8019219e>{sys_poll+350}
    <ffffffff80191380>{__pollwait+0} <ffffffff80123f81>{ia32_sysret+0}

    zsh S 00003e14d482c249 0 8727 1093 8775 1119 (NOTLB)
    0000010024b3ff18 0000000000000006 0000000000000216 000001003e61b370
    0000000000001960 ffffffff803e23e0 000001003e61b688 000001003e6b4940
    000001003e6b4940 ffffffff80145361
    Call Trace:<ffffffff80145361>{sigprocmask+241} <ffffffff801958d1>{dput+33}
    <ffffffff80123f81>{ia32_sysret+0} <ffffffff8010fde9>{sys_rt_sigsuspend+297}
    <ffffffff801249fe>{sys32_rt_sigprocmask+206} <ffffffff801240d8>{ia32_ptregs_common+40}

    dump D 00003e14e808f7df 0 8775 8727 (NOTLB)
    000001002ef31bb8 0000000000000006 000001003ffd5948 000001007f640d10
    0000000000007df3 ffffffff803e23e0 000001007f641028 ffffffff802fd683
    0000000000000212 0000000000000001
    Call Trace:<ffffffff802fd683>{raid5_unplug_device+291} <ffffffff80377d3b>{io_schedule+43}
    <ffffffff80155f6a>{__lock_page+250} <ffffffff80155c40>{page_wake_function+0}
    <ffffffff80155c40>{page_wake_function+0} <ffffffff801567f6>{do_generic_mapping_read+502}
    <ffffffff80156a70>{file_read_actor+0} <ffffffff80156d14>{__generic_file_aio_read+372}
    <ffffffff80156dfb>{generic_file_read+123} <ffffffff80169994>{handle_mm_fault+292}
    <ffffffff80122ad8>{do_page_fault+440} <ffffffff80130948>{recalc_task_prio+424}
    <ffffffff803774bc>{thread_return+41} <ffffffff8017bca7>{vfs_read+199}
    <ffffffff8017bf19>{sys_read+73} <ffffffff80123f81>{ia32_sysret+0}

    sleep S 00003e1fc5e0bf41 0 8824 1119 (NOTLB)
    0000010054f3fee8 0000000000000006 00000100115c0aa8 000001007febe1f0
    00000000000e9920 000001007ff9a850 000001007febe508 0000000000000246
    00000000ffffd724 0000000000000014
    Call Trace:<ffffffff80377ef4>{schedule_timeout+180} <ffffffff80141b20>{process_timeout+0}
    <ffffffff801541e1>{compat_sys_nanosleep+193} <ffffffff80123f81>{ia32_sysret+0}

    ls D 00003e242d2c4ffe 0 8825 1104 (NOTLB)
    00000100349d5d38 0000000000000006 000001003ffd5948 000001003e61ab50
    00000000001349ac ffffffff803e23e0 000001003e61ae68 ffffffff802fd683
    0000000000000216 000001003e26ed58
    Call Trace:<ffffffff802fd683>{raid5_unplug_device+291} <ffffffff80377d3b>{io_schedule+43}
    <ffffffff8017d4f8>{__wait_on_buffer+168} <ffffffff8017d310>{bh_wake_function+0}
    <ffffffff8017d310>{bh_wake_function+0} <ffffffff801be1cb>{ext3_bread+91}
    <ffffffff801bb348>{ext3_readdir+232} <ffffffff80191120>{filldir64+0}
    <ffffffff80122ad8>{do_page_fault+440} <ffffffff8016b23b>{vma_merge+331}
    <ffffffff80190db7>{vfs_readdir+119} <ffffffff80191294>{sys_getdents64+132}
    <ffffffff80111441>{error_exit+0} <ffffffff80123f81>{ia32_sysret+0}


    -ryan
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-03-22 14:03    [W:0.111 / U:1.724 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site