Messages in this thread | | | Date | Sun, 22 Feb 2004 16:49:41 +0100 | From | Mikael Wahlberg <> | Subject | Filesystem kernel hangup, 2.6.3 (bad: scheduling while atomic!) |
| |
Description:
On heavy FTP Load (About 1Gbit/s) running both reads and writes on two ServeRAID6m Raid5 controllers merged together to one filesystem with Raidtools we see the error below. The filesystem gets totally hanged up. Currently with XFS, but JFS gets the same problem (Actually even more often).
Anybody has got a good idea what can be wrong?
Distribution: RedHat 9.0
Hardware: IBM x345 2x2.4GHz Xeon 2xServeRAID6m
Error: (/var/log/messages) Feb 22 15:00:51 mserv1 kernel: Unable to handle kernel NULL pointer dereference at virtual address 000002a6 Feb 22 15:00:52 mserv1 kernel: printing eip: Feb 22 15:00:52 mserv1 kernel: c011e5b5 Feb 22 15:00:52 mserv1 kernel: *pde = 00000000 Feb 22 15:00:52 mserv1 kernel: Oops: 0002 [#1] Feb 22 15:00:52 mserv1 kernel: CPU: 1 Feb 22 15:00:52 mserv1 kernel: EIP: 0060:[<c011e5b5>] Not tainted Feb 22 15:00:52 mserv1 kernel: EFLAGS: 00010003 Feb 22 15:00:52 mserv1 kernel: Process proftpd (pid: 7432, threadinfo=ee530000 task=ef59a6b0) Feb 22 15:00:52 mserv1 kernel: Stack: c011e54a ee531a28 00000003 00000000 ddcdfeec ee530000 ddcdfee8 00000292 Feb 22 15:00:53 mserv1 kernel: ee531a50 c011e5af ddcdfee8 00000003 00000001 00000000 ddcdfee0 ddcdfe50 Feb 22 15:00:53 mserv1 kernel: 00000000 f5e36000 c0259e62 c0259b00 000000c8 c023038e ddcdfee0 000b2f80 Feb 22 15:00:53 mserv1 kernel: Call Trace: Feb 22 15:00:53 mserv1 kernel: [<c011e54a>] __wake_up_common+0x3a/0x60 Feb 22 15:00:53 mserv1 kernel: [<c011e5af>] __wake_up+0x3f/0x70 Feb 22 15:00:53 mserv1 kernel: [<c0259e62>] mrunlock+0x82/0xb0 Feb 22 15:00:53 mserv1 kernel: [<c0259b00>] mraccessf+0xc0/0xe0 Feb 22 15:00:53 mserv1 kernel: [<c023038e>] xfs_iunlock+0x3e/0x80 Feb 22 15:00:53 mserv1 kernel: [<c023727b>] xfs_iomap+0x3bb/0x540 Feb 22 15:00:53 mserv1 kernel: [<c0163fc7>] bio_alloc+0xd7/0x1c0 Feb 22 15:00:53 mserv1 kernel: [<c025a17a>] map_blocks+0x7a/0x170 Feb 22 15:00:53 mserv1 kernel: [<c025b40b>] page_state_convert+0x52b/0x6d0 Feb 22 15:00:53 mserv1 kernel: [<c0236cb9>] xfs_imap_to_bmap+0x39/0x240 Feb 22 15:00:53 mserv1 kernel: [<c025be48>] linvfs_release_page+0xa8/0xb0 Feb 22 15:00:53 mserv1 kernel: [<c025bce0>] linvfs_writepage+0x60/0x120 Feb 22 15:00:53 mserv1 kernel: [<c014990c>] shrink_list+0x41c/0x710 Feb 22 15:00:53 mserv1 kernel: [<c0149df8>] shrink_cache+0x1f8/0x3d0 Feb 22 15:00:53 mserv1 kernel: [<c01b3a00>] journal_stop+0x220/0x330 Feb 22 15:00:53 mserv1 kernel: [<c014a6dc>] shrink_zone+0xbc/0xc0 Feb 22 15:00:53 mserv1 kernel: [<c014a7a5>] shrink_caches+0xc5/0xe0 Feb 22 15:00:54 mserv1 kernel: [<c014a87c>] try_to_free_pages+0xbc/0x190 Feb 22 15:00:54 mserv1 kernel: [<c0143043>] __alloc_pages+0x203/0x370 Feb 22 15:00:54 mserv1 kernel: [<c01431d5>] __get_free_pages+0x25/0x40 Feb 22 15:00:54 mserv1 kernel: [<c0173241>] __pollwait+0x41/0xd0 Feb 22 15:00:54 mserv1 kernel: [<c0358f93>] tcp_poll+0x33/0x190 Feb 22 15:00:54 mserv1 kernel: [<c032b0f9>] sock_poll+0x29/0x40 Feb 22 15:00:54 mserv1 kernel: [<c01736b7>] do_select+0x2f7/0x320 Feb 22 15:00:54 mserv1 kernel: [<c0173200>] __pollwait+0x0/0xd0 Feb 22 15:00:54 mserv1 kernel: [<c0173a12>] sys_select+0x302/0x540 Feb 22 15:00:54 mserv1 kernel: [<c010b08b>] syscall_call+0x7/0xb Feb 22 15:00:54 mserv1 kernel: Feb 22 15:00:54 mserv1 kernel: Code: ff 4b 14 8b 43 08 83 e0 08 75 0d 8b 5d f4 8b 75 f8 8b 7d fc Feb 22 15:00:54 mserv1 kernel: [<c0173a12>] sys_select+0x302/0x540 Feb 22 15:00:54 mserv1 kernel: [<c010b08b>] syscall_call+0x7/0xb Feb 22 15:00:54 mserv1 kernel: Feb 22 15:00:54 mserv1 kernel: Code: ff 4b 14 8b 43 08 83 e0 08 75 0d 8b 5d f4 8b 75 f8 8b 7d fc Feb 22 15:00:54 mserv1 kernel: <6>note: proftpd[7432] exited with preempt_count 2 Feb 22 15:00:54 mserv1 kernel: bad: scheduling while atomic! Feb 22 15:00:54 mserv1 kernel: Call Trace: Feb 22 15:00:54 mserv1 kernel: [<c011e48e>] schedule+0x6ee/0x700 Feb 22 15:00:54 mserv1 kernel: [<c014cdeb>] zap_pmd_range+0x4b/0x70 Feb 22 15:00:54 mserv1 kernel: [<c01591ba>] free_pages_and_swap_cache+0x6a/0xa0 Feb 22 15:00:54 mserv1 kernel: [<c014d0cc>] unmap_vmas+0x23c/0x2f0 Feb 22 15:00:54 mserv1 kernel: [<c0151774>] exit_mmap+0xf4/0x250 Feb 22 15:00:54 mserv1 kernel: [<c0120d2d>] mmput+0x6d/0xa0 Feb 22 15:00:54 mserv1 kernel: [<c0125a33>] do_exit+0x1a3/0x500 Feb 22 15:00:54 mserv1 kernel: [<c010c1ac>] die+0xfc/0x100 Feb 22 15:00:54 mserv1 kernel: [<c011b349>] do_page_fault+0x1f9/0x523 Feb 22 15:00:54 mserv1 kernel: [<c0141bab>] mempool_alloc+0x8b/0x190 Feb 22 15:00:54 mserv1 kernel: [<c011b150>] do_page_fault+0x0/0x523 Feb 22 15:00:54 mserv1 kernel: [<c010baf5>] error_code+0x2d/0x38 Feb 22 15:00:54 mserv1 kernel: [<c011e5b5>] __wake_up+0x45/0x70 Feb 22 15:00:54 mserv1 kernel: [<c011e54a>] __wake_up_common+0x3a/0x60 Feb 22 15:00:55 mserv1 kernel: [<c011e5af>] __wake_up+0x3f/0x70 Feb 22 15:00:55 mserv1 kernel: [<c0259e62>] mrunlock+0x82/0xb0 Feb 22 15:00:55 mserv1 kernel: [<c0259b00>] mraccessf+0xc0/0xe0 Feb 22 15:00:55 mserv1 kernel: [<c023038e>] xfs_iunlock+0x3e/0x80 Feb 22 15:00:55 mserv1 kernel: [<c023727b>] xfs_iomap+0x3bb/0x540 Feb 22 15:00:55 mserv1 kernel: [<c0163fc7>] bio_alloc+0xd7/0x1c0 Feb 22 15:00:55 mserv1 kernel: [<c025a17a>] map_blocks+0x7a/0x170 Feb 22 15:00:55 mserv1 kernel: [<c025b40b>] page_state_convert+0x52b/0x6d0 Feb 22 15:00:55 mserv1 kernel: [<c0236cb9>] xfs_imap_to_bmap+0x39/0x240 Feb 22 15:00:55 mserv1 kernel: [<c025be48>] linvfs_release_page+0xa8/0xb0 Feb 22 15:00:55 mserv1 kernel: [<c025bce0>] linvfs_writepage+0x60/0x120 Feb 22 15:00:55 mserv1 kernel: [<c014990c>] shrink_list+0x41c/0x710 Feb 22 15:00:55 mserv1 kernel: [<c0149df8>] shrink_cache+0x1f8/0x3d0 Feb 22 15:00:55 mserv1 kernel: [<c01b3a00>] journal_stop+0x220/0x330 Feb 22 15:00:55 mserv1 kernel: [<c014a6dc>] shrink_zone+0xbc/0xc0 Feb 22 15:00:55 mserv1 kernel: [<c014a7a5>] shrink_caches+0xc5/0xe0 Feb 22 15:00:55 mserv1 kernel: [<c014a87c>] try_to_free_pages+0xbc/0x190 Feb 22 15:00:55 mserv1 kernel: [<c0143043>] __alloc_pages+0x203/0x370 Feb 22 15:00:55 mserv1 kernel: [<c01431d5>] __get_free_pages+0x25/0x40 Feb 22 15:00:55 mserv1 kernel: [<c0173241>] __pollwait+0x41/0xd0 Feb 22 15:00:55 mserv1 kernel: [<c0358f93>] tcp_poll+0x33/0x190 Feb 22 15:00:55 mserv1 kernel: [<c032b0f9>] sock_poll+0x29/0x40 Feb 22 15:00:55 mserv1 kernel: [<c01736b7>] do_select+0x2f7/0x320 Feb 22 15:00:55 mserv1 kernel: [<c0173200>] __pollwait+0x0/0xd0 Feb 22 15:00:55 mserv1 kernel: [<c0173a12>] sys_select+0x302/0x540 Feb 22 15:00:55 mserv1 kernel: [<c010b08b>] syscall_call+0x7/0xb Feb 22 15:00:55 mserv1 kernel: Feb 22 15:00:55 mserv1 kernel: Unable to handle kernel NULL pointer dereference at virtual address 000002a6 Feb 22 15:00:55 mserv1 kernel: printing eip: Feb 22 15:00:55 mserv1 kernel: c011e5b5 Feb 22 15:00:55 mserv1 kernel: *pde = 00000000 Feb 22 15:00:55 mserv1 kernel: Oops: 0002 [#2] Feb 22 15:00:55 mserv1 kernel: CPU: 1 Feb 22 15:00:55 mserv1 kernel: EIP: 0060:[<c011e5b5>] Not tainted Feb 22 15:00:55 mserv1 kernel: EFLAGS: 00010003 Feb 22 15:00:55 mserv1 kernel: EIP is at __wake_up+0x45/0x70 Feb 22 15:00:55 mserv1 kernel: eax: ee531a00 ebx: 00000292 ecx: 00000001 edx: 00000003 Feb 22 15:00:55 mserv1 kernel: esi: ddcdfee8 edi: 00000001 ebp: c2ae5d80 esp: c2ae5d60 Feb 22 15:00:55 mserv1 kernel: ds: 007b es: 007b ss: 0068 Feb 22 15:00:55 mserv1 kernel: Process pdflush (pid: 20, threadinfo=c2ae4000 task=c2aead20) Feb 22 15:00:55 mserv1 kernel: Stack: c011e54a ee531a28 00000003 00000000 ddcdfeec c2ae4000 ddcdfee8 00000296 Feb 22 15:00:55 mserv1 kernel: c2ae5da4 c011e5af ddcdfee8 00000003 00000001 00000000 ddcdfee0 ddcdfe50 Feb 22 15:00:56 mserv1 kernel: e25319a0 f5e36000 c0259e62 00000008 00000008 c023038e ddcdfee0 ddcdfe50 Feb 22 15:00:56 mserv1 kernel: Call Trace: Feb 22 15:00:56 mserv1 kernel: [<c011e54a>] __wake_up_common+0x3a/0x60 Feb 22 15:00:56 mserv1 kernel: [<c011e5af>] __wake_up+0x3f/0x70 Feb 22 15:00:56 mserv1 kernel: [<c0259e62>] mrunlock+0x82/0xb0 Feb 22 15:00:56 mserv1 kernel: [<c023038e>] xfs_iunlock+0x3e/0x80 Feb 22 15:00:56 mserv1 kernel: [<c023554a>] xfs_iflush+0x36a/0x550 Feb 22 15:00:56 mserv1 kernel: [<c02573b5>] xfs_inode_flush+0x255/0x2c0 Feb 22 15:00:56 mserv1 kernel: [<c011e4f0>] default_wake_function+0x0/0x20 Feb 22 15:00:56 mserv1 kernel: [<c025bc80>] linvfs_writepage+0x0/0x120 Feb 22 15:00:56 mserv1 kernel: [<c02645b2>] linvfs_write_inode+0x32/0x40 Feb 22 15:00:56 mserv1 kernel: [<c0182806>] write_inode+0x46/0x50 Feb 22 15:00:56 mserv1 kernel: [<c0182a60>] __sync_single_inode+0x250/0x2a0 Feb 22 15:00:56 mserv1 kernel: [<c0182d4c>] sync_sb_inodes+0x1ac/0x2a0 Feb 22 15:00:56 mserv1 kernel: [<c0182ec9>] writeback_inodes+0x89/0x130 Feb 22 15:00:56 mserv1 kernel: [<c01442c8>] background_writeout+0xb8/0x100 Feb 22 15:00:56 mserv1 kernel: [<c0144afa>] __pdflush+0x10a/0x220 Feb 22 15:00:56 mserv1 kernel: [<c0144c10>] pdflush+0x0/0x20 Feb 22 15:00:56 mserv1 kernel: [<c0144c1f>] pdflush+0xf/0x20 Feb 22 15:00:56 mserv1 kernel: [<c0144210>] background_writeout+0x0/0x100 Feb 22 15:00:56 mserv1 kernel: [<c0108d14>] kernel_thread_helper+0x0/0xc Feb 22 15:00:56 mserv1 kernel: [<c0108d19>] kernel_thread_helper+0x5/0xc Feb 22 15:00:56 mserv1 kernel: Feb 22 15:00:56 mserv1 kernel: Code: ff 4b 14 8b 43 08 83 e0 08 75 0d 8b 5d f4 8b 75 f8 8b 7d fc Feb 22 15:00:56 mserv1 kernel: <6>note: pdflush[20] exited with preempt_count 3
/Mikael -- ----------------------------------------------------------------------- Mikael Wahlberg, M.Sc. Ardendo Unit Manager Professional Services/ e-mail: mikael@ardendo.se Technical Project Manager GSM: +46 733 279 274 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |