lkml.org 
[lkml]   [2004]   [Aug]   [1]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
Subject2.6.8-rc2 ext3 + nfsv3 fails assertion in __journal_drop_transaction()

Greetings!

first, the kernel in question is not a vanilla kernel,
it contains the linux-vserver patches + the namespace
fix (which didn't make it into 2.6.8-rc2), nevertheless
the issue seems not linux-vserver related (though I
might be completely wrong here ;)

patches used (aside from 2.6.8-rc2):

http://vserver.13thfloor.at/Experimental/patch-2.6.8-rc2-vs1.9.2.14.diff
http://vserver.13thfloor.at/Experimental/delta-2.6.8-rc2-vs1.9.2.14-vs1.9.2.15.diff
http://vserver.13thfloor.at/Experimental/namespace-fix-2.6.8-rc2.diff


the issue happens after some time of heavy load, where
the box running this kernel only acts as nfs server for
several clients ...

the failing assertion is always the same, so I would
rule out memory or hardware issues (this is on a dual
PIII machine, 2GB memory, dell Perc/DC raid card)

it was observed with and without PREEMPT and HIGHMEM
it usually happens after 1-4 hours, sometimes it hangs.
a sysrq dump for this case is attached ...
(maybe both issues are totally unrelated, maybe not)

mount options for the ext3 are: rw,tagxid,data=journal
and for the nfs: rw,tagxid,intr,tcp,nfsvers=3

any hints would be appreciated ...

TIA,
Herbert

----------

Assertion failure in __journal_drop_transaction() at fs/jbd/checkpoint.c:613: "transaction->t_forget == NULL"
------------[ cut here ]------------
kernel BUG at fs/jbd/checkpoint.c:613!
invalid operand: 0000 [#1]
PREEMPT SMP
Modules linked in: megaraid
CPU: 0
EIP: 0060:[__journal_drop_transaction+848/1015] Not tainted
EFLAGS: 00010286 (2.6.8-rc2-vs1.9.2.15+namespace-fix-filer)
EIP is at __journal_drop_transaction+0x350/0x3f7
eax: 00000071 ebx: dfe41e00 ecx: c03faf64 edx: c03faf64
esi: f7fc3200 edi: d089462c ebp: dfe41e00 esp: f6c1dd58
ds: 007b es: 007b ss: 0068
Process kjournald (pid: 127, threadinfo=f6c1c000 task=f7899340)
Stack: c03b1c00 c0395d20 c03a8e6e 00000265 c03a8ebe dfe41e00 f7fc3200 c01eb43a
f7fc3200 dfe41e00 dd5f883c 00000000 c01ea816 d089462c d089462c f6c1c000
c01eb3e8 d089462c d089462c 00000000 ebf9f300 c0cad300 f6c1c000 00000000
Call Trace:
[__journal_remove_checkpoint+74/160] __journal_remove_checkpoint+0x4a/0xa0
[__try_to_free_cp_buf+118/192] __try_to_free_cp_buf+0x76/0xc0
[__journal_clean_checkpoint_list+168/176] __journal_clean_checkpoint_list+0xa8/0xb0
[autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60
[journal_commit_transaction+700/5760] journal_commit_transaction+0x2bc/0x1680
[autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60
[kfree_skbmem+36/48] kfree_skbmem+0x24/0x30
[autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60
[net_tx_action+90/368] net_tx_action+0x5a/0x170
[move_tasks+371/608] move_tasks+0x173/0x260
[recalc_task_prio+168/416] recalc_task_prio+0xa8/0x1a0
[schedule+767/1616] schedule+0x2ff/0x650
[del_timer_sync+170/224] del_timer_sync+0xaa/0xe0
[kjournald+241/736] kjournald+0xf1/0x2e0
[autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60
[autoremove_wake_function+0/96] autoremove_wake_function+0x0/0x60
[ret_from_fork+6/20] ret_from_fork+0x6/0x14
[commit_timeout+0/16] commit_timeout+0x0/0x10
[kjournald+0/736] kjournald+0x0/0x2e0
[kernel_thread_helper+5/16] kernel_thread_helper+0x5/0x10
Code: 0f 0b 65 02 6e 8e 3a c0 e9 6e fd ff ff 8d 76 00 c7 04 24 00
<6>note: kjournald[127] exited with preempt_count 3


--------

Assertion failure in __journal_drop_transaction() at fs/jbd/checkpoint.c:613: "transaction->t_forget == NULL"
------------[ cut here ]------------
kernel BUG at fs/jbd/checkpoint.c:613!
invalid operand: 0000 [#1]
SMP
Modules linked in: megaraid
CPU: 1
EIP: 0060:[<c01df970>] Not tainted
EFLAGS: 00010282 (2.6.8-rc2-vs1.9.2.15+namespace-fix+disabled-preempt-filer)
EIP is at __journal_drop_transaction+0x350/0x3f7
eax: 00000071 ebx: d51d1e00 ecx: c03e1f64 edx: c03e1f64
esi: f7fcbc00 edi: d487b0bc ebp: d51d1e00 esp: f5711d80
ds: 007b es: 007b ss: 0068
Process kjournald (pid: 127, threadinfo=f5710000 task=c23cc5e0)
Stack: c0398a40 c037ca80 c038fcae 00000265 c038fcfe d51d1e00 f7fcbc00 c01df4ea
f7fcbc00 d51d1e00 d48245fc d487b0bc c01debea d487b0bc d487b0bc c01df490
d487b0bc 00000000 e38bed80 f4fafb00 00000000 00000000 f6cb2f80 f6cb2f80
Call Trace:
[<c01df4ea>] __journal_remove_checkpoint+0x4a/0xa0
[<c01debea>] __try_to_free_cp_buf+0x5a/0x90
[<c01df490>] __journal_clean_checkpoint_list+0x80/0x90
[<c01dd11e>] journal_commit_transaction+0x1be/0x1235
[<c011d160>] autoremove_wake_function+0x0/0x60
[<c0311123>] netif_receive_skb+0x173/0x1b0
[<c011d160>] autoremove_wake_function+0x0/0x60
[<c0295286>] tg3_rx+0x2a6/0x3f0
[<c011aba6>] find_busiest_group+0x2a6/0x340
[<c011ace0>] find_busiest_queue+0xa0/0xb0
[<c011aef2>] load_balance_newidle+0x82/0xa0
[<c0375f3f>] schedule+0x2ef/0x5f0
[<c01271aa>] del_timer_sync+0xaa/0xc0
[<c01e07b8>] kjournald+0xd8/0x240
[<c011d160>] autoremove_wake_function+0x0/0x60
[<c011d160>] autoremove_wake_function+0x0/0x60
[<c0106062>] ret_from_fork+0x6/0x14
[<c01e06c0>] commit_timeout+0x0/0x10
[<c01e06e0>] kjournald+0x0/0x240
[<c01042c5>] kernel_thread_helper+0x5/0x10
Code: 0f 0b 65 02 ae fc 38 c0 e9 6e fd ff ff 8d 76 00 c7 04 24 40


>>EIP; c01df970 <__journal_drop_transaction+350/3f7> <=====

>>ebx; d51d1e00 <pg0+14ce4e00/3fb11000>
>>ecx; c03e1f64 <log_wait+4/c>
>>edx; c03e1f64 <log_wait+4/c>
>>esi; f7fcbc00 <pg0+37adec00/3fb11000>
>>edi; d487b0bc <pg0+1438e0bc/3fb11000>
>>ebp; d51d1e00 <pg0+14ce4e00/3fb11000>
>>esp; f5711d80 <pg0+35224d80/3fb11000>

Trace; c01df4ea <__journal_remove_checkpoint+4a/a0>
Trace; c01debea <__try_to_free_cp_buf+5a/90>
Trace; c01df490 <__journal_clean_checkpoint_list+80/90>
Trace; c01dd11e <journal_commit_transaction+1be/1235>
Trace; c011d160 <autoremove_wake_function+0/60>
Trace; c0311123 <netif_receive_skb+173/1b0>
Trace; c011d160 <autoremove_wake_function+0/60>
Trace; c0295286 <tg3_rx+2a6/3f0>
Trace; c011aba6 <find_busiest_group+2a6/340>
Trace; c011ace0 <find_busiest_queue+a0/b0>
Trace; c011aef2 <load_balance_newidle+82/a0>
Trace; c0375f3f <schedule+2ef/5f0>
Trace; c01271aa <del_timer_sync+aa/c0>
Trace; c01e07b8 <kjournald+d8/240>
Trace; c011d160 <autoremove_wake_function+0/60>
Trace; c011d160 <autoremove_wake_function+0/60>
Trace; c0106062 <ret_from_fork+6/14>
Trace; c01e06c0 <commit_timeout+0/10>
Trace; c01e06e0 <kjournald+0/240>
Trace; c01042c5 <kernel_thread_helper+5/10>

Code; c01df970 <__journal_drop_transaction+350/3f7>
00000000 <_EIP>:
Code; c01df970 <__journal_drop_transaction+350/3f7> <=====
0: 0f 0b ud2a <=====
Code; c01df972 <__journal_drop_transaction+352/3f7>
2: 65 02 ae fc 38 c0 e9 add %gs:0xe9c038fc(%esi),%ch
Code; c01df979 <__journal_drop_transaction+359/3f7>
9: 6e outsb %ds:(%esi),(%dx)
Code; c01df97a <__journal_drop_transaction+35a/3f7>
a: fd std
Code; c01df97b <__journal_drop_transaction+35b/3f7>
b: ff (bad)
Code; c01df97c <__journal_drop_transaction+35c/3f7>
c: ff 8d 76 00 c7 04 decl 0x4c70076(%ebp)
Code; c01df982 <__journal_drop_transaction+362/3f7>
12: 24 40 and $0x40,%al


-------

SysRq : Show Regs

Pid: 388, comm: nfsd
EIP: 0060:[<c01dcce6>] CPU: 0
EIP is at .text.lock.transaction+0x2/0x18c
EFLAGS: 00000282 Not tainted (2.6.8-rc2-vs1.9.2.15+namespace-fix+disabled-preempt-filer)
EAX: f7fcbc00 EBX: f6cb2f80 ECX: f7fcbc00 EDX: 00000800
ESI: 00000000 EDI: f7fcbc00 EBP: c1808a60 DS: 007b ES: 007b
CR0: 8005003b CR2: 4015ea00 CR3: 34cdb000 CR4: 000006d0
[<c020f048>] find_exported_dentry+0x98/0x650
[<c027f5e4>] __make_request+0x2c4/0x520
[<c01da74d>] journal_start+0xad/0xe0
[<c023170f>] radix_tree_node_alloc+0x1f/0x70
[<c01cc5d0>] ext3_prepare_write+0x40/0x140
[<c0140400>] add_to_page_cache+0x60/0xc0
[<c0142710>] generic_file_aio_write_nolock+0x3d0/0xba0
[<c030b434>] kfree_skbmem+0x24/0x30
[<c033273d>] tcp_recvmsg+0x30d/0x760
[<c0142f50>] generic_file_write_nolock+0x70/0x90
[<c0143166>] generic_file_writev+0x46/0x60
[<c01601ec>] do_readv_writev+0x22c/0x280
[<c0211d50>] nfsd_acceptable+0x0/0x100
[<c015fbc0>] do_sync_write+0x0/0xb0
[<c0211d50>] nfsd_acceptable+0x0/0x100
[<c0160a8d>] open_private_file+0x9d/0xa0
[<c0160308>] vfs_writev+0x58/0x70
[<c021418f>] nfsd_write+0x11f/0x360
[<c036c5cf>] svc_sock_enqueue+0x14f/0x2c0
[<c021b798>] nfsd3_proc_write+0xb8/0x120
[<c02100f7>] nfsd_dispatch+0xd7/0x1d5
[<c0210020>] nfsd_dispatch+0x0/0x1d5
[<c036c2fa>] svc_process+0x4aa/0x60d
[<c020fe72>] nfsd+0x1f2/0x3a0
[<c020fc80>] nfsd+0x0/0x3a0
[<c01042c5>] kernel_thread_helper+0x5/0x10


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 14:04    [W:0.034 / U:1.916 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site