Messages in this thread | | | From | glynis@butterfl ... | Date | Mon, 29 Jul 2002 04:50:33 -0400 | Subject | linux 2.4.19-rc3, snat, and syslog-ng strangenesses |
| |
i've been working on stabilizing a dual athlon 1800 here, and originally attributed my netfilter problems to smp, etc. over the weekend i've gather much more information that makes for an even more bizarre story.
using snat, my box would throw an error: Jul 29 00:12:31 butterfly kernel: LIST_DELETE: ip_conntrack_core.c:165 `&ct->tuplehash[IP_CT_DIR_REPLY]'(ddd3ee90) not in &ip_conntrack_hash [hash_conntrack(&ct->tuplehash[IP_CT_DIR_REPLY].tuple)].
then /proc/net/ip_conntrack would start going nuts filling with [UNREPLIED] connections, until cat just seemed to spew the same thing over and over without eof, then deadlock.
sometimes i got an oops out of it most often blaming syslog-ng or dnetc, and otherwise pinpointing it in ip_conntrack code. these are attached below. i captured what i thought was pertinent info then punched it back in to feed to ksymoops. i hope it looks right.
using MASQUERADE instead of snat, it wouldn't exhibit any problems.
so after discussing the netfilter implications to death, i was encouraged to widen my search, so i did.
i enabled snat, then started slowly eliminating programs and modules from the running system. i got it down to bind, syslog-ng, and my netfilter modules pretty much. it still would crash. when i disabled syslog-ng, the system stopped deadlocking/oopsing!
i brought back all my other services and the box would keep running! when i added syslog-ng back in, rebooted, it would die within 10 minutes as usual.
then i disabled syslog-ng's remote udp log stream to another host, but otherwise started syslog-ng for local logging, and the box still ran.
then i rearranged my /etc/rc2.d/ a bit to make sure my iptables rules ran before syslog-ng started WITH it's udp stream to a remote host. the box is still running. so it seems if i start syslog-ng and it's udp stream before setting up my firewall, it dies, but starting the firewall before everything else, then starting syslog-ng works!
so what's causing this? traffic through the firewall was relatively light -- 2-3 hosts doing ntp checks for the most part. does this offer any more info to the people who were trying to help me debug earlier?
thank you for your time.
here are some references: 2 different decoded oopses: --- ksymoops 2.4.5 on i686 2.4.19-rc3. Options used -V (default) -k /var/log/ksymoops/20020729010116.ksyms (specified) -l /var/log/ksymoops/20020729010116.modules (specified) -o /lib/modules/2.4.19-rc3/ (default) -m /boot/System.map-2.4.19-rc3 (default)
Oops: 0 EIP: 0010:[<E0C023C5>] Using defaults from ksymoops -t elf32-i386 -a i386 Call Trace: [<e0c02b76>] [<e0c130cf>] [<e0c1379b>] [<e0c16480>] [<e0c037cd>] Code: 39 4f 18 74 60 8d 47 08 89 45 f8 8b 47 08 8b 4d 08 31 db 31
>>EIP; e0c023c5 <[ip_conntrack]__ip_conntrack_find+155/1d0> <=====
Trace; e0c02b76 <[ip_conntrack]ip_conntrack_tuple_taken+a6/130> Trace; e0c130cf <[iptable_nat]ip_nat_used_tuple+1f/30> Trace; e0c1379b <[iptable_nat]get_unique_tuple+db/1e0> Trace; e0c16480 <[iptable_nat]ip_nat_protocol_udp+0/40> Trace; e0c037cd <[ip_conntrack]invert_tuplepr+1d/30>
Code; e0c023c5 <[ip_conntrack]__ip_conntrack_find+155/1d0> 00000000 <_EIP>: Code; e0c023c5 <[ip_conntrack]__ip_conntrack_find+155/1d0> <===== 0: 39 4f 18 cmp %ecx,0x18(%edi) <===== Code; e0c023c8 <[ip_conntrack]__ip_conntrack_find+158/1d0> 3: 74 60 je 65 <_EIP+0x65> e0c0242a <[ip_conntrack]__ip_conntrack_find+1ba/1d0> Code; e0c023ca <[ip_conntrack]__ip_conntrack_find+15a/1d0> 5: 8d 47 08 lea 0x8(%edi),%eax Code; e0c023cd <[ip_conntrack]__ip_conntrack_find+15d/1d0> 8: 89 45 f8 mov %eax,0xfffffff8(%ebp) Code; e0c023d0 <[ip_conntrack]__ip_conntrack_find+160/1d0> b: 8b 47 08 mov 0x8(%edi),%eax Code; e0c023d3 <[ip_conntrack]__ip_conntrack_find+163/1d0> e: 8b 4d 08 mov 0x8(%ebp),%ecx Code; e0c023d6 <[ip_conntrack]__ip_conntrack_find+166/1d0> 11: 31 db xor %ebx,%ebx Code; e0c023d8 <[ip_conntrack]__ip_conntrack_find+168/1d0> 13: 31 00 xor %eax,(%eax)
ksymoops 2.4.5 on i686 2.4.19-rc3. Options used -V (default) -k /var/log/ksymoops/20020729040201.ksyms (specified) -l /var/log/ksymoops/20020729040201.modules (specified) -o /lib/modules/2.4.19-rc3/ (default) -m /boot/System.map-2.4.19-rc3 (default)
EIP: 0010:[<e0d9a3c5>] Using defaults from ksymoops -t elf32-i386 -a i386 Call Trace: [<e0d9ab76>] [<e0dab0cf>] [<e0dab79b>] [<e0dae480>] [<e0d9b7cd>] [<e0d0eec0>] Code: 39 4f 18 74 60 8d 47 08 89 45 f8 8b 47 08 8b 4d 08 31 db 31
>>EIP; e0d9a3c5 <[ip_conntrack]__ip_conntrack_find+155/1d0> <=====
Trace; e0d9ab76 <[ip_conntrack]ip_conntrack_tuple_taken+a6/130> Trace; e0dab0cf <[iptable_nat]ip_nat_used_tuple+1f/30> Trace; e0dab79b <[iptable_nat]get_unique_tuple+db/1e0> Trace; e0dae480 <[iptable_nat]ip_nat_protocol_udp+0/40> Trace; e0d9b7cd <[ip_conntrack]invert_tuplepr+1d/30> Trace; e0d0eec0 <[cpia_usb].bss.end+b5c79/130db9>
Code; e0d9a3c5 <[ip_conntrack]__ip_conntrack_find+155/1d0> 00000000 <_EIP>: Code; e0d9a3c5 <[ip_conntrack]__ip_conntrack_find+155/1d0> <===== 0: 39 4f 18 cmp %ecx,0x18(%edi) <===== Code; e0d9a3c8 <[ip_conntrack]__ip_conntrack_find+158/1d0> 3: 74 60 je 65 <_EIP+0x65> e0d9a42a <[ip_conntrack]__ip_conntrack_find+1ba/1d0> Code; e0d9a3ca <[ip_conntrack]__ip_conntrack_find+15a/1d0> 5: 8d 47 08 lea 0x8(%edi),%eax Code; e0d9a3cd <[ip_conntrack]__ip_conntrack_find+15d/1d0> 8: 89 45 f8 mov %eax,0xfffffff8(%ebp) Code; e0d9a3d0 <[ip_conntrack]__ip_conntrack_find+160/1d0> b: 8b 47 08 mov 0x8(%edi),%eax Code; e0d9a3d3 <[ip_conntrack]__ip_conntrack_find+163/1d0> e: 8b 4d 08 mov 0x8(%ebp),%ecx Code; e0d9a3d6 <[ip_conntrack]__ip_conntrack_find+166/1d0> 11: 31 db xor %ebx,%ebx Code; e0d9a3d8 <[ip_conntrack]__ip_conntrack_find+168/1d0> 13: 31 00 xor %eax,(%eax) ---
and my snat rules i have running now, but had lots of trouble with in previous configurations: --- echo "1" > /proc/sys/net/ipv4/ip_forward iptables -t nat -A POSTROUTING -j SNAT --to 64.192.31.41 iptables -t nat -A POSTROUTING -j SNAT --to 192.168.1.1 iptables -t nat -A POSTROUTING -j SNAT --to 127.0.0.1
iptables -A INPUT -p tcp -i eth0 --destination-port 515 --syn -j REJECT iptables -A INPUT -p tcp -i eth0 --destination-port 8082 --syn -j REJECT iptables -A INPUT -p tcp -i eth0 --destination-port 8083 --syn -j REJECT iptables -A INPUT -p tcp -i eth0 --destination-port 8007 --syn -j REJECT iptables -A INPUT -p tcp -i eth0 --destination-port 111 --syn -j REJECT iptables -A INPUT -p udp -i eth0 --destination-port 111 -j REJECT iptables -A INPUT -p tcp -i eth0 --destination-port 2049 --syn -j REJECT iptables -A INPUT -p udp -i eth0 --destination-port 2049 -j REJECT --- -- ____________________}John Flinchbaugh{______________________ | glynis@hjsoft.com http://www.hjsoft.com/~glynis/ | ~~Powered by Linux: Reboots are for hardware upgrades only~~ [unhandled content-type:application/pgp-signature] | |