lkml.org 
[lkml]   [1999]   [Aug]   [8]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
From
SubjectRe: IPSEC transport mode w/2.2.x kernels and large packets
Date
-----BEGIN PGP SIGNED MESSAGE-----

> rgb@conscoop.ottawa.on.ca (Richard Guy Briggs) writes:

[I wish you had not removed the Cc:'s and crosspost addresses!
...particularly linux-ipsec@clinet.fi]

> > Large packet handling was not checked at the time and was tripped over
> > afterwards. When large packets hit a 2.2.x kernel with ipsec and were
> > passed to ip_fragment(), the machine oopsed, sometimes hard. The
> > output function passed to ip_fragment() was skb->dst->output(). Once
> > it was changed to neigh_compat_output(), as was used in the place of
> > dev_queue_xmit() from 2.0.x to 2.2.x, the problem went away *for me*
> > and got worse for others. This started my quest as to what function
> > should be used for output, and lead into cflow diagrams...
>
> Simple. Not _your_ functions, but the functions returned by the
> routing table lookung and the neighbour state machine. Trying
> to muck with dst->output is recipe for desaster

I didn't mess with dst->output, I simply replaced the argument to
ip_forward(..., skb->dst->output) with ip_forward(..., neigh_compat_output)

> > There have been reports of severe crashes under the 2.2 kernels with
> > large packets. I was seeing fairly regular crashes under 2.2.7 which
> > would start with a small oops and keep rolling to larger and larger
> > oopses until they finally exhausted themselves. I upgraded to 2.2.10
> > and those problems largely went away, with only the occasional crash.
> > Others have reported exactly the opposite behaviour with relative
> > stability under 2.2.7 and crashing consistently under 2.2.10. There
> > have been many reports on this list and linux-kernel about
> > instability, locking up, tcp stalling and such for many of the 2.2.x
> > series kernels, so we are not certain it is anything we are doing.
> > There was a report on the list that 2.2.11pre4 is more stable. I am
> > trying that, but still have some unexplained behaviour, which might be
> > the 2.0.36 machine at the other end, which is not able to re-assemble
> > the fragments for some reason. Investigation pending.
>
> It is easy to blame the kernel stack of course. Although there are bugs,
> you should be aware that about 80% of the reports are caused
> because the users use some equipment that can't deal with the new
> SACK/timestamp/window scaling TCP options.

Can you explain this in more detail? I am not touching the
upper-layer protocol headers at all. I don't need to know about them.
If the 2.2.10 box can talk to a 2.0.36 box, then that layer should
still work. I just need to muck with the mss for that route.

> > At the same time, I did a FreeS/WAN presentation at the Ottawa Linux
> > Symposium and in discussions with a number of other developpers,
> > describing the problem elucidated some of the issues surrounding doing
> > transport mode IPSEC. It confirmed that we need to move our
> > processing up the stack, before things are fragmented. In particular,
> > Alan Cox suggested setting the MTU to the maximum allowed:0xfff0, then
> > fragmenting after encryption to the MTU of the physical device.
>
> So when one fragment gets dropped the whole packet is lost ? This is the
> "NFS trap" and works badly over WANs, because you would disturb path
> MTU discovery. It is important that packets with DF get a icmp error
> when they exceed the effective size of the tunnel.

Yes, I know it can cause bad performance. In fact, maybe what should
be done is to send back an ICMP message only for tunnel mode packets,
and not for transport mode. I am not presently differentiating.

> > This seems to work under the 2.0.x kernels since before I call
> > dev_queue_xmit() or ip_fragment(), I can change the ownership of the
> > skb from the virtual device to the physical device. In the process of
> > testing all this, I started getting some skb's that were not getting
> > skb_copy_expand()ed properly. For some reason,
> > skb->truesize-sizeof(sk_buff) was not quite right. I have changed it
> > to be the same as that used in the 2.2.x kernels by Marc Boucher and I
> > am starting to see some 'cannot get free page' messages, which I am
> > not certain are from that change or the maximum mtu change. It only
> > seems to happen with connections directly to the gateway, and not
> > connections *through* the gateway that use ipsec.
>
> skb->truesize is broken in 2.2 (it should contains sizeof(sk_buff) but
> it does not ATM). Why are you using that anyways? You should use
> skb->len.

Yes, I know, that is why Marc did not use it. I was using it in 2.0,
since skb_copy_expand is a customised skb_copy(). In fact, skb->len
is not complete enough. I want to make the skb larger by 'headroom'
plus 'tailroom' over the (obviously too small, otherwise I would have
used it) existing skb. This is why I use skb->end - skb->head, which
includes all of the existing headroom and tailroom which is needed for
bits that have been or will be stripped off for processing.

> > Under 2.2.x kernels, this is not so simple, since even if I change the
> > ownership of the skb, skb->dev, the skb still has a dst cache entry
> > pointer, skb->dst, which still assumes the old device, skb->dst->dev.
> > How do we change the dst entry? We can't do another routing table
> > lookup since that is unreliable and prone to route stealing, besides
> > we are trying to move away from that method. Do we blow it away? It
> > would then need to do a routing lookup. How do we tell it to use the
> > new device without corrupting the dst cache? The only way I could
> > come up with was to patch net/ipv4/ip_output.c:ip_fragment() so that
> > if skb->dst->dev, the device pointed to in the dst cache pointer, and
> > skb->dev, the device pointed to by the skb, were different, it would
> > take its MTU information from skb->dev->mtu, rather than where it does
> > now, from skb->dst->pmtu.
>
> Why is another routing table lookup unreliable? For me it seems to
> be the cleanest way (e.g. ip_gre does it too). I don't know what you
> mean with route stealing. Trying to play your own games with the dst
> is not a good idea, see the dst as encapsulated object you shouldn't
> touch.

Right, I am not trying to play with the dst cache, just use the right
entry.

Take the following example:

A-----B=======C==========D=========E------F

A, F are endhosts which know nothing of IPSEC

B, E are security GWs, one foot in a trusted network, the other in the
big, bad internet

C, D are the nexthop routers from the security GWs

- ----- is an unprotected local link from an endhost to a security GW

===== is an IPSEC tunnel

Say we have a tunnel mode connection set up from B to E to cover
traffic from A to F. Say we also have a tunnel or transport mode
connection set up from B to E to cover packets with source and
destination addresses of B or E (which includes previously
encapsulated packets from A to F and vice versa). The routing table
will look like:

Destination Gateway Iface
A-B_subnet - eth1
B - lo
B-C_subnet - eth0
default C eth0
E C ipsec0
E-F_subnet C ipsec0

Since the packet destination has not changed for any packets going
from B to E or the device and next hop have not changed for any
packets going from A to F, then B to E, we don't have any physical I/F
to point to in the dst cache.

> -Andi

slainte mhath, RGB
- --
The first Ottawa Linux Symposium was a huge success! <ottawalinuxsymposium.org>
This SunRayce was a wet one! DroughtRelief_99? -- <www.sunrayce.com/sunrayce/>
Richard Guy Briggs -- PGP key available Auto-Free Ottawa! Canada
<http://www.conscoop.ottawa.on.ca/rgb/> </www.flora.org/afo/>
Prevent Internet Wiretapping! -- FreeS/WAN:<www.xs4all.nl/~freeswan>
Thanks for voting Green! -- <green.ca> Marillion:<www.marillion.co.uk>

-----BEGIN PGP SIGNATURE-----
Version: 2.6.3i
Charset: noconv

iQCVAwUBN6xWPd+sBuIhFagtAQFdYgQAnESbL6fTVicfJ9Cv/V2+84lDiUKmGM1m
ts5XYlj2Dy30cwWieTzimAbrt/0941DUGSIqGg7HeUTae3KU3YpU7DHZnNKipOFM
o1SapfaS60qETL28ct9yWhhUvZLINBXYWKXPtWMkvaAY3R0XXo9bbBtTr/8lWGZC
1WGzWrkSZ38=
=MJ5x
-----END PGP SIGNATURE-----

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:53    [W:0.225 / U:0.408 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site