[lkml]   [1999]   [Aug]   [8]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: IPSEC transport mode w/2.2.x kernels and large packets
    Date (Richard Guy Briggs) writes:

    > Large packet handling was not checked at the time and was tripped over
    > afterwards. When large packets hit a 2.2.x kernel with ipsec and were
    > passed to ip_fragment(), the machine oopsed, sometimes hard. The
    > output function passed to ip_fragment() was skb->dst->output(). Once
    > it was changed to neigh_compat_output(), as was used in the place of
    > dev_queue_xmit() from 2.0.x to 2.2.x, the problem went away *for me*
    > and got worse for others. This started my quest as to what function
    > should be used for output, and lead into cflow diagrams...

    Simple. Not _your_ functions, but the functions returned by the
    routing table lookung and the neighbour state machine. Trying
    to muck with dst->output is recipe for desaster

    > There have been reports of severe crashes under the 2.2 kernels with
    > large packets. I was seeing fairly regular crashes under 2.2.7 which
    > would start with a small oops and keep rolling to larger and larger
    > oopses until they finally exhausted themselves. I upgraded to 2.2.10
    > and those problems largely went away, with only the occasional crash.
    > Others have reported exactly the opposite behaviour with relative
    > stability under 2.2.7 and crashing consistently under 2.2.10. There
    > have been many reports on this list and linux-kernel about
    > instability, locking up, tcp stalling and such for many of the 2.2.x
    > series kernels, so we are not certain it is anything we are doing.
    > There was a report on the list that 2.2.11pre4 is more stable. I am
    > trying that, but still have some unexplained behaviour, which might be
    > the 2.0.36 machine at the other end, which is not able to re-assemble
    > the fragments for some reason. Investigation pending.

    It is easy to blame the kernel stack of course. Although there are bugs,
    you should be aware that about 80% of the reports are caused
    because the users use some equipment that can't deal with the new
    SACK/timestamp/window scaling TCP options.

    > At the same time, I did a FreeS/WAN presentation at the Ottawa Linux
    > Symposium and in discussions with a number of other developpers,
    > describing the problem elucidated some of the issues surrounding doing
    > transport mode IPSEC. It confirmed that we need to move our
    > processing up the stack, before things are fragmented. In particular,
    > Alan Cox suggested setting the MTU to the maximum allowed:0xfff0, then
    > fragmenting after encryption to the MTU of the physical device.

    So when one fragment gets dropped the whole packet is lost ? This is the
    "NFS trap" and works badly over WANs, because you would disturb path
    MTU discovery. It is important that packets with DF get a icmp error
    when they exceed the effective size of the tunnel.

    > This seems to work under the 2.0.x kernels since before I call
    > dev_queue_xmit() or ip_fragment(), I can change the ownership of the
    > skb from the virtual device to the physical device. In the process of
    > testing all this, I started getting some skb's that were not getting
    > skb_copy_expand()ed properly. For some reason,
    > skb->truesize-sizeof(sk_buff) was not quite right. I have changed it
    > to be the same as that used in the 2.2.x kernels by Marc Boucher and I
    > am starting to see some 'cannot get free page' messages, which I am
    > not certain are from that change or the maximum mtu change. It only
    > seems to happen with connections directly to the gateway, and not
    > connections *through* the gateway that use ipsec.

    skb->truesize is broken in 2.2 (it should contains sizeof(sk_buff) but
    it does not ATM). Why are you using that anyways? You should use

    > Under 2.2.x kernels, this is not so simple, since even if I change the
    > ownership of the skb, skb->dev, the skb still has a dst cache entry
    > pointer, skb->dst, which still assumes the old device, skb->dst->dev.
    > How do we change the dst entry? We can't do another routing table
    > lookup since that is unreliable and prone to route stealing, besides
    > we are trying to move away from that method. Do we blow it away? It
    > would then need to do a routing lookup. How do we tell it to use the
    > new device without corrupting the dst cache? The only way I could
    > come up with was to patch net/ipv4/ip_output.c:ip_fragment() so that
    > if skb->dst->dev, the device pointed to in the dst cache pointer, and
    > skb->dev, the device pointed to by the skb, were different, it would
    > take its MTU information from skb->dev->mtu, rather than where it does
    > now, from skb->dst->pmtu.

    Why is another routing table lookup unreliable? For me it seems to
    be the cleanest way (e.g. ip_gre does it too). I don't know what you
    mean with route stealing. Trying to play your own games with the dst
    is not a good idea, see the dst as encapsulated object you shouldn't

    This is like TV. I don't like TV.

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 13:53    [W:0.024 / U:90.316 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site