lkml.org 
[lkml]   [2007]   [Apr]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Subject2.6.20.7 mss negotiation and path mtu discovery mostly broken?
    Date
    From
    I'm seeing a problem where the kernel attempts to send packets with a
    MSS larger than the one negotiated when the TCP connection is
    established. Even after ICMP "can't fragment" messages arrive, the
    kernel still attempts to increase the MSS rather aggressively. The end
    result is extremely poor throughput when sending to a network with a
    smaller MTU.

    In /proc/sys/net/ipv4:
    ip_no_pmtu_disc:0
    tcp_mtu_probing:0

    The sending host (10.2.10.254) has an MTU of 9000. The destination host
    (12.33.234.69) has an MTU of 1500. There is one router between the hosts
    which will drop packets with the "DF" flag when they don't fit the
    destination interface's MTU and generates the required icmp can't
    fragment message.

    The dump shows the initial handshake with correct mss options sent:

    08:39:55.493029 IP 12.33.234.69.35026 > 10.2.10.254.22: S
    2768979373:2768979373(
    0) win 5840 <mss 1460,sackOK,timestamp 3873837730 0,nop,wscale 2>
    08:39:55.493119 IP 10.2.10.254.22 > 12.33.234.69.35026: S
    963242385:963242385(0)
    ack 2768979374 win 17896 <mss 8960,sackOK,timestamp 413751
    3873837730,nop,wscal
    e 5>

    Then I see the system send larger packets (larger than the mss),
    provoking a "can't fragment" from the router. Now I suppose it might be
    reasonable to occasionally probe a larger MSS when the current MSS is a
    result of reductions due to path mtu discovery. After all, the path
    taken could change over time. But when the current MSS is at the value
    negotiated by the MSS option during the TCP handshake, it seems like
    there's no sense in trying to send with a lager MSS. Even if there were,
    there's certainly no justification for making such an attempt every
    other packet (2.6.18) or every fourth packet (2.6.20.7).

    In the following dump, the system eventually gets in a state where it
    oscillates between sendng undeliverable 2896 byte packets and
    deliverable 1448 byte ones.

    08:39:55.649689 IP 10.2.10.254.22 > 12.33.234.69.35026: .
    5906:10250(4344) ack 1
    794 win 674 <nop,nop,timestamp 413790 3873837887>
    08:39:55.650532 IP 10.2.10.1 > 10.2.10.254: icmp 92: 12.33.234.69
    unreachable -
    need to frag (mtu 1500)
    08:39:55.689774 IP 12.33.234.69.35026 > 10.2.10.254.22: . ack 5906 win
    4544 <nop
    ,nop,timestamp 3873837927 413790>
    08:39:55.689784 IP 10.2.10.254.22 > 12.33.234.69.35026: .
    10250:13146(2896) ack
    1794 win 674 <nop,nop,timestamp 413800 3873837927>
    08:39:55.690497 IP 10.2.10.1 > 10.2.10.254: icmp 92: 12.33.234.69
    unreachable -
    need to frag (mtu 1500)
    08:39:55.902494 IP 10.2.10.254.22 > 12.33.234.69.35026: .
    5906:7354(1448) ack 17
    94 win 674 <nop,nop,timestamp 413853 3873837927>

    Since any sane router will only generate can't fragment ICMP's at a
    limited rate, for two hosts on gigabit ethernet, one on a MTU 1500
    subnet and another on a MTU 9000 subnet, I can move only 40-50KB/sec
    over an affected TCP connection.

    I was unable to find any reference to this problem in the kernel
    changelogs, or even any reports of anyone else having a similar problem.
    The above dumps are from 2.6.19.7. I could also reproduce the problem in
    2.6.18, although the dumps looked slightly different. I was unable to
    reproduce this problem with the 2.6.9-42.0.10.ELsmp kernel which ships
    in RHEL4.

    I can send a pcap dump to anyone interested.

    --
    Brian Ristuccia


    "This email message and any attachments are confidential information of Starent Networks, Corp. The information transmitted may not be used to create or change any contractual obligations of Starent Networks, Corp. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon this e-mail and its attachments by persons or entities other than the intended recipient is prohibited. If you are not the intended recipient, please notify the sender immediately -- by replying to this message or by sending an email to postmaster@starentnetworks.com -- and destroy all copies of this message and any attachments without reading or disclosing their contents. Thank you."
    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2007-04-25 15:31    [W:0.044 / U:2.080 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site