[lkml]   [2008]   [Jan]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: e1000 full-duplex TCP performance well below wire speed
    Hi Sangtae,

    Thanks for joining this discussion -- it's good to a CUBIC author and
    expert here!

    >> In our application (cluster computing) we use a very tightly coupled
    >> high-speed low-latency network. There is no 'wide area traffic'. So
    >> it's hard for me to understand why any networking components or
    >> software layers should take more than milliseconds to ramp up or back
    >> off in speed. Perhaps we should be asking for a TCP congestion
    >> avoidance algorithm which is designed for a data center environment
    >> where there are very few hops and typical packet delivery times are
    >> tens or hundreds of microseconds. It's very different than delivering
    >> data thousands of km across a WAN.

    > If your network latency is low, regardless of type of protocols should
    > give you more than 900Mbps.

    Yes, this is also what I had thought.

    In the graph that we posted, the two machines are connected by an ethernet
    crossover cable. The total RTT of the two machines is probably AT MOST a
    couple of hundred microseconds. Typically it takes 20 or 30 microseconds
    to get the first packet out the NIC. Travel across the wire is a few
    nanoseconds. Then getting the packet into the receiving NIC might be
    another 20 or 30 microseconds. The ACK should fly back in about the same

    > I can guess the RTT of two machines is less than 4ms in your case and I
    > remember the throughputs of all high-speed protocols (including
    > tcp-reno) were more than 900Mbps with 4ms RTT. So, my question which
    > kernel version did you use with your broadcomm NIC and got more than
    > 900Mbps?

    We are going to double-check this (we did the broadcom testing about two
    months ago). Carsten is going to re-run the broadcomm experiments later
    today and will then post the results.

    You can see results from some testing on crossover-cable wired systems
    with broadcomm NICs, that I did about 2 years ago, here:
    You'll notice that total TCP throughput on the crossover cable was about
    220 MB/sec. With TCP overhead this is very close to 2Gb/s.

    > I have two machines connected by a gig switch and I can see what happens
    > in my environment. Could you post what parameters did you use for
    > netperf testing?

    Carsten will post these in the next few hours. If you want to simplify
    further, you can even take away the gig switch and just use a crossover

    > and also if you set any parameters for your testing, please post them
    > here so that I can see that happens to me as well.

    Carsten will post all the sysctl and ethtool parameters shortly.

    Thanks again for chiming in. I am sure that with help from you, Jesse, and
    Rick, we can figure out what is going on here, and get it fixed.


     \ /
      Last update: 2008-01-31 09:55    [W:0.023 / U:0.208 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site