Messages in this thread | | | Subject | Re: Performance problem with bond interface | From | Eric Dumazet <> | Date | Mon, 21 Apr 2014 08:47:18 -0700 |
| |
On Mon, 2014-04-21 at 07:23 -0700, Venkat Venkatsubra wrote: > We see a performance problem when the slaves of the bond > don't support checksum offload features. What we see is > tcp_sendmsg's skb_add_data_nocache ending up not using the > csum_and_copy_from_user which would have computed the > checksum while copying from user buffer to kernel buffer. > Instead it computes later in dev_hard_start_xmit when it > figures out the slave doesn't support checksum offload and > ends up expensive . > > The bonding interface's "features" has NETIF_F_HW_CSUM > (or NETIF_F_NO_CSUM in 2.6.39) set which makes the > stack think checksum need not be computed in software. > /* > * Check whether we can use HW checksum. > */ > if (sk->sk_route_caps & NETIF_F_ALL_CSUM) > skb->ip_summed = CHECKSUM_PARTIAL; > > But later in dev_hard_start_xmit it finds out the slave does not > support checksumming and decides to compute in software. > > /* If packet is not checksummed and device does not > * support checksumming for this protocol, complete > * checksumming here. > */ > if (skb->ip_summed == CHECKSUM_PARTIAL) { > skb_set_transport_header(skb, > skb_checksum_start_offset(skb)); > if (!(features & NETIF_F_ALL_CSUM) && > skb_checksum_help(skb)) > goto out_kfree_skb; > } > > We see this problem after this commit: > commit 1742f183fc218798dab6fcf0ded25b6608fc0a48 > Author: MichaÅ<82> MirosÅ<82>aw <mirq-linux@rere.qmqm.pl> > Date: Fri Apr 22 06:31:16 2011 +0000 > > net: fix netdev_increment_features() > > Simplify and fix netdev_increment_features() to conform to what is > stated in netdevice.h comments about NETIF_F_ONE_FOR_ALL. > Include FCoE segmentation and VLAN-challedged flags in computation. > > Signed-off-by: MichaÅ<82> MirosÅ<82>aw <mirq-linux@rere.qmqm.pl> > Signed-off-by: David S. Miller <davem@davemloft.net> > > Prior to that the below code in netdev_increment_features was helping in > turning off NETIF_F_NO_CSUM on bond when the slaves don't support it: > /* If device needs checksumming, downgrade to it. */ > if (all & NETIF_F_NO_CSUM && !(one & NETIF_F_NO_CSUM)) > all ^= NETIF_F_NO_CSUM | (one & NETIF_F_ALL_CSUM); > > The slaves are Mellanox IB adapters. This is on x86_64 platform. > > Please let us know if you need any additional information.
Please CC patch author (I did), instead of sending this to hundred of people (linux-kernel ??? netdev is more appropriate...)
Do these NIC really not support TX checksum ?
You did not provide kernel version you use.
Please also provide : (using a recent ethtool to get extended offload info)
ethtool -k bond0 # or the bonding device name
ethtool -k eth1 # or the slave name
ethtool -i eth1
Thanks !
-- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |