Messages in this thread | ![/](/images/icornerl.gif) | | From | Ian Kumlien <> | Date | Mon, 26 Jun 2023 16:25:24 +0200 | Subject | Re: [Intel-wired-lan] bug with rx-udp-gro-forwarding offloading? |
| |
On Mon, Jun 26, 2023 at 4:18 PM Alexander Lobakin <aleksander.lobakin@intel.com> wrote: > > From: Ian Kumlien <ian.kumlien@gmail.com> > Date: Sun, 25 Jun 2023 12:59:54 +0200 > > > It could actually be that it's related to: rx-gro-list but > > rx-udp-gro-forwarding makes it trigger quicker... I have yet to > > trigger it on igb > > Hi, the rx-udp-gro-forwarding author here. > > (good thing this appeared on IWL, which I read time to time, but please > Cc netdev next time) > (thus +Cc Jakub, Eric, and netdev)
Well, two things, it seems like rx-udp-gro-forwarding accelerates it but the issue is actually in: rx-gro-list
And since i've only been able to trigger it in ixgbe i thought it might be a driver issue =)
> > On Sat, Jun 24, 2023 at 10:03 PM Ian Kumlien <ian.kumlien@gmail.com> wrote: > >> > >> Hi again, > >> > >> I suspect that I have rounded this down to the rx-udp-gro-forwarding > >> option... I know it's not on by default but.... > >> > >> So, I have a machine with four nics, all using ixgbe, they are all: > >> 06:00.0 Ethernet controller: Intel Corporation Ethernet Connection > >> X553 1GbE (rev 11) > >> 06:00.1 Ethernet controller: Intel Corporation Ethernet Connection > >> X553 1GbE (rev 11) > >> 07:00.0 Ethernet controller: Intel Corporation Ethernet Connection > >> X553 1GbE (rev 11) > >> 07:00.1 Ethernet controller: Intel Corporation Ethernet Connection > >> X553 1GbE (rev 11) > >> > >> But I have been playing with various... currently i do: > >> for interface in eno1 eno2 eno3 eno4 ; do > >> for offload in ntuple hw-tc-offload rx-gro-list ; do > >> ethtool -K $interface $offload on > /dev/null > >> done > >> ethtool -G $interface rx 8192 tx 8192 > /devYnull > >> done > >> > >> And it all seems to work just fine for my little firewall > >> > >> However, enabling rx-udp-gro-forwarding results in the attached oooops > >> (sorry, can't see more, been recreating by watching shows on HBO > >> max... ) > > Where's the mentioned oops? Where's the original message?
Held by the mailing list since i can only get a screenshot of it... Will attach the latest one to this email (I wish that i could easily get a larger backtrace but i haven't looked in further atm)
> Can't this be related to [0]?
Don't know, my main test has been running video streams in the background - eventually they cause a oops (within 40 minutes or so) But i doubt it's counted as tunnel data ;)
> rx-udp-gro-forwarding is here for, uhm... 3 years? And UDP GRO in > general is much longer. Is this a non-mainline kernel?
Mainline 6.3.9 and now 6.4.0
> So many questions :D > > >> > >> The code seems to decode to: > >> Code: c3 08 66 89 5c 02 04 45 84 e4 0f 85 27 fd ff ff 49 8b 1e 49 8b > >> ae c8 00 00 00 41 0f b7 86 b8 00 00 00 45 0f b7 a6 b6 00 00 00 <48> 8b > >> b3 c8 00 00 00 0f b7 8b b6 00 00 00 49 01 ec 48 01 c5 48 8d > >> All code > >> ======== > >> 0: c3 ret > >> 1: 08 66 89 or %ah,-0x77(%rsi) > >> 4: 5c pop %rsp > >> 5: 02 04 45 84 e4 0f 85 add -0x7af01b7c(,%rax,2),%al > >> c: 27 (bad) > >> d: fd std > >> e: ff (bad) > >> f: ff 49 8b decl -0x75(%rcx) > >> 12: 1e (bad) > >> 13: 49 8b ae c8 00 00 00 mov 0xc8(%r14),%rbp > >> 1a: 41 0f b7 86 b8 00 00 movzwl 0xb8(%r14),%eax > >> 21: 00 > >> 22: 45 0f b7 a6 b6 00 00 movzwl 0xb6(%r14),%r12d > >> 29: 00 > >> 2a:* 48 8b b3 c8 00 00 00 mov 0xc8(%rbx),%rsi <-- trapping instruction > >> 31: 0f b7 8b b6 00 00 00 movzwl 0xb6(%rbx),%ecx > >> 38: 49 01 ec add %rbp,%r12 > >> 3b: 48 01 c5 add %rax,%rbp > >> 3e: 48 rex.W > >> 3f: 8d .byte 0x8d > >> > >> Code starting with the faulting instruction > >> =========================================== > >> 0: 48 8b b3 c8 00 00 00 mov 0xc8(%rbx),%rsi > >> 7: 0f b7 8b b6 00 00 00 movzwl 0xb6(%rbx),%ecx > >> e: 49 01 ec add %rbp,%r12 > >> 11: 48 01 c5 add %rax,%rbp > >> 14: 48 rex.W > >> 15: 8d .byte 0x8d > >> > >> But correlating that with the source is beyond me, it could be generic > >> but i thought i'd send it you first since it's part of the redhat > >> guide to speeding up udp traffic > [0] > https://lore.kernel.org/netdev/f83d79d6-f8d7-a229-941a-7d7427975160@nvidia.com > > Thanks, > Olek [unhandled content-type:image/jpeg] | ![\](/images/icornerr.gif) |