Messages in this thread | | | Date | Wed, 30 Dec 2009 15:44:20 -0500 | From | Michael Breuer <> | Subject | Re: audit.c skb - tty race condition - was sky2 panic in 2.6.32.1 under load (new oops) |
| |
A couple more observations:
1) enabling auditd for runlevel 3 mitigates the issue 2) starting a remote x session (XDMCP) while under load and while auditd is already running also triggers the sky2 interrupt status messages - so maybe not tty1 - but some sort of X & auditd interaction. Even in this case, the frequency of the error messages is much less than when auditd is started in runlevel 5 for the first time.
On 12/30/2009 2:15 PM, Michael Breuer wrote: > And now looking at audit.c it seems reasonable that there is a race > condition when auditd is started at roughly the same time as X. I'm > guessing that the kaudit thread is fired up; the tty connected; and at > the same time X grabs the tty. Somewhere in there an skb gets hosed > and is then reused by whatever comes along - in my case sky2 as that's > where the subsequent demand is. If the demand happens first, the > contaminated skb (dk in what way yet) is probably waiting to manifest > as some other bug that's been frustrating people. > On 12/30/2009 12:49 PM, Michael Breuer wrote: >> On 12/30/2009 2:58 AM, Stephen Hemminger wrote: >>> On Wed, 30 Dec 2009 02:23:20 -0500 >>> Michael Breuer<mbreuer@majjas.com> wrote: >>> >>>> Ok - I called dump_txring from sky2_net_intr: >>>> --- a/drivers/net/sky2.c >>>> +++ b/drivers/net/sky2.c >>>> @@ -2725,8 +2791,10 @@ static void sky2_watchdog(unsigned long arg) >>>> /* Hardware/software error handling */ >>>> static void sky2_err_intr(struct sky2_hw *hw, u32 status) >>>> { >>>> - if (net_ratelimit()) >>>> + if (net_ratelimit()) { >>>> dev_warn(&hw->pdev->dev, "error interrupt >>>> status=%#x\n", status); >>>> + dump_txring(hw, 0); >>>> + } >>>> >>>> if (status& Y2_IS_HW_ERR) >>>> sky2_hw_intr(hw); >>>> >>>> And got this: >>>> Dec 30 02:17:23 mail kernel: sky2 0000:06:00.0: error interrupt >>>> status=0x40000008 >>>> Dec 30 02:17:23 mail kernel: sky2 0000:06:00.0: error interrupt >>>> status=0x40000008 >>>> Dec 30 02:17:23 mail kernel: sky2 Tx ring pending=28...30 report=29 >>>> done=29 >>>> Dec 30 02:17:23 mail kernel: sky2 Tx ring pending=28...30 report=29 >>>> done=29 >>>> Dec 30 02:17:23 mail kernel: sky2 0000:06:00.0: error interrupt >>>> status=0x8 >>>> Dec 30 02:17:23 mail kernel: sky2 0000:06:00.0: error interrupt >>>> status=0x8 >>>> Dec 30 02:17:23 mail kernel: sky2 Tx ring pending=30...32 report=30 >>>> done=31 >>>> Dec 30 02:17:23 mail kernel: sky2 Tx ring pending=30...32 report=30 >>>> done=31 >>>> >>> I notice that you have NOUVEAU Nvidia drivers loaded? The one >>> difference in HW >>> between your board and mine is that I have ATI video card. >>> >> Seems the problem is linked to auditd and X11 (but not nouveau). >> >> Today, I ran a bunch of scenarios. I first determined that the >> problem only manifest in runlevel 5. Next, this occurred with or >> without KMS and with or without nouveau. This happened whether or not >> I was logged in (local or remote), and regardless of window manager >> (xdm, gdm, kdm). I then checked to see what else was different >> between runlevel 3 and 5 - only thing was auditd. I disabled auditd >> and reran - no errors. >> >> Now for the odd stuff: >> >> The errors only manifest if the high throughput data transfer is >> initiated when the system is in runlevel 5 and auditd was started by >> init when transitioning from runlevel 3 to 5. For example, the >> following scenarios do not cause the errors to manifest: >> >> runlevel3; start auditd runlevel 5; start transfer >> runlevel3; chkconfig auditd off; runlevel5; start auditd; start transfer >> runlevel3; start transfer (note: errors do not occur if I transition >> to runlevel 5 after the high bandwidth transfer has started) >> runlevel3; startx; start transfer >> >> The only way I get the problem to manifest is transition to runlevel >> 5 with chkconfig auditd on (level 5 only) and then initate the >> windows backup. >> >> I'm guessing that there is some sort of race condition happening >> between X (xdm/gdm/kdm/greeter?) and auditd that is somehow >> corrupting something. I'd hazard a more or less obvious guess that >> whatever's being corrupted differs when there is already a high >> throughput transfer under way. >> >> -- >> To unsubscribe from this list: send the line "unsubscribe >> linux-kernel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> Please read the FAQ at http://www.tux.org/lkml/ > > -- > To unsubscribe from this list: send the line "unsubscribe > linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/
| |