Messages in this thread |  | | Date | Tue, 28 Jan 2003 09:42:21 -0800 | From | Ross Biro <> | Subject | Re: BUG: [2.4.18+] IDE Race Condition |
| |
Sorry, we are not using sub-microsecond timers, we are just using the latest-greatest-bugiest hard drives and lots of them. We can see problems like this because we can run on so many machines at once. Even something that has a small percentage chance of showing up, we can see pretty quick.
We do have a bus analyzer that can time things down to about 4ns. That has been invaluable for tracking down some of these issues.
Ross
Andre Hedrick wrote:
>Ross, > >How did you create the sub-microsecond timers to profile the device/driver >behavior? I had been working on this for a while but little success. >This is one of the key methods to predict device failure. > >One of the goals of the prebuilder it find slam prebuild commands down the >pipes to force breakages and races to show up. > >So you have a possible solution? > >Cheers, > >Andre Hedrick >LAD Storage Consulting Group > > >On Tue, 28 Jan 2003, Ross Biro wrote: > > > >>Easy, it happens all the time, there are just no tests in place to see it. >> >>We are keeping a histogram of how long every IDE DMA transfer takes >>place. In ide_intr we record the time and set the start time in >>ide_drive_t to 0. In ide_dma_proc, ide_dma_begin right AFTER activating >>the dma, we store the current value of jiffies in start time in ide_drive_t. >> >>In both those places we check to make sure that the value of start_time >>is sensible. In ide_dma_begin, we make sure it's 0 and in ide_dma_intr, >>we make sure its non-zero. Because of this race condition, we often saw >>DMAs finish before they began. >> >>In the normal kernel, the only thing I can see that could go wrong would >>be that the printk >> >>printk("%s: ide_intr: huh? expected NULL handler on exit\n", drive->name); >> >>in ide_intr could be triggered. I've never seen it happen, but I >>believe with enough effort, it could be made to occur. >> >> Ross >> >> >>Andre Hedrick wrote: >> >> >> >>>Okay, how do you reproduce it to see the effects? >>> >>>On Mon, 27 Jan 2003, Ross Biro wrote: >>> >>> >>> >>> >>> >>>>The net effect of this race condition and the other one I spotted is >>>>that you may see some interesting messages in your log file and you can >>>>detect the race condition if you look for it hard enough. I don't >>>>currently see any bad effects. >>>> >>>> Ross >>>> >>>>Ross Biro wrote: >>>> >>>> >>>> >>>> >>>> >>>>>There is at least one more IDE race condition in 2.4.18 and >>>>>2.4.21-pre3. Basically the interrupt for the controller being serviced >>>>>is left on while setting up the next command. I'm not sure how much >>>>>trouble it can cause but it does lead to some interesting stack traces. >>>>> >>>>>The condition >>>>>if (masked_irq && hwif->irq != masked_irq) >>>>>in ide_do_request should be replaced with >>>>>if (!masked_irq || hwif->irq != masked_irq) >>>>>in two places. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>- >>>>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in >>>>the body of a message to majordomo@vger.kernel.org >>>>More majordomo info at http://vger.kernel.org/majordomo-info.html >>>>Please read the FAQ at http://www.tux.org/lkml/ >>>> >>>> >>>> >>>> >>>> >>>Andre Hedrick >>>LAD Storage Consulting Group >>> >>> >>> >>> >>> > > >
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
|  |