Messages in this thread | | | From | Roland Dreier <> | Subject | Re: MMIO and gcc re-ordering issue | Date | Tue, 27 May 2008 08:50:17 -0700 |
| |
> Though it's my understanding that at least ia64 does require the > explicit barriers anyway, so we are still in a dodgy situation here > where it's not clear what drivers should do and we end up with > possibly excessive barriers on powerpc where I end up with both > the wmb/rmb/mb that were added for ia64 -and- the ones I have in > readl/writel to make them look synchronous... Not nice.
ia64 is a disaster with a slightly different ordering problem -- the mmiowb() issue. I know Ben knows far too much about this, but for big SGI boxes, you sometimes need mmiowb() to avoid problems with driver code that does totally sane stuff like
spin_lock(&mmio_lock); writel(val1, reg1); writel(val2, reg2); spin_unlock(&mmio_lock);
If that snippet is called on two CPUs at the same time, then the device might see a sequence like
CPU1 -- write reg1 CPU2 -- write reg1 CPU1 -- write reg2 CPU2 -- write reg2
in spite of the fact that everything is totally ordered on the CPUs by the spin lock.
The reason this is such a disaster is because the code looks right, makes sense, and works fine on 99.99% of all systems out there. So I would bet that 99% of our drivers have missing mmiowb() "bugs" -- no one has plugged the hardware into an Altix box and cared enough to stress test it.
However for the issue at hand, my expectation as a driver writer is that readl()/writel() are ordered with respect to MMIO operations, but not necessarily with respect to normal writes to coherent CPU memory. And I've included explicit wmb()s in code I've written like drivers/infiniband/hw/mthca.
- R.
| |