[lkml]   [2006]   [Oct]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: Ordering between PCI config space writes and MMIO reads?
    Quoting r. Matthew Wilcox <>:
    > Subject: Re: Ordering between PCI config space writes and MMIO reads?
    > On Wed, Oct 25, 2006 at 12:30:22AM -0600, Grant Grundler wrote:
    > > Can someone provide a quote of the PCI Local bus spec that allows this?
    > > (Or at least a reference to a spec version and section number)
    > PCI-PCI bridges are allowed to do it. If you look in table E-1 of PCI
    > 2.3, or table 8-3 of PCI-X 2.0, you'll see that a Posted Memory Write
    > can pass a Delayed Write Request (or in PCI-X, a Memory Write can pass a
    > Split Write Request).
    > So mmiowb() will solve the problem for Altix, but leave everybody else
    > vulnerable. I actually don't see a way of forcing the config write to
    > complete before a memory write -- everything is allowed to pass a config
    > write, even a config read. I initially thought "But only a crack monkey
    > would implement a system where a config read could pass a config write",
    > but the spec explains that:
    > In most PCI-X implementations, Split Requests are managed in separate
    > buffers from Split Completions, so Split Requests naturally pass Split
    > Completions. However, no deadlocks occur if Split Completions block
    > Split Requests.
    > So all this code that checks to see if a write had an effect is unsafe.
    > I'm a little perturbed by this. It means the only way to reliably
    > distinguish between a write that hasn't taken effect yet and a bit (say,
    > MWI) the device hasn't implemented is to do a memory access to the
    > device. Which is hard when you're trying to program the BARs.
    > I suppose this hasn't bitten us before in, what, 7 years of PCI-X, so
    > it can't be *that* common a thing for bridges to do. And we would have
    > noticed the BAR sizing code going wrong (as it does config write
    > followed immediately by config read), so maybe implementations aren't as
    > crackful as the PCI spec seems to permit them to be.
    > I find it really hard to believe the PCI committee have done something
    > this stupid. There must be another rule somewhere that I'm missing.

    I think typically CPUs stall until a non-posted operation completes.
    And since config writes are non posted,

    write ....

    does not *start* the write until config write has completed.
    So there's only a single outstanding config operation and that's why
    there's never any re-ordering, without any need for flushes.

    Your Altix system seems the weird one here in that CPU actually
    treats config writes as posted and does not wait for their completion.
    I wander whether you can do a bus lock or something and force
    waiting till the completion.
    This would be much cleaner than trying to fix all drivers.

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2006-10-25 20:25    [W:0.025 / U:8.356 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site