Messages in this thread | | | Date | Wed, 7 Jan 2004 09:58:02 -0800 | Subject | [RFC] Relaxed PIO read vs. DMA write ordering | From | (Jesse Barnes) |
| |
I've already talked with Grant a little about this, but I'm having second thoughts about the approach we discussed. PCI-X allows PIO read responses to 'pass' DMA writes to system memory when the relaxed ordering bit is set in the PCI-X command word _and_ the transaction has the relaxed ordering bit set (so called "Relaxed Read Ordering" in section 11.2 of the PCI-X addendum). This effectively 'unserializes' PIO vs. DMA transactions so that PIO reads doesn't get stuck behind an unrelated DMA writes from the same device; something which can potentially take awhile since cacheline ownership has to be acquired, etc.
I'd like Linux to support relaxed read ordering in some way since on large systems having PIO reads stuck behind DMA writes can end up eating into CPU time and limit IOPS (do I have this right, Jeremy?).
The proposal I gave to Grant added a new readX() variant, readX_relaxed(), that drivers could use when they don't need strict ordering semantics (this may actually be the majority of cases, but it's safer to be strict by default than create a read_ordered and open a window for data corruption). It might be confusing, however, to add yet another readX() routine, and there are other ways we might go about it. One suggestion was to overload the pci_sync_* calls so that they'd explicitly flush DMA writes to system memory, implying that all reads on some platforms would use relaxed semantics, but that we'd have to modify drivers to add in pci_sync_* calls where needed.
Thoughts?
Thanks, Jesse - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |