Messages in this thread | | | Subject | Re: [RFC] MMIO accessors & barriers documentation | From | Roland Dreier <> | Date | Mon, 11 Sep 2006 16:08:23 -0700 |
| |
Benjamin> and rmb is heavy handed for a compiler barrier :) what Benjamin> you might need on some platforms is an rmb between the Benjamin> MMIO read of whatever status/index register and the Benjamin> following memory reads of descriptors, and you may want Benjamin> an rmb in case where it matters if the chip has been Benjamin> changing a value behind your back (which it generally Benjamin> doesn't) but that's pretty much it....
In drivers/infiniband/hw/mthca/mthca_eq.c, there is:
while ((eqe = next_eqe_sw(eq))) { /* * Make sure we read EQ entry contents after we've * checked the ownership bit. */ rmb();
switch (eqe->type) {
where next_eqe_sw() checks a "valid" bit of a 32-byte event queue entry that is DMA-ed into memory by the device. The device is careful to write the valid bit (byte actually) last, but on PowerPC 970 without the rmb(), we actually saw the CPU reordering the read of eqe->type (which is another field of the EQ entry written by the device) so it happened before the entry was valid, but then executing the check of the valid bit far enough into the future so that the entry tested as valid.
This isn't that surprising: if you had two CPUs, with one CPU writing into a queue and the other CPU polling the queue, you would obviously need smp_rmb() on the CPU doing the reading. But somehow it's not quite as obvious when a device plays the role of one of the CPUs.
Of course there's no MMIO anywhere in sight here, so this isn't directly applicable I guess.
- R. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |