lkml.org 
[lkml]   [1999]   [Dec]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: readX/writeX semantic and ordering


On Thu, 16 Dec 1999, Gerard Roudier wrote:

> > Cachable CPU stores (whether buffered due to a cache miss or not) will not
> > be carried out to system bus, they will only be put into the cache. The
>
> If the cache controller snoops memory accesses (including DMA) from the
> system BUS and agrees with the memory/bus controller for performing
> cache line write-back when a dirty cache line is hit, ordering is as
> expected.

All PPC snoop (the data and L2 cache) provided the host bridge flags the
accesses it performs on behalf of PCI devices as global (with the GBL bus
signal). Setting or not this signal is an option on some bridges but you
can safely assume that it will be set since it makes things much simpler
(otherwise you'd have to explicitly flush the caches).

> > architecture does not guarantee anything about the ordering in theory,
> > but in all current implementations the only effect is that loads may be
> > moved ahead of pending stores and stores may be combined (but not on
> > guarded memory which happens to be the case of areas returned by ioremap).
>
> No problem there. A driver should normally expect IO/MMIO accesses to
> follow strong ordering, but should insert explicit memory barrier for
> STOREs to memory to be observed in order from the BUS each time it is
> needed. OTOH, having reads from memory passing writes when possible should
> not make problems.

Sorry I was not clear:
on PPC an MMIO read can go ahead of an MMIO write unless an eieio or sync
instruction is in between (or the read is from the same address as the
write). Writes can be reordered / gathered in host bridge too (or in the
processor for non guarded storage) unless there is an explicit ordering
barrier. This cause all {read,write,in,out}[bwl] on PPC to be followed by
an eieio instruction.

If I understand correctly all the issues (a big if), you have to put a
strong memory barrier to have guaranteed behaviour in your case. By
strong I mean a PPC 'sync' instruction, 'eieio' is not enough since it
does not order writeback memory accesses relative to noncacheable memory
accesses.

AFAIK, all _current_ PPC processors execute stores in order, and store
buffers will not be a problem since they are inside the coherent memory
domain (this is clearly stated on the 604 documentation, unfortunately the
available 7400/G4 documentation is still very poor). This means that
stores to memory will enter the coherent memory domain before a write to a
device register takes place. But this is not required and therefore not
safe...

I still don't see why they would want to execute the write to the device
register before the earlier stores enters the coherent memory domain. The
implementation complexity is probably not worth the performance gain.

> > So in the end I defined my own macros to access the registers of this
> > device: sometimes I don't care about reordering/merging for 5 or so
> > consecutive register writes which are a mixture of big and little endian
> > accesses. Without considering code bloat, eieio is expensive on some
> > processors since it goes to the bus to tell the bridge not to perform
> > reordering.
>
> When actual data transfer uses BUS mastering, the number of IOs/MMIOs is
> generally small. I would prefer to be safe rather than to spare a couple
> of micro-seconds, since ordering problems have very weird effects.

Indeed, but in my case (generating VMEbus locked cycle through Tundra's
Universe chip) the things are clear:
1) acquire a spinlock for SMP
2) write 5 registers which describe the locked cycle
3) ordering barrier
4) read from a register to actually perform the locked cycle
5) ordering barrier
6) write to disable the locked cycle generator
7) release the spinlock

the order in which registers are accessed in step 2 is totally irrelevant
and actually 5) is a nop since I don't know of any processor which would
reorder a write before a read. If you count the bus cycles with an eieio
after each read/write versus the minimum actually implemented, 14 versus
9, it becomes significant.

> Indeed I am interested.

Ok, I've put them on ftp://vlab1.iram.es/pub/ppcdocs since it's too big
for email (even private, I never considered posting it to the list).

Gabriel.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:55    [W:0.085 / U:0.156 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site