Messages in this thread | | | Date | Tue, 29 Mar 2011 10:16:30 +0100 | From | Alan Cox <> | Subject | Re: advice sought: practicality of SMP cache coherency implemented in assembler (and a hardware detect line) |
| |
> hmmm, the question is, therefore: would the MOSIX DSM solution be > preferable, which i presume assumes that memory cannot be shared at > all, to a situation where you *could* at least get cache coherency in > userspace, if you're happy to tolerate a software interrupt handler > flushing the cache line manually?
In theory DSM goes further than this. One way to think about DSM is cache coherency in software with a page size granularity. So you could imagine a hypothetical example where the physical MMU of each node and a memory manager layer comnunicating between them implemented a virtualised machine on top which was cache coherent.
The detail (and devil no doubt) is in the performance.
Basically however providing your MMU can trap both reads and writes you can implement a MESI cache in software. Mosix just took this to an extreme as part of a distributed Unix (originally V7 based).
So you've got
Modified: page on one node, MMU set to fault on any other so you can fix it up
Exclusive: page on one node, MMU set to fault on any other or on writes by self (latter taking you to modified so you know to write back)
Shared: any write set to be caught by the MMU, the fun bit then is handling invalidating across other nodes with the page in cache. (and the fact multiple nodes may fault the page at once)
Invalid: our copy is invalid (its M or E elsewhere probably), MMU set so we fault on any access. For shared this is also relevant so you can track for faster invalidates
and the rest is a software problem.
| |