Messages in this thread |  | | | Date | Fri, 4 Aug 2000 18:00:53 +0200 | | From | Ralf Baechle <> | | Subject | Re: Cache coherency... and locking |
| |
On Mon, Jul 31, 2000 at 03:04:15PM -0500, Oliver Xymoron wrote:
> I think you'll find that the really big machines (ASCI, etc) are > non-coherent. Coherence is expensive hardware-wise and the programming > model it's designed to support just doesn't scale well. People don't > ridicule Beowulfs and it's only a matter of time before commodity systems > have competitive memory bandwidth with the SGIs and DECs of the world.
The only ccNUMA ASCI system I know is actually a cluster of 48 128CPU ccNUMAs. Why? The Origin 2000 uses directories to avoid broadcasting of invalidates. For machines larger than 128 CPUs the directories don't have enough bits to allow a per-node granularity of invalidates, so for systems beyond that size the hw has to broadcast invalidates into a region of machines thereby causing an increase of traffic on the internal network.
Directories btw. are fairly expensive but yet a proven way to reduce the coherency traffic on the internal network. The extra price is why I don't believe that we'll see wide offerings of ccNUMAs any time soon. In fact even SGI offers two memory configurations (standard / premium) which implement different size directories and therefore support different system sizes before having to use coarse invalidates.
Ralf
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
|  |