[lkml]   [2000]   [Aug]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Cache coherency... and locking
On Mon, Jul 31, 2000 at 03:04:15PM -0500, Oliver Xymoron wrote:

> I think you'll find that the really big machines (ASCI, etc) are
> non-coherent. Coherence is expensive hardware-wise and the programming
> model it's designed to support just doesn't scale well. People don't
> ridicule Beowulfs and it's only a matter of time before commodity systems
> have competitive memory bandwidth with the SGIs and DECs of the world.

The only ccNUMA ASCI system I know is actually a cluster of 48 128CPU ccNUMAs.
Why? The Origin 2000 uses directories to avoid broadcasting of invalidates.
For machines larger than 128 CPUs the directories don't have enough bits to
allow a per-node granularity of invalidates, so for systems beyond that size
the hw has to broadcast invalidates into a region of machines thereby
causing an increase of traffic on the internal network.

Directories btw. are fairly expensive but yet a proven way to reduce the
coherency traffic on the internal network. The extra price is why I don't
believe that we'll see wide offerings of ccNUMAs any time soon. In fact
even SGI offers two memory configurations (standard / premium) which
implement different size directories and therefore support different
system sizes before having to use coarse invalidates.


To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
Please read the FAQ at

 \ /
  Last update: 2005-03-22 13:58    [W:0.076 / U:0.700 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site