[lkml]   [2001]   [Dec]   [4]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: SMP/cc Cluster description
    On Tue, Dec 04, 2001 at 06:36:01PM -0800, David S. Miller wrote:
    > From: Larry McVoy <>
    > Date: Tue, 4 Dec 2001 16:36:46 -0800
    > OK, so start throwing stones at this. Once we have a memory model that
    > works, I'll go through the process model.
    > What is the difference between your messages and spin locks?
    > Both seem to shuffle between cpus anytime anything interesting
    > happens.
    > In the spinlock case, I can thread out the locks in the page cache
    > hash table so that the shuffling is reduced. In the message case, I
    > always have to talk to someone.

    Two points:

    a) if you haven't already, go read the Solaris doors paper. Now think about
    a cross OS instead of a cross address space doors call. They are very
    similar. Think about a TLB shootdown, it's sort of a doors style cross
    call already, not exactly the same, but do you see what I mean?

    b) I am not claiming that you will have less threading in the page cache.
    I suspect it will be virtually identical except that in my case 90%+
    of the threading is in the ccCluster file system, not the generic code.
    I do not want to get into a "my threading is better than your threading"
    discussion. I'll even go so far as to say that this approach will have
    more threading if that makes you happy, as long as we agree it is outside
    of the generic part of the kernel. So unless I have CONFIG_CC_CLUSTER,
    you pay nothing or very, very little.

    Where this approach wins big is everywhere except the page cache. Every
    single data structure in the system becomes N-way more parallel -- with
    no additional locks -- when you boot up N instances of the OS. That's
    a powerful statement. Sure, I freely admit that you'll add a few locks
    to deal with cross OS synchronization but all of those are configed out
    unless you are running a ccCluster. There is absolutely zero chance that
    you can get the same level of scaling with the same number of locks using
    the traditional approach. I'll go out on a limb and predict it is at
    least 100x as many locks. Think about it, there are a lot of things that
    are threaded simply because of some corner case that happens to show up
    in some benchmark or workload. In the ccCluster case, those by and large
    go away. Less locks, less deadlocks, less memory barriers, more reliable,
    less filling, tastes great and Mikey likes it :-)
    Larry McVoy lm at
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 13:13    [W:0.026 / U:12.136 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site