lkml.org 
[lkml]   [2003]   [Sep]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRe: Scaling noise
    Date
    On Thursday 04 September 2003 04:19, Steven Cole wrote:
    > On Wed, 2003-09-03 at 19:50, Daniel Phillips wrote:
    > > There was a time when SMP locking overhead actually cost something in the
    > > high single digits on Linux, on certain loads. Today, you'd have to work
    > > at it to find a real load where the 2.5/6 kernel spends more than 1% of
    > > its time in locking overhead, even on a large SMP machine (sample size of
    > > one: I asked Bill Irwin how his 32 node Numa cluster is running these
    > > days). This blows the ccCluster idea out of the water, sorry. The only
    > > way ccCluster gets to live is if SMP locking is pathetic and it's not.
    >
    > I would never call the SMP locking pathetic, but it could be improved.
    > Looking at Figure 6 (Star-CD, 1-64 processors on Altix) and Figure 7
    > (Gaussian 1-32 processors on Altix) on page 13 of "Linux Scalability for
    > Large NUMA Systems", available for download here:
    > http://archive.linuxsymposium.org/ols2003/Proceedings/
    > it appears that for those applications, the curves begin to flatten
    > rather alarmingly. This may have little to do with locking overhead.

    2.4.17 is getting a little old, don't you think? This is the thing that
    changed most in 2.4 -> 2.6, and indeed, much of the work was in locking.

    > One possible benefit of using ccClusters would be to stay on that lower
    > part of the curve for the nodes, using perhaps 16 CPUs in a node. That
    > way, a 256 CPU (e.g. Altix 3000) system might perform better than if a
    > single kernel were to be used. I say might. It's likely that only
    > empirical data will tell the tale for sure.

    Right, and we do not see SGI contributing patches for partitioning their 256
    CPU boxes. That's all the empirical data I need at this point.

    They surely do partition them, but not at the Linux OS level.

    > > As for Karim's work, it's a quintessentially flashy trick to make two UP
    > > kernels run on a dual processor. It's worth doing, but not because it
    > > blazes the way forward for ccClusters. It can be the basis for hot
    > > kernel swap: migrate all the processes to one of the two CPUs, load and
    > > start a new kernel on the other one, migrate all processes to it, and let
    > > the new kernel restart the first processor, which is now idle.
    >
    > Thank you for that very succinct summary of my rather long-winded
    > exposition on that subject which I posted here:
    > http://marc.theaimsgroup.com/?l=linux-kernel&m=105214105131450&w=2

    I swear I made the above up on the spot, just now :-)

    > Quite a bit of the complexity which I mentioned, if it were necessary at
    > all, could go into user space helper processes which get spawned for the
    > kernel going away, and before init for the on-coming kernel. Also, my
    > comment about not being able to shoe-horn two kernels in at once for
    > 32-bit arches may have been addressed by Ingo's 4G/4G split.

    I don't see what you're worried about, they are separate kernels and you get
    two instances of whatever split you want.

    Regards,

    Daniel

    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-03-22 13:48    [W:0.026 / U:0.132 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site