[lkml]   [1999]   [Mar]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: Lets get this right (WAS RE:MOSIX and kernel mods)
    Michael Loftis writes:
    > On Sun, 7 Mar 1999 16:55:10 +1100 Richard Gooch <>
    > wrote:
    > > OK, let's look at some numbers and compare. 100 Mb/s EtherNet is now
    > > a
    > > commidity item: Gb/s is still experimental. So we have networks with
    > > 10 MB/s bandwidth and 2 millisecond latencies.
    > > So: we have a factor of 40 in bandwidth and 10000 in latency. And
    > > this
    > > comparison is only done with the boring 64 bit bus in your average
    > > PC.
    > > It gets worse if you look at a 256 bit bus: bandwidth goes to 1.6
    > > GB/s,
    > > which is 160 times better than current EtherNet.
    > >
    > > Right now, there's no competition: networks are so much slower than
    > > what you find in a computer.
    > >
    > > Now, I hear some of you saying that networks are getting faster all
    > > the time. But so are computers! Let's look back a bit:
    > > Networking has not kept up with Moore's Law. It seems computers
    > > have,
    > > however. So the logical progression is that in the future, we're
    > > going
    > > to see more of the same. It's going to take years before gigabit
    > > EtherNet is affordable, and by then computers will be even further
    > > ahead.
    > > Let me make a bottom-line statement: networking will *never* be as
    > > fast as internal computer speeds. Technology trends point to
    > > computers
    > > increasing at a greater rate. And then there's physics. Unless
    > > you're
    > > planning on breaking the speed of light, it's always going to take a
    > > lot longer to send information across a room than across a chip.
    > I never said it wouldn't... I'm just saying to *think*. With
    > advances in technology the latency of box->box communications is
    > going to drop enough so as to be neglable *if managed properly*. If
    > you're throwing tasks and data about willy-nilly then it won't
    > matter how fast the external connection is. If switches are made
    > properly (as Linux does quite well with SMP scheduling, unlike NT4
    > and it's pingpong effect) and dumping from box to box is kept low
    > then a point is reached where the external network is as
    > *perceptably* fast as the internal system.

    This completely misses the point. The point is *not* that in 10 years
    we're going to have Tb/s bandwidths and microsecond latencies for
    networks, and thus that those networks will be "fast enough". The
    point is that in 10 years networks are going to still be a lot slower
    than something inside a computer.

    No matter how fast networking technology becomes, it's never going to
    be negligible, *because it's always going to be orders of magnitude
    slower than a computer*. So it's *always going to be faster to use
    SHM locks than DSM locks*. The problem space will expand to fill the
    available hardware.

    My recent goal was sub-second rendering times on a modest dataset (13
    MBytes) using a cluster of machines ranging from dual P133 to dual PII
    400. That goal was achieved and I'm now looking to rendering times
    under 100 milliseconds.

    If you hand me a network with computers and network orders of
    magnitude faster, do you think I'm going to keep doing the same thing?
    No way! I'm going to want render times under 30 milliseconds and
    datasets of a few hundred MBytes. I'm still going to care that
    grabbing a remote lock is going to be 4+ orders of magnitude more
    expensive than grabbing a local lock. So I'm still going to do SMP for
    the local nodes and MPI for the remote nodes, because I need to think
    about the two cases in a different way. I use locks because they're
    fast and there are benefits to using them and they're neater; but when
    locks would be slow, I do things a different way, and there the locks
    just get in the way: MPI is neater.

    Different problems require different solutions. Even in the same
    programme. One size does not fit all.

    > Once you get past a certain speed the human cannot percieve the
    > difference. Cumulatively, yes, but instant-to-instant, no.
    > Starfire servers have a 'low-latency' backplane in the 500ns range.
    > .5ms.... Ever ping a system across an Giga ether switch? Sub 1ms
    > unless it's loaded pretty heavy.

    Humans perceive the difference, because they throw bigger problems at
    the hardware. The bigger problem magnifies these so-called "really
    fast overheads".

    > Acceptable, yes.

    No way! I'll not throw away 4 orders of magnitude in performance no
    matter how fast the hardware is. If ever the hardware is much faster
    than I need for any of my computational problems, I'll start running a
    Copy[1] of myself on it. And if the hardware is still mostly idling,
    I'll migrate to it[2].

    No such thing as "fast enough".

    1: "Permutation City", Greg Egan (a good read)
    2: "Diaspora", Greg Egan. (even better)



    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 13:50    [W:0.035 / U:2.404 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site