[lkml]   [1999]   [Mar]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Lets get this right (WAS RE:MOSIX and kernel mods)
Michael Loftis writes:
> On Sun, 7 Mar 1999 16:55:10 +1100 Richard Gooch <>
> wrote:
> > OK, let's look at some numbers and compare. 100 Mb/s EtherNet is now
> > a
> > commidity item: Gb/s is still experimental. So we have networks with
> > 10 MB/s bandwidth and 2 millisecond latencies.
> > So: we have a factor of 40 in bandwidth and 10000 in latency. And
> > this
> > comparison is only done with the boring 64 bit bus in your average
> > PC.
> > It gets worse if you look at a 256 bit bus: bandwidth goes to 1.6
> > GB/s,
> > which is 160 times better than current EtherNet.
> >
> > Right now, there's no competition: networks are so much slower than
> > what you find in a computer.
> >
> > Now, I hear some of you saying that networks are getting faster all
> > the time. But so are computers! Let's look back a bit:
> > Networking has not kept up with Moore's Law. It seems computers
> > have,
> > however. So the logical progression is that in the future, we're
> > going
> > to see more of the same. It's going to take years before gigabit
> > EtherNet is affordable, and by then computers will be even further
> > ahead.
> > Let me make a bottom-line statement: networking will *never* be as
> > fast as internal computer speeds. Technology trends point to
> > computers
> > increasing at a greater rate. And then there's physics. Unless
> > you're
> > planning on breaking the speed of light, it's always going to take a
> > lot longer to send information across a room than across a chip.
> I never said it wouldn't... I'm just saying to *think*. With
> advances in technology the latency of box->box communications is
> going to drop enough so as to be neglable *if managed properly*. If
> you're throwing tasks and data about willy-nilly then it won't
> matter how fast the external connection is. If switches are made
> properly (as Linux does quite well with SMP scheduling, unlike NT4
> and it's pingpong effect) and dumping from box to box is kept low
> then a point is reached where the external network is as
> *perceptably* fast as the internal system.

This completely misses the point. The point is *not* that in 10 years
we're going to have Tb/s bandwidths and microsecond latencies for
networks, and thus that those networks will be "fast enough". The
point is that in 10 years networks are going to still be a lot slower
than something inside a computer.

No matter how fast networking technology becomes, it's never going to
be negligible, *because it's always going to be orders of magnitude
slower than a computer*. So it's *always going to be faster to use
SHM locks than DSM locks*. The problem space will expand to fill the
available hardware.

My recent goal was sub-second rendering times on a modest dataset (13
MBytes) using a cluster of machines ranging from dual P133 to dual PII
400. That goal was achieved and I'm now looking to rendering times
under 100 milliseconds.

If you hand me a network with computers and network orders of
magnitude faster, do you think I'm going to keep doing the same thing?
No way! I'm going to want render times under 30 milliseconds and
datasets of a few hundred MBytes. I'm still going to care that
grabbing a remote lock is going to be 4+ orders of magnitude more
expensive than grabbing a local lock. So I'm still going to do SMP for
the local nodes and MPI for the remote nodes, because I need to think
about the two cases in a different way. I use locks because they're
fast and there are benefits to using them and they're neater; but when
locks would be slow, I do things a different way, and there the locks
just get in the way: MPI is neater.

Different problems require different solutions. Even in the same
programme. One size does not fit all.

> Once you get past a certain speed the human cannot percieve the
> difference. Cumulatively, yes, but instant-to-instant, no.
> Starfire servers have a 'low-latency' backplane in the 500ns range.
> .5ms.... Ever ping a system across an Giga ether switch? Sub 1ms
> unless it's loaded pretty heavy.

Humans perceive the difference, because they throw bigger problems at
the hardware. The bigger problem magnifies these so-called "really
fast overheads".

> Acceptable, yes.

No way! I'll not throw away 4 orders of magnitude in performance no
matter how fast the hardware is. If ever the hardware is much faster
than I need for any of my computational problems, I'll start running a
Copy[1] of myself on it. And if the hardware is still mostly idling,
I'll migrate to it[2].

No such thing as "fast enough".

1: "Permutation City", Greg Egan (a good read)
2: "Diaspora", Greg Egan. (even better)



To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
Please read the FAQ at

 \ /
  Last update: 2005-03-22 13:50    [W:0.066 / U:1.892 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site