[lkml]   [1999]   [Mar]   [7]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Lets get this right (WAS RE:MOSIX and kernel mods)
On Sun, 7 Mar 1999 16:55:10 +1100 Richard Gooch <> 

> OK, let's look at some numbers and compare. 100 Mb/s EtherNet is now
> a
> commidity item: Gb/s is still experimental. So we have networks with
> 10 MB/s bandwidth and 2 millisecond latencies.
> Now let's look at the bandwidth of a modern, commidity computer, say
> PII. With 100 MHz SDRAM, a 4-1-1-1 burst gives you 8 bytes per step;
> with 4 steps that's 32 bytes for the burst. It takes 7 cycles. Let's
> say 8 to make the maths simpler. So 8 cycles gives 32 bytes. That's
> 4
> bytes per cycle: 400 MBytes/s. The latency is going to be well under
> 200 nanoseconds.
> So: we have a factor of 40 in bandwidth and 10000 in latency. And
> this
> comparison is only done with the boring 64 bit bus in your average
> PC.
> It gets worse if you look at a 256 bit bus: bandwidth goes to 1.6
> GB/s,
> which is 160 times better than current EtherNet.
> Right now, there's no competition: networks are so much slower than
> what you find in a computer.
> Now, I hear some of you saying that networks are getting faster all
> the time. But so are computers! Let's look back a bit:
> 6 years ago, a 486 DX266 was hot stuff, but still a commidity
> item. IIRC, bandwidth back then was 20 MB/s. Hm. That's a factor of
> 20
> from where we are now. Moore's Law seems to be holding up (he
> predicts
> 16).
> 6 years ago, I had the pleasure of hacking on a VX/MVX system. This
> was a distributed memory machine costing $50000 and up. The
> interconnect between the boards was rated at 320 MB/s. What was
> "leading edge" back then is now available for the price of an
> average
> pushbike.
> And 6 years ago, where was networking? 1MB/s (10Mb/s), that's
> where. In fact, it was there in 1989 too: 10 years ago. It's taken
> around 10 years to get a factor of 10 improvement in bandwidth. And
> I
> don't recall the latency changing much over that time, either. 6
> years ago, all we had were whispers of FDDI. $50000 wasn't going to
> buy you something 16 times faster (like how the VX/MVX was faster).
> Networking has not kept up with Moore's Law. It seems computers
> have,
> however. So the logical progression is that in the future, we're
> going
> to see more of the same. It's going to take years before gigabit
> EtherNet is affordable, and by then computers will be even further
> ahead.
> > Heck even if this view is off and skewed to heck, *it is
> possible*.
> > And things are getting faster, it's only a matter of time before
> > networking is fast enough (or integrated enough) that clustering
> > such as this would be possible.
> Let me make a bottom-line statement: networking will *never* be as
> fast as internal computer speeds. Technology trends point to
> computers
> increasing at a greater rate. And then there's physics. Unless
> you're
> planning on breaking the speed of light, it's always going to take a
> lot longer to send information across a room than across a chip.

I never said it wouldn't... I'm just saying to *think*. With advances
in technology the latency of box->box communications is going to drop
enough so as to be neglable *if managed properly*. If you're throwing
tasks and data about willy-nilly then it won't matter how fast the
external connection is. If switches are made properly (as Linux does
quite well with SMP scheduling, unlike NT4 and it's pingpong effect) and
dumping from box to box is kept low then a point is reached where the
external network is as *perceptably* fast as the internal system.

Once you get past a certain speed the human cannot percieve the
difference. Cumulatively, yes, but instant-to-instant, no. Starfire
servers have a 'low-latency' backplane in the 500ns range. .5ms....
Ever ping a system across an Giga ether switch? Sub 1ms unless it's
loaded pretty heavy.

Acceptable, yes.

Practical, at this juncture in time no. But in the near future, yes.

Perhaps my view comes from wanting to see a great technology come about.
There are many uses for a 'true' cluster.

> [...]
> > Think.
> Count.
> Regards,
> Richard....
> -
> To unsubscribe from this list: send the line "unsubscribe
> linux-kernel" in
> the body of a message to
> Please read the FAQ at
Michael Loftis

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
Please read the FAQ at

 \ /
  Last update: 2005-03-22 13:50    [W:0.069 / U:0.232 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site