Messages in this thread | | | From | Malcolm Beattie <> | Subject | Re: Figured you should probably know this... | Date | Fri, 14 Nov 1997 12:47:34 +0000 (GMT) |
| |
Mike Jagdis writes: > On Thu, 13 Nov 1997, Billy Harvey wrote: > > > The new Linux motto: > > > > Linux: Software that performs like a penguin with an attitude. > > Wouldn't we want clustering then? :-)
Funnily enough, I'm implementing IP clustering in order to try out a Beowulf-style cluster for a (highly) multi-user interactive service. It's sort of like inverse masqerading, only different :-). Although each cluster node has an IP address for communicating with the other nodes (on the high-bandwidth interconnect), it also has an alias-style device on a shared IP number (which doesn't respond to arps). A front-end node catches traffic to that shared address and routes new connections to "shared" ports to an appropriate cluster node, keeping track of it for future traffic on that connection.
Each node also has its own "private" part of the port space (e.g. for a 16-node cluster, node1 gets 4096-7936, node1 gets 7937-12033) which the kernel uses for non-explicitly bound connections (e.g. outgoing telnet/rlogin etc.) and the front-end node routes those ports directly to the relevant node. From the outside world (and for users logged in doing "ordinary" things), the cluster appears to be a single machine, with the Beowulf-style crossmounting, shared PID space and a few other bells and whistles making everything fit together.
Implementation involves a new firewall chain plus code in ip_rcv() (for 2.0.x) to decide which host to route to depending on ipfwadm rules, a current connection hash table and dynamically created "pool" and "node" devices which let you split the cluster up into pools of nodes each with given costs/loads for distributing particular types of connections. From userland, it's a combination of "ipfwadm -C", "clconfig" (hacked ifconfig to add support for ioctls to add, delete or alter pool/node devices) and a daemon to update the current loads for each node in each node device in the front-end so that it can make good decisions for how to distribute new connections. Nodes other than the front-end don't need anything special other than a change to the inet_autobind/good_socknum stuff so that you can configure each node to auto-bind into its private port space for outgoing connections.
The design is all done, it's now just a SMOP when time pressure eases up a little in December. So if anyone has an online picture of a mob of "penguins with attitude", let me know.
--Malcolm
-- Malcolm Beattie <mbeattie@sable.ox.ac.uk> Unix Systems Programmer Oxford University Computing Services
| |