Messages in this thread | ![/](/images/icornerl.gif) | | Subject | Re: automatic routing in 2.2.* | Date | Mon, 27 Mar 2000 22:43:27 -0600 | From | Robert Manning <> |
| |
> which still doesn't explain why we need this 'kernel knows better' > behaviour in the case of using ifconfig. > > > -d There is a design issue of IP routing implementations that is perhaps misunderstood here. And it's a design issue within kernel routing tables.
I want to address this, because I've had to explain this to sooo many people, that I hope this will save someone else having to answer the same question again, and again in the future.
By definition: a network is defined by a single ip address and netmask combination. (Unless you are also a cost-based router, more about that later..).
There is no way around this, as it is basic to what defines an IP network. The fact that you brought up an interface with both an IP address and a netmask does 2 things: 1.) It gives definition to a network of machines. The number of machines, and the upper and lower limits of the address range are defined by the sheer act of creating such an interface, by applying the mask you gave to the address you gave. 2.) It makes the interface an (immediate, direct) member of that network.
Attempts to do otherwise defy the very simple logic which defines IP routing within a machine.
Even if you use of an all bits ones mask, you have defined a "network of one" machine. -In such a network, the network number, the broadcast address, and the host address are all compressed into a single address. -But still, a route is required for the kernel's routing mechanism to understand where to deliver a packet, even if it is later determined that the route is to a local address and the packets are shunted to the loopback mechanism for delivery).
Interface configuration utilities (such as ifconfig) which implement IP are correct to add an interface route because to do anything else is inconsistent with IP's definition of a network. Historically, NOT adding the route has only served to confuse the operator into believeing that any route he would calculate could be stuffed in and work (properly), when quite the opposite is true. (To fully understand this, you need to look at radix tree designs, but I am trying to keep this readable for those who don't want to). ;) Some quite nasty, tangled, partially-dysfunctional routing webs (tables) have been weaved by admins on systems where the interface routes were not auto-added based on IP's routing design. -Such an environment has led to networks which appeared to conform to nobody's set of rules, even to the point of network routes would overlap each other, and all manner of unexpected (self-induced) routing problems would occur, ther outcome of which would be largely dependent on which of the offending duplicate/overlapping routes was added first/last.
Some early versions of ifconfig would fail to calculate the derived route from the user given IP and netmask, and auto-add it into the routing table, and thus were part of the problem, (after fixing this, they are part of the solution). (The "problem" being one of people who didn't fully understand the basic definition of an IP network, and all that bringing an interface into such a network truly implies...and thus would spend much wasted time and effort fumbling thier way through invalid routing configurations, until finally asking some advice of someone more experiemced, and finding their way back to "IP + netmask = <definition of a range of addresses known collectively as a network, and yea, you just joined it>. Now, if you "join" an interface to a network, then by all logic, you MUST have a route to THAT network through THAT interface. (Or you really didn't join that interface into that network)...
All interfaces join a network, and get a route. -Whether they have joined a network of ONE (netmask 255.255.255.255) or MANY (any other netmask) makes no difference, they have still joined one.
Note that until an implementation implements "equal-cost multipath" support into it's kernel routing tables, then the ONLY criteria for deciding which route an IP packet will take, is to take the packets destination IP (from the packet header), and search the radix tree (kernels routing table structure) for the "most specific" match. (In the case of a "tie" (a very *bad thing*) wherein TWO or more interfaces have been defined into the same network, - the action to be taken in routing the outbound packet to it's destination is implementation-specific, and undefined by any standards, and the results are unpredictable) -although, what USUALLY happens, is that either the first, or the last interface added that duplicates the network ends up winning the route). (whether it's first or last depends on how/where the node is added to the radix tree, and again later on how well the radix tree is searched, and whether a duplicate route search is bothered with, and then what final decision is made in the case of complete duplicates: (same network number, and same netmask, but 2 interfaces decide: turn left, turn right, eenie,meenie,minee,mo...Hmmmmm....). An unfortunate side-effect, but this is what always happens, because this type of use (ie: duplicate routes) was never originally intended in IP.
BTW: This is also an effective method of "black-holing" a route, (creating the route to an entire network on the loopback, -currently in use by routers at large, and the gated routing daemon on Unix machines). And as the term implies, all packets which enter therein, appear in another universe, never to be seen in this one again.
This probably explains what has happened to your eth0 routes, when lo0 was created into the same exact network. (Thereby duplicating all of the available criteria by which a decision on how to route the packets to that network can be made -woops, we turned left, shit...too late...).
A side-effect of this is that the "netstat" walk of the kernel routing table would fail to find the duplicate routes, -leading one to beleve that they were not there, (even though, in truth they are, but one is "covering up" the others because of the way the walk through the radix trees is done). The proof was in the pudding: On such an OS, you could remove the interface of the route that netstat -rn showed that network belonged to, then run the netstat -rn command again and see what interface the route "moved" to.... (What "really happened is this: - the route NEVER MOVED...it way always there, you have just uncovered it, by removing the duplicate route that was covering up the others existance when the radix tree / netmasks were searched). (because the final tie-breaker -the netmask was equal for both).
So, the action of defining multiple interfaces into the same network ( or "subnet", in pre-CIDR terms ) is largely incorrect, unless you specifically use a third criteria that will be used to determine the "tie-breaker" situation that you have created.
Some imbedded operating systems, (used in routers) do implement this third criteria, (known largely as "cost") directly on interfaces, so they have an additional determining factor to route packets by... (as in gated/OSPF, route with the lowest cost to same destination wins). In addition, they implement a round-robin feature (for a rough implementation of load-balancing) should even the costs of the two-or-more interfaces be found to be equal. They may even have algorithms for "true" load balancing along "equal-cost multipath" routes based on bandwidth availability. but then so much horsepower is needed to make up for the time spent in constantly recalculating this, that CPU bandwidth can quickly become a determining factor (clients comparison-shop Unixen for raw speed) and many win sales based on which is the "fastest"....therefore adding such an implementation into the kernel which consumes more CPU, for what appears to be a fringe market looking for special features, can cause the OS to lose market share). (what a shame, but true).
Pure routers whose entire CPU can be dedicated to the tasks of routing, may not need address such concerns as nearly as full operating systems - which are expected to reserve CPU for user-space applications (databases, data mining, etc.)
So, if you are going to need to use equal-cost multipath and/or true bandwidth availability based load-balancing algorithms (the only legitimate reasons you might have for configuring multiple interfaces into the same subnet) - then a dedicated router is what fits best here. (It's IOS doesn't have to meet the same SPECint design criteria for CPU consumption that a full blown multi-user OS is expected to). In fact, routers tend to be over-blown with routing features, because they do not need to leave ANY CPU time for users...(what users?- there are no users on a router, so the "who-cares, it's all our CPU anyway" attitude is quickly adopted in router design work - giving them a distinct advantage in the bells-and-whistles dept.).
If you want to configure a machine to respond to aliases in such a way that is network independent, adding them to the loopback is an acceptable way of doing this, but a netmask of all ones (/32) should be used, to make the route more specific than those of any competing network interfaces, and to make sure you don't run into the black-hole mechanism which is used (by design) to simplify CIDR based routing schemes.
Additionally, there are many other penalties (other than address depletion) to consider when using aliases, but I'll save that as it is not pertinent here.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| ![\](/images/icornerr.gif) |