Tuesday, February 24, 2009

Why linux keeps connected routes when link goes down?

I came across this interesting discussion on a particular behavior of linux. This issues was brought up by a system administrator who was facing an interesting situation. When the link of an interface goes down possibly due to hardware failure or cable pull out, the ip address and routes associated with that interface is retained until someone explicitly deletes it. This behavior is built into linux for more than 10 years and the reason for this was stated by David Miller.

The default behavior for a general purpose operating system is
to increase the likelyhood of successful communication.

And the way to maximize successful communication is to associate
addresses with the host rather than the interface.
It becomes clear why the developers decided to go with associating ip address with a particular node rather than an interface in the node. The problem with this implementation is relevant when linux is being used as a router. A router is a multi-homed host and tends to have multiple interfaces each connected to different network. So if a situation arises where the router finds a route entry to send a particular packet there are chances that the link may be down and packet never reaches the destination. Even if there are alternated routes to reach the particular destination it cannot be used as there is already an existing valid route with a interface that is down. Lennart Sorensen argued against David Miller's claim that different interface of the router might be connected to same network in which case the alternate interface can be taken. According to Lennart it still useless as we would end up having 2 route entries to the same destination with different interfaces and would pose the same problem when the first route entry points to an interface which is down. The no conclusion to this little problem as David points out in his replay to Lennart.

This decision was made at least 10 years ago, and if you think what we
have now is broken just imagine how much struff would explode if we
changed things.
All is not lost in this regard for people using linux as router. Stephen Hemminger suggested using quagga for this purpose. There are patches submitted to the quagga mailing list which does what Lennart wants without any modification to the kernel behaviour. Unfortunately those patches are not yet upstream in the quagga branch and has to be applied as patch. So in future would quagga become the routing management daemon for linux? Thats something we need to watch out.


Matthew Jakeman said...

Since when is a router always a multi homed host as you state? Many routers are single homed, this is an important distinction.

varun said...

it not necessary to have a linux as a multi homes system, this discussion is based on routers that have multiple interfaces (2 atleast)