When configuring routing on your network devices, you have to carefully configure static or dynamic routing based on the design. If you have a problem and are not able to send/receive traffic across the network, then you probably have some kind of configuration issue. When you initially set up a router there will most likely be some type of configuration problem that you will have to troubleshoot. If the router has been running for a while and you suddenly have a complete failure of traffic (no communication), you should analyse the situation and figure out whether the routing protocol functions as expected.
Sometimes certain routes might intermittently disappear and appear from the routing table, causing intermittent connectivity to specific destinations. This may be because a certain network area has some kind of communication problem and routers along the path propagate new routing information every time that area becomes accessible. This process is called “route flapping,” and the specific routes can be blocked so the entire network is not affected using a feature called “route dampening.”
| Note: When using static routing, the routing table never changes so you will have no idea about problems that occur in different network areas. |
When troubleshooting routing issues the standard approach is to follow the routing table for every route along the path. You might perform a traceroute to find out exactly where the packets are going and to see the routers along the path. This way you would know exactly which device might be causing the issues and you can start investigating the routing tables of the specific routers.
A common mistake when performing such a troubleshooting process is investigating the issue in a single direction (for example, source to destination). Instead, you should perform the troubleshooting in both directions because you might come across situations in which packets are blocked in a single direction and you have no return traffic from the destination to the source. The routing tables on devices along the path between two points should correctly point in both directions in order to ensure an optimal traffic flow.
Often you will be using connections provided by third parties, so when you want to troubleshoot issues in a certain area you should communicate with the provider and synchronise the troubleshooting effort. This includes sharing routing table information.
Using dynamic routing protocols makes the troubleshooting process easier because you can inspect the routing updates as they are sent and received by the router. This can be done via packet capture or internal device mechanisms and will help you to see how and when the routing table is populated. Having a topology map and other documentation that lists where every prefix is located in the network would further help your understanding of the routing updates and would shorten the troubleshooting process. The general idea in such a troubleshooting process is deciding which path a specific packet should take, based on the network design, and investigating where exactly it is deviating from this path.
Different tools can be used to monitor network devices. A common network management protocol used by these tools is Simple Network Management Protocol (SNMP), which was designed to query network devices for different parameters from a management station (SNMP is covered in ICND2). Besides the standard “health” parameters checked (e.g., CPU, memory, disk space, etc.), SNMP can also query the router for things like:
- Interface packet counters
- Used bandwidth and throughput
- CRC or other types of errors on device interfaces
- Routing table information
Other types of tools you can use are standard ping and traceroute utilities in order to verify end-to-end connectivity. They can also show relevant output that might help you determine the point in the network where problems occur.
The steps to troubleshooting almost all routing issues include the following:
- Verifying that routing is enabled
- Verifying that the routing table is valid
- Verifying the correct path selection
Verifying That Routing Is Enabled
The first step in troubleshooting routing is verifying that the routing protocol is enabled and properly configured. This can be done either by inspecting the current running configuration (i.e., the show run command) or by using show commands associated with each particular routing protocol. Some of these options are listed below:
Router#show ip ospf ?
<1-65535> Process ID number
border-routers Border and boundary router information
database Database summary
flood-list Link state flood list
interface Interface information
max-metric Max-metric origination information
mpls MPLS related information
neighbor Neighbor list
request-list Link state request list
retransmission-list Link state retransmission list
rib Routing information base (RIB)
sham-links Sham link information
statistics Various OSPF Statistics
summary-address Summary-address redistribution information
timers OSPF timers information
traffic Traffic related statistics
virtual-links Virtual link information
| Output modifiers
<cr>
Router#show ip eigrp ?
<1-65535> Autonomous System
accounting IP-EIGRP accounting
interfaces IP-EIGRP interfaces
neighbors IP-EIGRP neighbors
topology IP-EIGRP topology table
traffic IP-EIGRP traffic statistics
vrf Select a VPN routing/forwarding instance
Router#show ip bgp ?
A.B.C.D Network in the BGP routing table to display
A.B.C.D/nn IP prefix <network>/<length>, e.g., 35.0.0.0/8
all All address families
cidr-only Display only routes with non-natural netmasks
community Display routes matching the communities
community-list Display routes matching the community-list
dampening Display detailed information about dampening
extcommunity-list Display routes matching the extcommunity-list
filter-list Display routes conforming to the filter-list
inconsistent-as Display only routes with inconsistent origin ASs
injected-paths Display all injected paths
ipv4 Address family
ipv6 Address family
labels Display labels for IPv4 NLRI specific information
neighbors Detailed information on TCP and BGP neighbor connections
nsap Address family
oer-paths Display all oer controlled paths
paths Path information
peer-group Display information on peer-groups
pending-prefixes Display prefixes pending deletion
prefix-list Display routes matching the prefix-list
quote-regexp Display routes matching the AS path “regular expression”
regexp Display routes matching the AS path regular expression
replication Display replication status of update-group(s)
rib-failure Display bgp routes that failed to install in the routing table (RIB)
route-map Display routes matching the route-map
summary Summary of BGP neighbor status
template Display peer-policy/peer-session templates
update-group Display information on update-groups
vpnv4 Address family
| Output modifiers
<cr>
Verifying That the Routing Table Is Valid
After successfully determining that the routing process is enabled, the next step is to analyse the routing table and see whether the information listed there is valid. Some of the points you should focus on include:
- Verifying that the correct prefixes are being learned via the correct routing protocol
- Verifying the number of learned prefixes
- Verifying route metrics and next-hop information
Depending on the routing protocol, you should also verify that the correct prefixes are being advertised outbound from your device.
Verifying the Correct Path Selection
After verifying that the relevant prefixes are indeed present in the routing table, you should carefully analyse their attributes and the path selection method for each of them. This might include:
- Verifying all the routing protocols that advertise that specific prefix (including static routing)
- Comparing and manipulating the AD in order to prefer it over the correct routing protocol
- Verifying and adjusting protocol metrics
By properly configuring the router in your network, documenting each step along the way, and constantly monitoring the path between different points in the network, you will have a solid understanding of exactly how the traffic should traverse all the devices in the network.