Routers running OSPF transition through several states before establishing an adjacency. These different states are the Down, Attempt, Init, 2-Way, Exstart, Exchange, Loading and Full states. The preferred state for an OSPF adjacency is the Full state. This state indicates that the neighbors have exchanged their entire databases and both have the same view of the network. While the Full state is the preferred adjacency state, it is possible that during the adjacency establishment process, the neighbors get ‘stuck’ in one of the other states. For this reason, it is important to understand what to look for in order to troubleshoot the issue. The following sections describe the following problems pertaining to OSPF neighbor relationships:
- The Neighbor Table is Empty
- The Neighbor is stuck in the ATTEMPT state
- The Neighbor is stuck in the INIT state
- The Neighbor is stuck in the 2WAY state
- The Neighbor is stuck in the EXSTART/EXCHANGE state
- The Neighbor is stuck in the LOADING state
The Neighbor Table is Empty
There are several reasons why the OSPF Neighbor Table may be empty, i.e. why the output of the show ip ospf neighbor command might not yield any results. Common reasons are:
- Basic OSPF Misconfigurations
- Layer 1 and Layer 2 Issues
- Access List Filtering
- Interface Misconfigurations
Basic OSPF misconfigurations span a broad number of things. These could include mismatched timers, area IDs, authentication parameters, and stub configuration for example. There are a plethora of tools available in Cisco IOS software to troubleshoot basic OSPF misconfigurations. For example, you could use the show ip protocols command to determine information such as what networks OSPF is enabled for; the show ip ospf command to determine area configuration and the interfaces per area; and the show ip ospf interface brief command to determine what interfaces reside in what area and what OSPF process ID those interfaces have been enabled for, assuming that OSPF has been enabled for the interface.
Another common misconfiguration is specifying the interface as passive. If this is so, then the interface will not send out Hello packets, and a neighbor relationship will not be established using that interface. You can verify which interfaces have been configured or specified as passive using either the show ip protocols or the show ip ospf interface commands. Following is a sample output of the latter command on a passive interface:
| R1#show ip ospf interface serial 0/0 Serial0/0 is up, line protocol is up Internet Address 172.16.0.1/30, Area 0 Process ID 1, Router ID 10.1.0.1, Network Type POINT_TO_POINT, Cost: 64 Transmit Delay is 1 sec, State POINT_TO_POINT Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5 oob-resync timeout 40 No Hellos (Passive interface) Supports Link-local Signaling (LLS) Index 1/1, flood queue length 0 Next 0x0(0)/0x0(0) Last flood scan length is 0, maximum is 0 Last flood scan time is 0 msec, maximum is 0 msec Neighbor Count is 0, Adjacent neighbor count is 0 Suppress hello for 0 neighbor(s) |
And finally, when enabling OSPF over NBMA technologies such as Frame Relay, remember that the neighbors must be statically defined otherwise as OSPF does not use Multicast transmission for neighbor discovery for the default non-Broadcast network type. This is a common reason for empty Neighbor Tables when implementing OSPF.
Layer 1 and Layer 2 issues can also result in a formation of OSPF neighbor relationships. Layer 1 and Layer 2 troubleshooting was described in detail in the previous section. Use commands such as the show interfaces command to check for interface status, i.e. line protocol, as well as any received errors on the interface. If the OSPF-enabled routers reside in a VLAN that spans multiple switches, verify that there is end-to-end connectivity within the VLAN and that all ports or interfaces are in the correct Spanning Tree states, for example.
ACL filtering is another common cause for adjacencies failing to establish. It is important to be familiar with the topology in order to troubleshoot such issues. For example, if the routers failing to establish an adjacency are connected via different physical switches, it may be that the ACL filtering is being implemented in the form of a VACL that has been configured on the switches for security purposes. A useful troubleshooting tool that may indicate that OSPF packets are either being blocked or discarded is the show ip ospf traffic command which prints information on transmitted and sent OSPF packets as illustrated in the output below:
| R1#show ip ospf traffic serial 0/0Interface Serial0/0
OSPF packets received/sent OSPF header errors OSPF LSA errors |
In the output above, notice that the local router is sending OSPF Hello packets but is not receiving any. If the configuration on the routers is correct, check ACLs on the routers or intermediate devices to ensure that OSPF packets are not being filtered or discarded.
Another common reason for an empty Neighbor Table is interface misconfigurations. Similar to EIGRP, OSPF will not establish a neighbor relationship using secondary interface addresses. However, unlike EIGRP, OSPF will also not establish a neighbor relationship if interface subnet masks are not consistent.
EIGRP-enabled routers will establish neighbor relationships even if the interface subnet masks are different. For example, if two routers, one with an interface using the address 10.1.1.1/24 and another with an interface using the address 10.1.1.2/30 are configured in back-to-back EIGRP implementation, they will successfully establish a neighbor relationship. However, it should be noted that such implementations can cause routing loops between the routers. Because such implementations fall outside of the range of the TSHOOT exam requirements, they will not be described in any further detail in this chapter. In addition to mismatched subnet masks, EIGRP-enabled routers also ignore MTU configurations and will still establish neighbor relationships even if the interface MTU values are different. Use the show ip interfaces and show interfacescommand to verify IP address and mask configuration.
The Neighbor is stuck in the ATTEMPT state
The Attempt state is valid only for OSPF neighbors on NBMA networks. This state means that a Hello has been sent but no information has been received from the statically configured neighbor, however, some effort is being made to establish an adjacency with this neighbor. There are several possible reasons for the adjacency to remain in this state. These include:
- Incorrect NBMA Configuration
- Incorrect OSPF Configuration
- ACL Filtering
- Unidirectional Connectivity
Static Frame Relay mappings are susceptible to misconfigurations. When configuring OSPF over NBMA technologies, make sure that the DLCI or PVC mappings are correct. Use the appropriate Cisco IOS commands to verify bidirectional communication. For example, you could use theshow frame-relay pvc command to perform such validations for Frame Relay networks as follows:
| R1#show frame-relay pvc 100PVC Statistics for interface Serial0/0 (Frame Relay DCE)DLCI = 100, DLCI USAGE = LOCAL, PVC STATUS = ACTIVE, INTERFACE = Serial0/0
input pkts 22 output pkts 18 in bytes 2050 |
NOTE: An alternative would be to use the debug serial interface command as follows:
| R1#debug serial interface Serial network interface debugging is on R1# *Mar 5 10:23:38.854: Serial0/0(in): StEnq, myseq 26 *Mar 5 10:23:38.854: Serial0/0(out): Status, myseq 27, yourseen 27, DCE up *Mar 5 10:23:48.855: Serial0/0(in): StEnq, myseq 27 *Mar 5 10:23:48.855: Serial0/0(out): Status, myseq 28, yourseen 28, DCE up *Mar 5 10:23:58.855: Serial0/0(in): StEnq, myseq 28 *Mar 5 10:23:58.855: Serial0/0(out): Status, myseq 29, yourseen 29, DCE up *Mar 5 10:24:08.855: Serial0/0(in): StEnq, myseq 29 *Mar 5 10:24:08.855: Serial0/0(out): Status, myseq 30, yourseen 30, DCE up *Mar 5 10:24:18.856: Serial0/0(in): StEnq, myseq 30 *Mar 5 10:24:18.856: Serial0/0(out): Status, myseq 31, yourseen 31, DCE up |
Incorrect OSPF configuration can also result in neighbors being stuck in the attempt state. This includes typographical errors, such as typing 172.16.1.1 instead of 172.16.11.1 for example. Check the configuration to ensure that neighbor statements are configured correctly using the correct IP addresses (remember to use the interface address and not the router ID).
As always, check for ACL filtering and ensure that OSPF packets are being allowed between the endpoints. Keeping in mind that OSPF uses Unicast packets and not Multicast on NBMA networks, check for ACL or other filtering configurations that may prevent host-to-host connectivity between the endpoints. And finally, if you suspect a unidirectional connectivity issue over the NBMA network, use the appropriate technology show and debug commands to identify and resolve this issue. This may include Layer 1 and Layer 2 troubleshooting.
Following is the output of the show ip ospf neighbor command showing a neighbor stuck in the ATTEMPT/DROTHER state:
| R1#show ip ospf neighborNeighbor ID Pri State Dead Time Address Interface N/A 0 ATTEMPT/DROTHER – 172.16.0.2 Serial0/0 |
The Neighbor is stuck in the INIT state
The Init state is reached when an OSPF router receives a Hello packet from a neighbor but the local RID is not listed in the received Neighbor field. When OSPF routers are stuck in the INIT state, it typically indicates a unidirectional communication issue, i.e. the neighbor is not receiving the Hellos sent by the local router and is therefore not including the RID of that originating or sending router its Hello packet(s). Some common causes for this scenario are:
- ACL Filtering on One Side
- Layer 1 and Layer 2 Issues
- NBMA Misconfigurations
It is possible that an ACL configured on either one of the devices could prevent the OSPF Hello packets from being received by the local router. The result of this would be that one router would reflect an empty Neighbor Table while the other router would be stuck in the INIT state. Consider the topology illustrated in Figure 6-2:
Fig. 6-2. OSPF Neighbors Stuck in the INIT State
Figure 6-2 illustrates a basic network comprised of two routers. Assuming that all configuration parameters are correct, the routers should be able to establish an adjacency. However, we will assume that the following ACL has been configured on R1 and applied inbound on Serial0/0:
| R1#show ip access-lists NETWORK-SECURITY Extended IP access list NETWORK-SECURITY 10 deny ip host 0.0.0.0 any log 20 deny ip 127.0.0.0 0.255.255.255 any log 30 deny ip 10.0.0.0 0.255.255.255 any log 40 deny ip 172.16.0.0 0.15.255.255 any log 50 deny ip 192.168.0.0 0.0.255.255 any log 60 deny ip any 224.0.0.0 15.255.255.255 log (16 matches) 70 permit ip any any |
Based on this configuration, R1 does not receive any Hello packets from R3 and will therefore show an empty Neighbor Table as it is unaware of that router. However, because there is no ACL applied inbound on the Serial0/0 interface of R2, and OSPF is enabled on the 172.16.0.0/30 subnet, R3 will receive the Hello packets from R1. The problem arises due to the fact that because R1 never receives R2s Hello packets, it will never include that routers RID in the Hello packets it sends out. Given this, R2 becomes stuck in the INIT state. These two states can be validated using the show ip ospf neighbor command on both routers as follows:
| R1#show ip ospf neighborR1# R1# |
The same command, if issued on R3, would show the following instead:
| R3#show ip ospf neighborNeighbor ID Pri State Dead Time Address Interface 1.1.1.1 0 INIT/ – 00:00:36 150.0.0.1 Serial0/0 R3# |
This problem can be resolved by reconfiguring the ACL applied to R1s Serial0/0 as follows:
| R1(config)#ip access-list extended NETWORK-SECURITY R1(config-ext-nacl)#no 60 R1(config-ext-nacl)#60 deny ip 224.0.0.0 15.255.255.255 any log R1(config-ext-nacl)# *Mar 5 11:19:57.629: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Serial0/0 from LOADING to FULL, Loading Done |
The same problem can also be caused by Layer 1 and Layer 2 issues, as well as basic NBMA misconfigurations, such as forgetting to append the broadcast keyword to the end of theframe-relay map statements. Follow the same basic process to troubleshoot the issue.
The Neighbor is stuck in the 2WAY state
The 2-Way state indicates bi-directional communication with the OSPF neighbor(s). This means that the local router has received a Hello packet with its own RID in the Neighbor field and Hello packet parameters are identical on the two routers. On multi-access networks, the DR and BDR routers are elected during this phase. It should be noted that this state is perfectly acceptable for non-DR/BDR routers as they will never complete database exchange between themselves and thus never reach the FULL state. Consider the network illustrated in Figure 6-3:
Fig. 6-3. OSPF DR and BDR Fundamentals
Referencing Figure 6-3, each router on the segment establishes an adjacency with the DR and BDR, but not with each other. In other words, non-DR/BDR routers do not establish an adjacency with each other. This prevents the routers on the segment from forming N(N-1) adjacencies with each other, which reduces excessive OSPF packet flooding on the segment.
Assuming that all routers on the segment have the default priority, R4 is elected Designated Router for the segment because it has the highest router ID. R3 is elected as Backup Designated Router for the segment because it has the second highest router ID. Because R2 and R1 are neither the DR nor the BDR, they are referred to as DROther routers. This can be validated using the show ip ospf neighbor command on all routers as follows:
| R1#show ip ospf neighborNeighbor ID Pri State Dead Time Address Interface 2.2.2.2 1 2WAY/DROTHER 00:00:38 192.168.1.2 Ethernet0/0 3.3.3.3 1 FULL/BDR 00:00:39 192.168.1.3 Ethernet0/0 4.4.4.4 1 FULL/DR 00:00:38 192.168.1.4 Ethernet0/0 |
| R2#show ip ospf neighborNeighbor ID Pri State Dead Time Address Interface 1.1.1.1 1 2WAY/DROTHER 00:00:32 192.168.1.1 FastEthernet0/0 3.3.3.3 1 FULL/BDR 00:00:33 192.168.1.3 FastEthernet0/0 4.4.4.4 1 FULL/DR 00:00:32 192.168.1.4 FastEthernet0/0 |
| R3#show ip ospf neighborNeighbor ID Pri State Dead Time Address Interface 1.1.1.1 1 FULL/DROTHER 00:00:36 192.168.1.1 FastEthernet0/0 2.2.2.2 1 FULL/DROTHER 00:00:36 192.168.1.2 FastEthernet0/0 4.4.4.4 1 FULL/DR 00:00:35 192.168.1.4 FastEthernet0/0 |
| R4#show ip ospf neighborNeighbor ID Pri State Dead Time Address Interface 1.1.1.1 1 FULL/DROTHER 00:00:39 192.168.1.1 FastEthernet0/0 2.2.2.2 1 FULL/DROTHER 00:00:39 192.168.1.2 FastEthernet0/0 3.3.3.3 1 FULL/BDR 00:00:30 192.168.1.3 FastEthernet0/0 |
Notice that the DROther routers remain in the 2WAY/DROTHER state. This is normal because they only exchange their databases with the DR and BDR routers. Therefore, because there is no full database exchange between the DROther routers, they will never reach the FULL state.
While the 2WAY state is acceptable in scenarios such as the above, it is not fine when only two routers reside on a multi-access segment. In such cases, the cause of this problem is due to both routers being configured with a priority of 0 via the ip ospf priority command. You can troubleshoot this issue using the show ip ospf neighbor and show ip ospf interfacecommands. Following is a sample output of the show ip ospf neighbor command when a priority of zero has been configured for the neighbor:
| R1#show ip ospf neighborNeighbor ID Pri State Dead Time Address Interface 3.3.3.3 0 2WAY/DROTHER 00:00:35 150.0.0.2 FastEthernet0/0 |
Following is the output of the show ip ospf interface command on the same router:
| R1#show ip ospf interface fastethernet 0/0 FastEthernet0/0 is up, line protocol is up Internet Address 150.0.0.1/24, Area 0 Process ID 1, Router ID 1.1.1.1, Network Type BROADCAST, Cost: 1 Transmit Delay is 1 sec, State DROTHER, Priority 0 No designated router on this network No backup designated router on this network Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5 oob-resync timeout 40 Hello due in 00:00:08 Supports Link-local Signaling (LLS) Index 1/1, flood queue length 0 Next 0x0(0)/0x0(0) Last flood scan length is 1, maximum is 1 Last flood scan time is 0 msec, maximum is 0 msec Neighbor Count is 1, Adjacent neighbor count is 0 Suppress hello for 0 neighbor(s) |
The Neighbor is stuck in the EXSTART/EXCHANGE state
The Exstart state is used for the initialization of the database synchronization process. It is at this stage that the local router and its neighbor establish which router is in charge of the database synchronization process. The Master and Slave are elected in this state and the first sequence number for DBD exchange is decided by the Master in this stage.
The Exchange state is where routers describe the contents of their databases using DBD packets. Each DBD sequence is explicitly acknowledged, and only one outstanding DBD is allowed at a time. During this phase, LSR packets are also sent to request a new instance of the LSA. The M (More) bit is used to request missing information during this stage. When both routers have exchanged their complete databases, they will both set the M bit to 0.
There are several possible reasons for the neighbor to be stuck in the EXSTART/EXCHANGE state. Common reasons include the following:
- Mismatched MTU Values
- Duplicate RIDs
- Broken Unicast Connectivity
While EIGRP ignores interface MTU values and allows routers with different interface MTUs to establish a neighbor relationship, the same is not applicable with OSPF. Referencing the topology illustrated in Figure 6-2, which depicts routers R1 and R2 connected via a back-to-back Serial connection, the following shows the neighbor state on R1 assuming an MTU mismatch:
| R1#show ip ospf neighborNeighbor ID Pri State Dead Time Address Interface 3.3.3.3 0 EXCHANGE/ – 00:00:38 150.0.0.2 Serial0/0 |
The same command would show the following output on R3:
| R3#show ip ospf neighborNeighbor ID Pri State Dead Time Address Interface 1.1.1.1 0 EXSTART/ – 00:00:39 150.0.0.1 Serial0/0 |
The simplest way to troubleshoot this issue is to validate interface MTU configurations by using the show running-config interface [name] privileged EXEC command. Alternatively, interface MTUs are also printed in the output of the show interfaces and show ip interfacescommands. Following is a sample output of the information printed by the latter:
| R1#show ip interface serial 0/0 Serial0/0 is up, line protocol is up Internet address is 150.0.0.1/30 Broadcast address is 255.255.255.255 Address determined by setup command MTU is 1504 bytes Helper address is not set Directed broadcast forwarding is disabled Multicast reserved groups joined: 224.0.0.5 Outgoing access list is not set Inbound access list is NETWORK-SECURITY Proxy ARP is enabled |
The EXSTART/EXCHANGE state can also be caused by duplicate router IDs or broken Unicast connectivity between the routers. As is the case with mismatched MTU values, the troubleshooting process for duplicate RIDs should begin by simply looking at device configurations to ensure that the same RID was not entered on two different devices. The Unidirectional issues may be caused by Layer 1 / Layer 2 issues, as well as by Layer 3 functions such as Access Control Lists. Eliminate each layer as you troubleshoot to isolate the cause.
The Neighbor is stuck in the LOADING state
In the Loading state OSPF routers build an LSR and Link State Retransmission list. LSR packets are sent to request the more recent instance of an LSA that has not been received during the Exchange process. Updates that are sent during this phase are placed on the Link State Retransmission list until the local router receives an acknowledgement. If the local router also receives an LSR during this phase, it will respond with a Link State Update that contains the requested information. It is rare to see a neighbor stuck in the LOADING state. However, a corrupted Link State Request packet may cause this issue.
If such an event occurs, troubleshoot the issue by looking at error counters on the applicable interfaces and use a component swapping approach to attempt to isolate the problem. Typically, such issues are caused by faulty hardware, which may then need to be replaced.