Search this Blog

Friday, January 31, 2014

How is the cisco devices generate a default route in OSPF domain?

What are the different ways to generate a default route in OSPF domain and what are the configuration steps involved?

Cisco device uses “default-information originate” statement to generate a default route in OSPF domain. There are two ways to originate default route by using this command:
1) When you configure “default-information originate”  under OSPF process  without any argument after this statement, OSPF process will  first check,if any default route is already present in routing table. If a default route is already present in routing table via static or any dynamic protocol, OSPF originate default route. If default route does not exist in routing table, OSPF will not originate default route.
2)This behavior can be modified by adding "always" argument to the "default-infomation orginate" statement,which essentially skips the checking for a default route already being installed in the table.

This document discuss about conditional route origination in OSPF domain. Conditional default route in OSPF originate by using route-map in default-information command under router OSPF process. The route map configured in the default-information originates command check the existing IP prefixes in the IP routing table.

Configuration steps:

1) prefix-list or access-list: To originate default route only when necessary prefixes are present in routing table.First you need to match these prefixes by configuring prefix-list or access-list.

Prefix-list configuration:

Router (config)#ip prefix-list Default_route sequence 10 permit

Access-list configuration:

Router(config)#ip access-list standard Default_route
Router(config-std-nacl)# permit

2) Attach prefix-list or access-list in route map:


Router(config)#route-map Ospf_default permit 10
Router(config-route-map)#match ip address prefix-list Default_route

3) Configure “default-information originate always route-map” statement:


Router(config)#router ospf 100
Router(config-router)#default-information originate always route-map Ospf_default 


In the below topology R1 is connected to two ISP using serial interface. The router R1 and R2 runs OSPF in Area 0, where R1 is advertising default route to R2 only when both serial links are up. Once both link goes down R1 should withdraw default route from OSPF domain.

Topology Diagram:


Configuring R1:

R1(config)#ip prefix-list default_route sequence 10 permit
R1(config)#ip prefix-list default_route sequence 20 permit
R1(config)#route-map ospf_default permit 10
R1(config-route-map)#match ip add prefix-list default_route
R1(config)#router ospf 100
R1(config-router)#default-information originate always route-map ospf_default

R1 advertise a default route to OSPF domain, as both ISP links are up.

Verifying default route on R2

R2#sh ip ospf database | in     371         0x80000001 0x0018CA 100
R2#sh ip route ospf
O*E2 [110/1] via, 00:06:22, FastEthernet0/0

We need at least one prefix match in list present in R1’s routing table to advertise default route in OSPF domain.

Disabling one link towards ISP1:

R1(config)#int s0/0
*Mar  1 00:17:09.971: %LINK-5-CHANGED: Interface Serial0/0, changed state to administratively down
*Mar  1 00:17:10.971: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/0, changed state to down

Verification on R2:

R2#sh ip route ospf
O*E2 [110/1] via, 00:10:30, FastEthernet0/0

Disabling remaining link towards ISP2:

Verification on R2:

R1(config-if)#int s0/1
*Mar  1 00:18:39.131: %SYS-5-CONFIG_I: Configured from console by console
*Mar  1 00:18:39.731: %LINK-5-CHANGED: Interface Serial0/1, changed state to administratively down

Turned on debug in R2 to get closer view of default route deletion

R2#debug ip routing
IP routing debugging is on
*Mar  1 00:19:25.799: RT: del via, ospf metric [110/1]
*Mar  1 00:19:25.803: RT: delete network route to
*Mar  1 00:19:25.803: RT: NET-RED
*Mar  1 00:19:25.803: RT: NET-RED
R2#sh ip route ospf

From the above output it is clear that when the both links connected to ISP goes down, router R1 stop generation of default route to OSPF domain.

Citation - This blog post does not reflect original content from the author. Rather it summarizes content that are relevant to the topic from different sources in the web. The sources might include any online discussion boards, forums, websites and others.

Thursday, January 30, 2014

How can we retrieve core files from Cisco Nexus switching platforms ?

What are the  common steps and commands used to retrieve core files in Nexus switching platforms -  Nexus7000, Nexus5000, Nexus 4000, Nexus 3000 and Nexus2000.

When a specific process (called as Service) crahes, the device should report a log message, as follows:

Scenario 1:

%SYSMGR-2-SERVICE_CRASHED: Service "vpc" (PID 5883) hasn't caught signal 11 (core will be saved)

Here, service "vpc" has crashed and a core file will be saved.

Scenario 2:

The device may report message, with no core file created.

%SYSMGR-2-SERVICE_CRASHED: Service "stp" (PID 4668) hasn't caught signal 9 (no core).

Here, sevice "stp" crashed but has not generated any core file.

Retreiving Core files:

For Scenario 1 (as mentioned above):

If there is a process crash/exception reported and the switch has NOT reloaded (since the exception/crash), then do "show cores" to get list of cores.

N7K# show cores
VDC Module Instance Process-name     PID       Date(Year-Month-Day Time)
--- ------ -------- --------------- -------- -------------------------
1   6       1         vpc             4763     2011-01-10 11:33:01
1   6       1         vpc             5883     2011-01-10 11:33:05

Please do "show cores vdc-all" to see core files in all VDCs.

The above results indicate that the exception was reported for "vpc" service in VDC #1, Module #6.
The results provide different Process ID (PID) - 4763 and 5883 - the specific process had at exception, with timestamps.
Instance number will be useful to identify the core files when a specific process with same PID (for the same VDC) experience multiple exceptions.

Please be aware that "show cores" command do NOT provide any information,  if the switch has rebooted since the exception.

To copy the core files to FTP or TFTP server, follow the steps:

N7K# copy core:?
   core: Enter URL "core:///[/instance-num]“

N7K# copy core://6/4763/1 ?
   bootflash: Select destination filesystem
   ftp:       Select destination filesystem
   scp:       Select destination filesystem
   sftp:      Select destination filesystem
   slot0:     Select destination filesystem
   tftp:      Select destination filesystem
   usb1:      Select destination filesystem
   usb2:      Select destination filesystem

The above command collects all relevant info (system info, log files etc.) from the switch and bundles them into .tar file.
It is NOT recommended to copy files directly from different filesystems manually.

If the switch has rebooted, do following command to see if there are core files generated earlier:

N7K# dir logflash://sup-1/core 
100499456   Aug 29 22:36:54 2011 0x501_ethpm_core.16574
   8638991   Aug 29 22:45:14 2011 0x501_ethpm_core.4165.gz
     37139   Aug 29 22:36:54 2011 0x501_ethpm_log.16574
   7699061   Aug 29 22:36:32 2011 0x501_ethpm_log.16576.tar.gz
   8208542   Aug 29 22:36:32 2011 0x501_ethpm_log.4165.tar.gz
   7698622   Aug 29 22:45:30 2011 1314657930_0x501_ethpm_log.16576.tar.gz
   8208230   Aug 29 22:45:30 2011 1314657930_0x501_ethpm_log.4165.tar.gz

If there is Supervisor failover occurred, please check the other/standby sup for core files.

N7K# dir logflash://sup-2/core

In Nexus5000, Nexus4000, Nexus3000 and Nexus2000 platforms, as there is no supervisor engine redundancy, there will not be any failover.


In Nexus5000, Nexus4000, Nexus3000 and Nexus2000 platforms the core files are stored in the "volatile:" and not in the "logflash:" file system.

N3k-3# dir volatile:?

Please be aware that contents of "volatile:" file system are flushed on reload.

For Scenario 2 (as mentioned above):

N7K# show process log vdc-all
VDC Process         PID     Normal-exit Stack Core   Log-create-time
--- --------------- ------ ----------- ----- ----- ---------------
1 installer       10544             N     N     N Thu Jun 10 17:49:21 2010
1 ethpm           16574             N     Y     N Mon Aug 29 22:36:15 2011

Here, the "ethpm" sevice crashed and generated "Stack" (flagged with Y) but no "Core" file (flagged with N).
At the same, for the "installer" process, neither "Stack" nor "Core" file is generated.

For the "installer" process, furher information can be obtained by:

N7K# show process log pid 10544
Service: installer
Description: Installer
Started at Thu Jun 10 17:45:42 2010 (483528 us)
Stopped at Thu Jun 10 17:49:21 2010 (719259 us)
Uptime: 3 minutes 39 seconds
Last heartbeat 0.00 secs ago
RLIMIT_AS: 69909875
System image name: n7000-s1-dk9.4.2.4.bin
System image version: 4.2(4) S32
PID: 10544
SAP: 0

For the "ethpm" process, the stack trace can be obtained by:

N7K# show process log pid 16574
Service: ethpm
Description: Test Ethernet Port Manager
Executable: /isan/bin/ethpm
Started at Mon Aug 29 22:36:15 2011 (188136 us)
Stopped at Mon Aug 29 22:36:15 2011 (746741 us)
Uptime: 0 seconds
Virtual Memory:
   CODE     08048000 - 08356C90
   DATA     08357000 - 08369BA8
   BRK       083F0000 - 086F9000
   STACK     BFBB25C0
   TOTAL     98996 KB
Memory Map: 08048000 ethp 08357000 ethp 4143F000 ld-2.8.s 41459000 ld-2.8.s 4145
A000 ld-2.8.s 4145D000 libc-2.8.s 41596000 libc-2.8.s 41598000 libc-2.8.s 4159E0
Register Set:
   EBX BFBB0ADC         ECX 00000000         EDX 00000002
   ESI BFBB15B0         EDI 00000009         EBP BFBB1148
Stack: 6976 bytes. ESP BFBB0A80, TOP BFBB25C0
0xBFBB0A80: 0000001F 00000000 00000000 00000001 ................

 Why the core file is missing ? :

If the switch does not have enough space in the specific filesystem (logflash: or volatile: depending on the platform), then the core file may not be successfully generated/stored.

N7K# dir logflash://sup-1/
Usage for logflash://sup-1
  498237440 bytes used
7394926592 bytes free
7893164032 bytes total

To check the free space available in different file systems, you can also do:

N7K# show system internal flash
Mount-on                  1K-blocks      Used   Available   Use%  Filesystem
/                            409600     61372      348228     15   /dev/root
/proc                             0         0           0      0   proc
/sys                              0         0           0      0   none
/isan                       1048576    339184      709392     33   none
/bootflash                  1809684    673252     1044504     40   /dev/hda3
/logflash                   7708168     95004     7221608      2   /dev/hde1
/bootflash_sup-remote       1809688    672952     1044808     40
/logflash_sup-remote        7708168     34976     7281640      1

Same set of commands, from a Nexus3000 switch:

N3K# dir volatile://sup-1/
Usage for volatile://sup-1
          0 bytes used
  104857600 bytes free
  104857600 bytes total

N3K# sh system internal flash
Mount-on                  1K-blocks      Used   Available   Use%  Filesystem
/                            204800    112436       92364     55   /dev/root
/proc                             0         0           0      0   proc
/post                          2048         4        2044      1   none
/sys                              0         0           0      0   none
/volatile                    102400         0      102400      0   none
/debug                        20480         8       20472      1   none
/bootflash                  1609984    582492      945708     39   /dev/sda3

 Logs/Files to Capture:

If further analysis required on process exception / core files, please open a Service Request and send following logs:

- show cores vdc-all
- Core files saved using "copy core:///[/instance-num]..." command
- show process log vdc-all
- show process log details
- show logging onboard internal reset-reason
- show logging onboard stack-trace
- show logging onboard kernel-trace
- show module internal exceptionlog module

Please make sure all these logs are captured to a file(s), as the logs may go several pages.

Citation - This blog post does not reflect original content from the author. Rather it summarizes content that are relevant to the topic from different sources in the web. The sources might include any online discussion boards, forums, websites and others.
/* Google Analytics begin ----------------------------------------------- */ /* Google Analytics end ----------------------------------------------- */