Search this Blog

Wednesday, March 27, 2013

Memory Allocation Error with IOS 15.0(2)

We updated a switch (C3560G-48PS-S) from 12.2(58)SE2 to IOS 15.0(2)SE1. Some time after the upgrade we got error messages about memory allocations to our syslog server in regular intervals (each 30 seconds).

We weren't able to connect to the switch over SSH anymore. It wasn't even possible to access the CLI over the console cable (error message: "Low on memory; try again later"). After a reboot of the switch, it went fine for some hours, but the error appeared again. It seems that the switching process still works fine as there aren't any complaints of users about network issues.

Important: We installed the same IOS version to another switch of the type C2960-24PC-L and the memory allocation appeared there after some hours as well. We thought that this issue is maybe solved with the newest release of IOS for that particular device. But even with IOS 15.0(2)SE2 on the C2960-24PC-L the memory allocation error happens again. Just the traceback is a little bit different.

Does anyone have the same issue with IOS 15.0(2) as well? Could maybe give me someone a hint what to do for solving that issue?
 

Error Message of C3560G-48PS-S with IOS 15.0(2)SE1

031513: Feb 22 11:00:26.848: %SYS-2-MALLOCFAIL: Memory allocation of 65536 bytes failed from 0x2C13A88, alignment 0
Pool: Processor  Free: 693180  Cause: Memory fragmentation
Alternate Pool: None  Free: 0  Cause: No Alternate pool
-Process= "CDP Protocol", ipl= 0, pid= 205
-Traceback= 1FBAAF4z 2BF7F08z 2BFEAE4z 2C13A8Cz 1EB4DF4z 1EB8A0Cz 1EB8B00z 1E82974z 1A2ABF0z 1A2F1B0z 12EA4FCz 12EE85Cz 19EDC84z 19E83D8z


This issue appears to be a memory leak.  If you look at the memory there is a large increase (relatively speaking) in:


    PC          Total   Count  Name
0x00D03218    4065960      62  AAA AttrL Sub

In the capture at 0 hours this is holding about 65K, however after 6 hours it's risen to 4 MB.  As I mentioned before a small leak can have a large impact on a switch like this because it doesn't have much memory free to begin with.  This issue looks very similar to one the bug that was fixed by Cisco in 15.0(2)SE:

CSCty49762 (Cisco login required).
EAP Framework and AAA AttrL Sub Uses All Process Memory.
This bug deals with AAA and dot1x authentications.

You can apply the workaround from here (Cisco login required) and memory holding should stabilize.

 


Citation - This blog post does not reflect original content from the author. Rather it summarizes content that are relevant to the topic from different sources in the web. The sources might include any online discussion boards, forums, websites and others.

Tuesday, March 19, 2013

Cisco Catalyst WS-C2960S-48TD-L + SFP-10G-SR-AX issues

We ordered 2 WS-C2960S-48TD-L switches. We also ordered the Axiom equivelent to the SFP-10G-SR (SFP-10G-SR-AX) per the 10-Gigabit Ethernet Transceiver Modules Compatibility Matrix (http://www.cisco.com/en/US/docs/interfaces_modules/transceiver_modules/compatibility/matrix/OL_6974.html)
 

We have tested this issue with both the c2960s-universalk9-mz.122-55.SE5 and c2960s-universalk9-mz.150-2.SE2 (currently loading c2960s-universalk9-mz.122-55.SE5).

When inserting the SFP+ module, I am presented with the following:
*Mar  1 00:16:00.831: %GBIC_SECURITY_CRYPT-4-VN_DATA_CRC_ERROR: GBIC in port Gi1/0/49 has bad crc
*Mar  1 00:16:00.831: %PM-4-ERR_DISABLE: gbic-invalid error detected on Gi1/0/49, putting Gi1/0/49 in err-disable state

The first thing we noticed was that the output was related to Gi1/0/49, which is the other incorrect combo port for the 10GB transceiver modle.  We ran the following commands:
Switch(config)# service unsupported-transceiver
Switch(config)# no errdisable detect cause gbic-invalid

After running the commands, We are still getting the following:
*Mar  1 00:28:25.000: %GBIC_SECURITY_CRYPT-4-VN_DATA_CRC_ERROR: GBIC in port Gi1/0/49 has bad crc

With the above configurations present, or otherwise, the following is shown with the SFP+ module inserted:
Switch#sh ip int g1/0/49
GigabitEthernet1/0/49 is down, line protocol is down (notconnect)
  Hardware is Gigabit Ethernet, address is c062.6bbd.1f31 (bia c062.6bbd.1f31)

Switch#sh int t1/0/1
TenGigabitEthernet1/0/1 is down, line protocol is down (notconnect)
  Hardware is not present

Removing the SFP+ module:
Switch#sh int g1/0/49
GigabitEthernet1/0/49 is down, line protocol is down (notconnect)
  Hardware is not present

Switch#sh int t1/0/1
TenGigabitEthernet1/0/1 is down, line protocol is down (notconnect)
  Hardware is Ten Gigabit Ethernet, address is c062.6bbd.1f33 (bia c062.6bbd.1f33)

Why does the 10GB SFP+ module persuade the switch to mark the T1/0/1 interface as "Hardware Not Present", and choose the G1/0/49 port instead?


The Axiom-branded module may not work because the IDPROM inside has a MD5 code which does not match the Cisco's code check.  This is why the port went into error-disable.  This behavior is normal when you use a non-Cisco branded module.

We have seen the manufacturer of the SFP/SFP+ to reflash the module's IDPROM in order to "trick" the appliance.



Citation - This blog post does not reflect original content from the author. Rather it summarizes content that are relevant to the topic from different sources in the web. The sources might include any online discussion boards, forums, websites and others.

Thursday, March 14, 2013

Multiple Tacacs Groups for different Interfaces on a Router

A Cisco 888 is managed by us and a Provider Support Team. Since we both want to access our own TACACS Server, we want to create two TACACS Groups. Is it possible for us, to bind a Tacacs Group to one Interface, and the second TACACS Group to another ?

Means that our stuff is connecting to the LAN Interface FastEthernet0 that is applied to the SVI in VLAN 1.The service technicans from the Provider are connecting to the external Interface or through a possible Lo. (another IP). We do not want to mix our 2 TACACS+ Server and their's together in one Group. So have anybody tried this before ?

 
Please follows the steps below

- create one tacacs goup that specifies his authentication servers. Perhaps name it OURS.
-create one tacacs group that specifies the authentication servers for the Provider Support Team. Perhaps name it PST.
-create one named authentication method to authenticate using group OURS. Perhaps call the method INTERNAL.
- create one named authentication method to authenticate using group PST. Perhaps call the method EXTERNAL.
- configure several vty ports specifying authentication method INTERNAL and specifying transport input telnet.
- configure several other vty ports specifying authentication method EXTERNAL and specifying transport input ssh.

Then if the Provider Support Team will SSH to the router they will use the vty that authenticates with their tacacs server. And if he will telnet to the router then he will use the vty that authenticates with his tacacs server.


Citation - This blog post does not reflect original content from the author. Rather it summarizes content that are relevant to the topic from different sources in the web. The sources might include any online discussion boards, forums, websites and others

Tuesday, March 12, 2013

How do you connect a 4510 TenGig port to 2960 1Gb SFP?


We are planning on deploying a 2960 switch and will need to uplink it to a 4510 switch. There are 2 TenGig Ports available and we are  thinking of uplinking  one of them to the 1Gb SFP port on the 2960. Would this work? This is the model of the 2960: WS-C2960-48TC-L. It comes with the 2-Gig SFPs. We have the TenGig Converter for the 4510. 

You can connect the 4500 to a 2960 using a TwinGig converted.  This means that the link will be on a 1 Gbps. The 2960 will only talk 1 Gbps so your 4500 should also talk in the same speed or the link won't work. The only way to get this done in the 4500 is to use a TwinGig coverter.  The TwinGig converter is an insert that will convert a single 10 Gbps port into two 1 Gbps ports. 



Citation - This blog post does not reflect original content from the author. Rather it summarizes content that are relevant to the topic from different sources in the web. The sources might include any online discussion boards, forums, websites and others.

Tuesday, March 5, 2013

Nexus 5010 outage after enabling jumbo frames (mtu 9216)


We attempted to enable jumbo frames on a Nexus 5010 (NX-OS version 4.2(1)N1(1)).  We created the policy map below and lost access to the switch.

policy-map type network-qos jumbo
class type network-qos class-default
mtu 9216

After recovery we see from the logs that all vlans and interface were suspended.  We have attempted to look for reasons for a compatibility issue but we are unable to find what is checked and what could have been incompatible.  The other troubling thing is the adjacent switch suspended its interfaces too but no change was done there.  Can anyone explain what we have done wrong?

2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 1,10,601 on Interface port-channel1 are being suspen
ded. (Reason: QoSMgr Network QoS configuration incompatible)
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-5-IF_TRUNK_DOWN: Interface port-channel1, vlan 1,10,601 down
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 10 on Interface port-channel508 are being suspended.
(Reason: Global compat check failed)
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 10 on Interface port-channel507 are being suspended.
(Reason: Global compat check failed)
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 10 on Interface port-channel506 are being suspended.
(Reason: Global compat check failed)
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 10 on Interface port-channel505 are being suspended.
(Reason: Global compat check failed)
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 601 on Interface port-channel18 are being suspended.
(Reason: Global compat check failed)
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-5-IF_DOWN_INACTIVE: Interface port-channel508 is down (Inactive)
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-5-IF_DOWN_INACTIVE: Interface port-channel507 is down (Inactive)
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-5-IF_DOWN_INACTIVE: Interface port-channel506 is down (Inactive)
2011 Nov 22 23:43:09 phx-ipcg1dwfcma %ETHPORT-5-IF_DOWN_INACTIVE: Interface port-channel505 is down (Inactive)

Most probably this have happened because you have changed the compatibility requirements of the port-channel. Because on the Nexus 5000 for Layer 2 the MTU is changed on the global basis it had this effect.

Please click here for more information on configuring Ether Channels in Nexus 5000.

Please click here for more information on Configuration replace and Configuration roll back.



Citation - This blog post does not reflect original content from the author. Rather it summarizes content that are relevant to the topic from different sources in the web. The sources might include any online discussion boards, forums, websites and others.
 
/* Google Analytics begin ----------------------------------------------- */ /* Google Analytics end ----------------------------------------------- */