'Network'에 해당되는 글 9건
- 2010/06/08 Autonegotiation Valid Configuration
- 2010/01/08 QoS
- 2009/11/25 Configuring the Switch for the First Time
- 2009/08/12 유비쿼스 premier 7024G 비밀번호 분실 시 초기화
- 2009/04/15 Alteon Password Recovery 방법
- 2009/03/18 STP Decision Sequence
- 2008/08/21 SNMP OID
- 2008/08/19 중요 Protocols
- 2008/08/18 High CPU Utilization on Cisco IOS Software-Based Catalyst 4500 Switches (2)
Table 1—Autonegotiation Valid Configuration
Configuration NIC (Speed/Duplex) | Configuration Switch (Speed/Duplex) | Resulting NIC Speed/Duplex | Resulting Catalyst Speed/Duplex | Comments |
|---|---|---|---|---|
AUTO | AUTO | 1000 Mbps, Full-duplex | 1000 Mbps, Full-duplex | Assuming maximum capability of Catalyst switch, and NIC is 1000 Mbps, full-duplex. |
1000 Mbps, Full-duplex | AUTO | 1000 Mbps, Full-duplex | 1000 Mbps, Full-duplex | Link is established, but the switch does not see any autonegotiation information from NIC. Since Catalyst switches support only full-duplex operation with 1000 Mbps, they default to full-duplex, and this happens only when operating at 1000 Mbps. |
AUTO | 1000 Mbps, Full-duplex | 1000 Mbps, Full-duplex | 1000 Mbps, Full-duplex | Assuming maximum capability of NIC is 1000 Mbps, full-duplex. |
1000 Mbps, Full-duplex | 1000 Mbps, Full-duplex | 1000 Mbps, Full-duplex | 1000 Mbps, Full-duplex | Correct Manual Configuration |
100 Mbps, Full-duplex | 1000 Mbps, Full-duplex | No Link | No Link | Neither side establishes link, due to speed mismatch |
100 Mbps, Full-duplex | AUTO | 100 Mbps, Full-duplex | 100 Mbps, Half-duplex | Duplex Mismatch 1 |
AUTO | 100 Mbps, Full-duplex | 100 Mbps, Half-duplex | 100 Mbps, Full-duplex | Duplex Mismatch 1 |
100 Mbps, Full-duplex | 100 Mbps, Full-duplex | 100 Mbps, Full-duplex | 100 Mbps, Full-duplex | Correct Manual Configuration2 |
100 Mbps, Half-duplex | AUTO | 100 Mbps, Half-duplex | 100 Mbps, Half-duplex | Link is established, but switch does not see any autonegotiation information from NIC and defaults to half-duplex when operating at 10/100 Mbps. |
10 Mbps, Half-duplex | AUTO | 10 Mbps, Half-duplex | 10 Mbps, Half-duplex | Link is established, but switch does not see Fast Link Pulse (FLP) and defaults to 10 Mbps half-duplex. |
10 Mbps, Half-duplex | 100 Mbps, Half-duplex | No Link | No Link | Neither side establishes link, due to speed mismatch. |
AUTO | 100 Mbps, Half-duplex | 100 Mbps, Half-duplex | 100 Mbps, Half-duplex | Link is established, but NIC does not see any autonegotiation information and defaults to 100 Mbps, half-duplex. |
AUTO | 10 Mbps, Half-duplex | 10 Mbps, Half-duplex | 10 Mbps, Half-duplex | Link is established, but NIC does not see FLP and defaults to 10 Mbps, half-duplex. |
1 A duplex mismatch can result in performance issues, intermittent connectivity, and loss of communication. When you troubleshoot NIC issues, verify that the NIC and switch use a valid configuration.
2 Some third-party NIC cards can fall back to half-duplex operation mode, even though both the switchport and NIC configuration are manually configured for 100 Mbps, full-duplex. This is because NIC autonegotiation link detection still operates when the NIC is manually configured. This causes duplex inconsistency between the switchport and the NIC. Symptoms include poor port performance and frame check sequence (FCS) errors that increment on the switchport. In order to troubleshoot this issue, try to manually configure the switchport to 100 Mbps, half-duplex. If this action resolves the connectivity problems, this NIC issue is the possible cause. Try to update to the latest drivers for your NIC, or contact your NIC card vendor for additional support.
http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note09186a00800a7af0.shtml
ㅇ QoS 측정 요소 9가지 => 결국은 대역폭과 지연, 지터, 손실의 4가지, 지연 = 연속+전달+큐잉+포워딩+세이핑+네트웍 지연
- 가용 대역폭(available bandwidth) : bps(bit per second) 단위로 표시, 보통은 물리적인 링크 속도(link speed)와 동일, 전송되는 구간의 대역폭이 각각 다른 경우에는 가장 낮은 구간의 대역폭에 의해 결정
=> 대역 사용 효율화 방법 : 헤더 압축(header compress), 페이로드 압축(payload compress) 등을 사용해 더 많은 패킷을 전송하는 방법과 큐잉을 사용해 중요한 패킷을 먼저 전송해 대역폭이 증가된 것 같은 효과를 낼 수 있는 방법이 있음 => 두 방법 모두 CPU의 부하와 지연을 증가시키는 등 또 다른 문제를 유발
=> LFI(Link Fragmentation and Interleaving) : 큰 패킷의 연속지연에 의해 작은 패킷들이 지연되는 문제를 해결, 작은 패킷의 앞에 있는 큰 패킷을 여러 조각으로 분해(Fragment)한 후 작아진 조각들 사이에 작은 패킷을 끼워넣어 먼저 서비스되게 만드는 기술
- 연속 지연(serialization delay) : 패킷을 링크상에서 시리얼화 하는데 걸리는 시간, 125바이트(=1000비트)로 구성된 패킷을 고속 이더넷(100Mbps) 링크에서 시리얼화 하는 데는 1000비트 / 1,000,000,000bps = 0.1ms가 소요 => 패킷 길이에 비례, 링크 대역폭에 반비례
- 전달 지연(Propagation Delay) : 링크를 통해 한 비트를 전달하는데 걸리는 시간, 동일한 물리적인 링크상에서 대역폭과 패킷의 길이와는 상관없이 동일한 지연 시간을 갖는다, 일반적으로 구리(copper)나 광(optical) 케이블 상에서의 전송 속도는 진공 상태의 약 70% 속도인 2.1×108m/s 이며 망 거리가 1000km이라고 하면 전달 지연 = 1,000,000/(2.1×108) = 4.8ms
- 큐잉 지연(queuing delay) : 패킷이 전송되기 전에 라우터의 큐(하드웨어와 소프트웨어 큐 모두 포함)에 패킷이 대기하는 시간. 56Kbps의 대역폭에서 1500바이트 패킷을 4개 보낸다고 가정할 때, 첫번째 패킷이 라우터의 큐를 떠나는데 걸리는 시간(연속지연)은 1500×8 / 56,000=214ms이다. 즉, 두번째 패킷은 라우터의 큐에서 214ms를 대기하고 있다가, 전송되기 시작한다는 의미다. 세번째 패킷은 428ms를, 마지막 네번째 패킷은 642ms를 큐에서 대기하고 있는 것이다. 대부분의 TCP 재전송은 혼잡(Congestion)이나 큐잉 지연 때문에 발생한다. 적정한 길이의 큐는 패킷 손실을 예방하지만, 너무 긴 큐는 장시간 지연의 원인이 된다.
- 포워딩 지연 : 패킷이 라우터나 스위치로 들어오는 인터페이스로 들어와서, 라우팅과 스위칭 프로세서에 의해 경로가 결정된 후, 나가는 인터페이스의 나가는 큐에 놓이기까지 걸리는 시간 => 스위치는 축적전송(store-and-forward) 보다는 컷쓰루(cut-through) 방식이 보다 적은 포워딩 지연을 발생시키며, 라우터의 경우에는 포워딩 지연을 줄이기 위해 CEF(Cisco Express Forwarding) 방법을 사용하기도 한다
- 세이핑 지연 : 트래픽 세이핑 프로세스에 의해 지연되는 시간
- 네트워크 지연 : ISP의 망 내에서의 지연 = Ingress의 연속지연 + Egress의 연속지연 + 망내 전달지연
- 지터(Jitter) : 어떤 신호가 네트워크를 통해 전달되면서 원래의 신호로부터 왜곡되는 정도를 나타내는데 사용되는 값으로 패킷에 대한 지연이 동일하게 발생하지 않고 편차가 발생되어 결과적으로 신호가 왜곡되는 것
- 패킷 손실(packet loss) : 보통 혼잡(Congestion)에 의한 버퍼 오버플로우시 발생하거나, 흐름 제어 알고리즘(RED : Random Early Detection)에 의해 임의적으로 패킷을 버리는 현상에 의해 발생 (하드웨어 불량에 의한 발생은 논외)
ㅇ Integrated Service Model (IntServ) :
- 각 패킷의 흐름을 QoS 보장(guaranteed)/비보장(controlled)으로 구분하고 보장형 서비스는 RSVP 프로토콜을 이용하여 사전에 라우터에서 연결수락과 자원예약을 수행 => 대규모 망에서는 적용 불가, 비효율적
ㅇ Differentiated Service Model (DiffServ) : IntServ 모델이 확장성이 취약한 것을 극복하기 위해 ISP들에 의해 제안된 모델
- 입력 - 분류자(ACL(Access-Control List) 이나 PBR(Policy-Based Routing)을 사용하여 트래픽을 클래스로 구분) - 미터(트래픽의 특성-보통 입력속도 측정) - 마커(사전 약속된 QoS 특성과 비교하여 만족(Green), 일정한 범위 내에서 초과(Yellow), 일정한 범위를 넘어서 초과(Red) 마킹) - 컨디셔너(초과된 패킷 대역폭 조절, Shaping : 버퍼에서의 지연을 이용해 대역폭을 조절, policing : 드로퍼(deopper)를 이용해 대역폭을 조절) - 큐잉(분류자에서 결정된 클래스에 맞는 큐로 저장) - 스케쥴링 - 출력
- DSCP(DiffServ Code Point) : IP헤더의 ToS(Type of Service) 필드(8bits) 의 상위 6비트를 사용하여 클래스를 정의하며 다음 4가지의 PHB(Per-Hop Behavior)가 있음
. 디폴트 PHB(Best-effort) : 가장 낮은 우선순위
. CS(Class Selector) PHB : 상위 3비트(기존의 IP Precedence 필드)만 이용
. AF(Assured Forwarding) PHB : 상위 3비트를 클래스 우선순위 4,5비트를 drop precedence로 사용
. EF(Expedited Forwarding) PHB : 우선순위가 가장 높은 클래스
![]() |
Catalyst 4500 Series Switch Software Configuration Guide
Table Of ContentsConfiguring the Switch for the First Time Configuring DHCP-Based Autoconfiguration Understanding DHCP-Based Autoconfiguration Using Configuration Mode to Configure Your Switch Verifying the Running Configuration Settings Saving the Running Configuration Settings to Your Start-Up File Reviewing the Configuration in NVRAM Controlling Access to Privileged EXEC Commands Setting or Changing a Static enable Password Using the enable password and enable secret Commands Setting or Changing a Privileged Password Controlling Switch Access with TACACS+ Displaying the TACACS+ Configuration Configuring Multiple Privilege Levels Setting the Privilege Level for a Command Changing the Default Privilege Level for Lines Logging In to a Privilege Level Displaying the Password, Access Level, and Privilege Level Configuration Recovering a Lost Enable Password Modifying the Supervisor Engine Startup Configuration Understanding the Supervisor Engine Boot Configuration Configuring the Software Configuration Register Modifying the Boot Field and Using the boot Command Verifying the Configuration Register Setting Specifying the Startup System Image Controlling Environment Variables Resetting a Switch to Factory Default Settings Configuring the Switch for the First TimeThis chapter describes how to initially configure a Catalyst 4500 series switch. The information presented here supplements the administration information and procedures in this publication: Cisco IOS Configuration Fundamentals Command Reference, Release 12.2SR, at this URL: This chapter includes the following major sections: • • • • • ![]() Note Default Switch ConfigurationThis section describes the default configurations for the Catalyst 4500 series switch. Table 3-1 shows the default configuration settings for each feature. Configuring DHCP-Based AutoconfigurationThese sections describe how to configure DHCP-based autoconfiguration. • • If your DHCP server is a Cisco device, or if you are configuring the switch as a DHCP server, refer to the "IP Addressing and Services" section in the Cisco IOS IP and IP Routing Configuration Guide for Cisco IOS Release 12.1 for additional information about configuring DHCP. Understanding DHCP-Based Autoconfiguration![]() Note DHCP provides configuration information to Internet hosts and internetworking devices. This protocol consists of two components: one component for delivering configuration parameters from a DHCP server to a device and another component that is a mechanism for allocating network addresses to devices. DHCP is built on a client-server model, in which designated DHCP servers allocate network addresses and deliver configuration parameters to dynamically configured devices. The switch can act as both a DHCP client and a DHCP server. With DHCP-based autoconfiguration, no DHCP client-side configuration is needed on your switch because your switch (the DHCP client) is automatically configured at startup with IP address information and a configuration file. However, you need to configure the DHCP server or the DHCP server feature on your switch for various lease options associated with IP addresses. If you are using DHCP to relay the configuration file location on the network, you might also need to configure a Trivial File Transfer Protocol (TFTP) server and a Domain Name System (DNS) server. DHCP-based autoconfiguration replaces the BOOTP client functionality on your switch. DHCP Client Request ProcessAt startup the switch automatically requests configuration information from a DHCP server if a configuration file is not present on the switch. Figure 3-1 shows the sequence of messages that are exchanged between the DHCP client and the DHCP server. Figure 3-1 DHCP Client and Server Message Exchange ![]() The client, Switch A, broadcasts a DHCPDISCOVER message to locate a DHCP server. The DHCP server offers configuration parameters (such as an IP address, subnet mask, gateway IP address, DNS IP address, lease for the IP address, and so forth) to the client in a DHCPOFFER unicast message. In a DHCPREQUEST broadcast message, the client returns a formal request for the offered configuration information to the DHCP server. The formal request is broadcast so that all other DHCP servers that received the DHCPDISCOVER broadcast message from the client can reclaim the IP addresses that they offered to the client. The DHCP server confirms that the IP address has been allocated to the client by returning a DHCPACK unicast message to the client. With this message, the client and server are bound, and the client uses the configuration information that it received from the server. The amount of information the switch receives depends on how you configure the DHCP server. For more information, see the "Configuring the DHCP Server" section. If the configuration parameters sent to the client in the DHCPOFFER unicast message are invalid (if configuration error exists), the client returns a DHCPDECLINE broadcast message to the DHCP server. The DHCP server sends the client a DHCPNAK denial broadcast message, which means that the offered configuration parameters have not been assigned, that an error has occurred during the negotiation of the parameters, or that the client has been slow in responding to the DHCPOFFER message. (The DHCP server might have assigned the parameters to another client.) A DHCP client might receive offers from multiple DHCP servers and can accept any of them; however, the client usually accepts the first offer it receives. The offer from the DHCP server is not a guarantee that the IP address will be allocated to the client; however, the server usually reserves the address until the client has had a chance to formally request the address. Configuring the DHCP ServerA switch can act as both the DHCP client and the DHCP server. By default, the Cisco IOS DHCP server and relay agent features are enabled on your switch. You should configure the DHCP server, or the DHCP server feature running on your switch, with reserved leases that are bound to each switch by the switch hardware address. If you want the switch to receive IP address information, you must configure the DHCP server with these lease options: • • • • ![]() Note If you want the switch to receive the configuration file from a TFTP server, you must configure the DHCP server with these lease options: • • • Depending on the settings of the DHCP server or the DHCP server feature running on your switch, the switch can receive IP address information, the configuration file, or both. If you do not configure the DHCP server, or the DHCP server feature running on your switch, with the lease options described earlier, the switch replies to client requests with only those parameters that are configured. If the IP address and subnet mask are not in the reply, the switch is not configured. If the router IP address or TFTP server name (or IP address) are not found, the switch might send broadcast, instead of unicast, TFTP requests. Unavailability of other lease options does not impact autoconfiguration. The DHCP server, or the DHCP server feature running on your switch, can be on the same LAN or on a different LAN than the switch. If the DHCP server is running on a different LAN, you should configure a DHCP relay, which forwards broadcast traffic between two directly connected LANs. A router does not forward broadcast packets, but it forwards packets based on the destination IP address in the received packet. For more information on relay devices, see the "Configuring the Relay Device" section. Configuring the TFTP ServerBased on the DHCP server configuration, the switch attempts to download one or more configuration files from the TFTP server. If you configured the DHCP server to respond to the switch with all the options required for IP connectivity to the TFTP server, and if you configured the DHCP server with a TFTP server name, address, and configuration filename, the switch attempts to download the specified configuration file from the specified TFTP server. If you did not specify the configuration filename or the TFTP server name, or if the configuration file could not be downloaded, the switch attempts to download a configuration file using various combinations of filenames and TFTP server addresses. The files include the specified configuration filename (if any) and the following files: network-confg, cisconet.cfg, hostname.confg, or hostname.cfg, where hostname is the current hostname of the switch and router-confg and ciscortr.cfg. The TFTP server addresses used include the specified TFTP server address (if any) and the broadcast address (255.255.255.255). For the switch to successfully download a configuration file, the TFTP server must contain one or more configuration files in its base directory. The files can include the following: • • • If you specify the TFTP server name in the DHCP server-lease database, you must also configure the TFTP server name-to-IP-address mapping in the DNS-server database. If the TFTP server you plan to use is on a different LAN from the switch, or if you plan to access it with the switch through the broadcast address (which occurs if the DHCP server response does not contain all the required information described earlier), you must configure a relay to forward the TFTP packets to the TFTP server. For more information, see the "Configuring the Relay Device" section. The preferred solution is to configure either the DHCP server or the DHCP server feature running on your switch with all the required information. Configuring the DNS ServerThe DHCP server, or the DHCP server feature running on your switch, uses the DNS server to resolve the TFTP server name to an IP address. You must configure the TFTP server name-to-IP address map on the DNS server. The TFTP server contains the configuration files for the switch. You can configure the IP addresses of the DNS servers in the lease database of the DHCP server where the DHCP replies will retrieve them. You can enter up to two DNS server IP addresses in the lease database. The DNS server can be on the same or on a different LAN as the switch. If it is on a different LAN, the switch must be able to access it through a router. Configuring the Relay DeviceYou must configure a relay device to forward received broadcast packets to the destination host whenever a switch sends broadcast packets to which a host on a different LAN must respond. Examples of such broadcast packets are DHCP, DNS, and in some cases, TFTP packets. If the relay device is a Cisco router, enable IP routing (ip routing global configuration command), and configure helper addresses (ip helper-address interface configuration command). For example, in Figure 3-2, configure the router interfaces as follows: On interface 10.0.0.2: router(config-if)# ip helper-address 20.0.0.2 router(config-if)# ip helper-address 20.0.0.3 router(config-if)# ip helper-address 20.0.0.4 On interface 20.0.0.1 router(config-if)# ip helper-address 10.0.0.1 Figure 3-2 Relay Device Used in Autoconfiguration ![]() Obtaining Configuration FilesDepending on the availability of the IP address and the configuration filename in the DHCP reserved lease, the switch obtains its configuration information in these ways: • The switch receives its IP address, subnet mask, TFTP server address, and the configuration filename from either the DHCP server or the DHCP server feature running on your switch. The switch sends a unicast message to the TFTP server to retrieve the named configuration file from the base directory of the server, and upon receipt, completes its boot-up process. • The switch receives its IP address, subnet mask, and the configuration filename from either the DHCP server or the DHCP server feature running on your switch. The switch sends a broadcast message to a TFTP server to retrieve the named configuration file from the base directory of the server, and upon receipt, completes its boot-up process. • The switch receives its IP address, subnet mask, and the TFTP server address from either the DHCP server or the DHCP server feature running on your switch. The switch sends a unicast message to the TFTP server to retrieve the network-confg or cisconet.cfg default configuration file. (If the network-confg file cannot be read, the switch reads the cisconet.cfg file.) The default configuration file contains the host names-to-IP-address mapping for the switch. The switch fills its host table with the information in the file and obtains its host name. If the host name is not found in the file, the switch uses the host name in the DHCP reply. If the host name is not specified in the DHCP reply, the switch uses the default Switch as its host name. After obtaining its host name from the default configuration file or the DHCP reply, the switch reads the configuration file that has the same name as its host name (hostname-confg or hostname.cfg, depending on whether or not the network-confg file or the cisconet.cfg file was read earlier) from the TFTP server. If the cisconet.cfg file is read, the filename of the host is truncated to eight characters. If the switch cannot read the network-confg, cisconet.cfg, or the hostname file, it reads the router-confg file. If the switch cannot read the router-confg file, it reads the ciscortr.cfg file. ![]() Note Example ConfigurationFigure 3-3 shows a network example for retrieving IP information using DHCP-based autoconfiguration. Figure 3-3 DHCP-Based Autoconfiguration Network Example ![]() Table 3-2 shows the configuration of the reserved leases on either the DHCP server or the DHCP server feature running on your switch. DNS Server Configuration The DNS server maps the TFTP server name maritsu to IP address 10.0.0.3. TFTP Server Configuration (on UNIX) The TFTP server base directory is set to /tftpserver/work/. This directory contains the network-confg file used in the two-file read method. This file contains the host name that you plan to assign to the switch based on its IP address. The base directory also contains a configuration file for each switch (switch1-confg, switch2-confg, and so forth) as shown in the following display: prompt> cd /tftpserver/work/ prompt> ls network-confg switch1-confg switch2-confg switch3-confg switch4-confg prompt> cat network-confg ip host switch1 10.0.0.21 ip host switch2 10.0.0.22 ip host switch3 10.0.0.23 ip host switch4 10.0.0.24 DHCP Client Configuration No configuration file is present on Switch 1 through Switch 4. Configuration Explanation In Figure 3-3, Switch 1 reads its configuration file as follows: • • • • • Switches 2 through 4 retrieve their configuration files and IP addresses in the same way. Configuring the SwitchThe following sections describe how to configure your switch: • • • • • Using Configuration Mode to Configure Your SwitchTo configure your switch from configuration mode, perform this procedure: Step 1 Step 2 Switch> enable ![]() Note The prompt will change to the enable prompt (#): Switch# Step 3 Switch# configure terminal Enter configuration commands, one per line. End with CNTL/Z. Switch(config)# Step 4 Switch(config)# interface fastethernet 5/1
Switch(config-if)# Step 5 Step 6 Step 7 Your switch is now minimally configured and can boot with the configuration you entered. To see a list of the configuration commands, enter ? at the prompt or press the help key in configuration mode. Verifying the Running Configuration SettingsTo verify the configuration settings you entered or the changes you made, enter the show running-config command at the enable prompt (#), as shown in this example: Switch# show running-config
Building configuration... Current configuration: ! version 12.0 service timestamps debug uptime service timestamps log uptime no service password-encryption ! hostname Switch <...output truncated...> ! line con 0 transport input none line vty 0 4 exec-timeout 0 0 password lab login transport input lat pad dsipcon mop telnet rlogin udptn nasi ! end Switch# Saving the Running Configuration Settings to Your Start-Up File![]() Caution
To store the configuration, changes to the configuration, or changes to the startup configuration in NVRAM, enter the copy running-config startup-config command at the enable prompt (#), as follows: Switch# copy running-config startup-config Reviewing the Configuration in NVRAMTo display information stored in NVRAM, enter the show startup-config EXEC command. The following example shows a typical system configuration: Switch# show startup-config Using 1579 out of 491500 bytes, uncompressed size = 7372 bytes Uncompressed configuration from 1579 bytes to 7372 bytes ! version 12.1 no service pad service timestamps debug uptime service timestamps log uptime no service password-encryption service compress-config ! hostname Switch ! ! ip subnet-zero ! ! ! ! interface GigabitEthernet1/1 no snmp trap link-status ! interface GigabitEthernet1/2 no snmp trap link-status !--More--
<...output truncated...> ! line con 0 exec-timeout 0 0 transport input none line vty 0 4 exec-timeout 0 0 password lab login transport input lat pad dsipcon mop telnet rlogin udptn nasi ! end Switch# Configuring a Default Gateway![]() Note Configure a default gateway to send data to subnets other than its own when the switch is not configured with a routing protocol. The default gateway must be the IP address of an interface on a router that is directly connected to the switch. To configure a default gateway, perform this task: This example shows how to configure a default gateway and how to verify the configuration: Switch# configure terminal
Enter configuration commands, one per line. End with CNTL/Z. Switch(config)# ip default-gateway 172.20.52.35
Switch(config)# end
3d17h: %SYS-5-CONFIG_I: Configured from console by console Switch# show ip route
Default gateway is 172.20.52.35 Host Gateway Last Use Total Uses Interface ICMP redirect cache is empty Switch# Configuring a Static RouteIf your Telnet station or SNMP network management workstation is on a different network from your switch and a routing protocol has not been configured, you might need to add a static routing table entry for the network where your end station is located. To configure a static route, perform this task: This example shows how to use the ip route command to configure a static route to a workstation at IP address 171.10.5.10 on the switch with a subnet mask and IP address 172.20.3.35 of the forwarding router: Switch# configure terminal
Enter configuration commands, one per line. End with CNTL/Z. Switch(config)# ip route 171.10.5.10 255.255.255.255 172.20.3.35
Switch(config)# end
Switch# This example shows how to use the show running-config command to confirm the configuration of the static route: Switch# show running-config Building configuration... . <...output truncated...> . ip default-gateway 172.20.52.35 ip classless ip route 171.10.5.10 255.255.255.255 172.20.3.35 no ip http server ! line con 0 transport input none line vty 0 4 exec-timeout 0 0 password lab login transport input lat pad dsipcon mop telnet rlogin udptn nasi ! end Switch# This example shows how to use the ip route command to configure the static route IP address 171.20.5.3 with subnet mask and connected over VLAN 1 to a workstation on the switch: Switch# configure terminal
Switch(config)# ip route 171.20.5.3 255.255.255.255 vlan 1
Switch(config)# end
Switch# This example shows how to use the show running-config command to confirm the configuration of the static route: Switch# show running-config
Building configuration... . <...output truncated...> . ip default-gateway 172.20.52.35 ip classless ip route 171.20.5.3 255.255.255.255 Vlan1 no ip http server ! ! x25 host z ! line con 0 transport input none line vty 0 4 exec-timeout 0 0 password lab login transport input lat pad dsipcon mop telnet rlogin udptn nasi ! end Switch# Controlling Access to Privileged EXEC CommandsThe procedures in these sections let you control access to the system configuration file and privileged EXEC commands: • • • • Setting or Changing a Static enable PasswordTo set or change a static password that controls access to the enable mode, perform this task:
This example shows how to configure an enable password as "lab" at the privileged EXEC mode: Switch# configure terminal
Switch(config)# enable password lab
Switch(config)# For instructions on how to display the password or access level configuration, see the "Displaying the Password, Access Level, and Privilege Level Configuration" section. Using the enable password and enable secret CommandsTo provide an additional layer of security, particularly for passwords that cross the network or that are stored on a TFTP server, you can use either the enable password or enable secret command. Both commands configure an encrypted password that you must enter to access the enable mode (the default) or any other privilege level that you specify. We recommend that you use the enable secret command. If you configure the enable secret command, it takes precedence over the enable password command; the two commands cannot be in effect simultaneously. To configure the switch to require an enable password, perform either one of these tasks: When you enter either of these password commands with the level option, you define a password for a specific privilege level. After you specify the level and set a password, give the password only to users who need to have access at this level. Use the privilege level configuration command to specify commands accessible at various levels. If you enable the service password-encryption command, the password you enter is encrypted. When you display the password with the more system:running-config command, the password displays the password in encrypted form. If you specify an encryption type, you must provide an encrypted password—an encrypted password you copy from another Catalyst 4500 series switch configuration. ![]() Note For information on how to display the password or access level configuration, see the "Displaying the Password, Access Level, and Privilege Level Configuration" section. Setting or Changing a Privileged PasswordTo set or change a privileged password, perform this task:
For information on how to display the password or access level configuration, see the "Displaying the Password, Access Level, and Privilege Level Configuration" section. Controlling Switch Access with TACACS+This section describes how to enable and configure TACACS+, which provides detailed accounting information and flexible administrative control over authentication and authorization processes. TACACS+ is facilitated through authentication, authorization, accounting (AAA) and can be enabled only through AAA commands. ![]() Note This section contains this configuration information: • Understanding TACACS+TACACS+ is a security application that provides centralized validation of users attempting to gain access to your switch. TACACS+ services are maintained in a database on a TACACS+ daemon typically running on a UNIX or Windows NT workstation. You should have access to and should configure a TACACS+ server before configuring TACACS+ features on your switch. TACACS+ provides for separate and modular AAA facilities. TACACS+ allows for a single access control server (the TACACS+ daemon) to provide each service—authentication, authorization, and accounting—independently. Each service can be locked into its own database to take advantage of other services available on that server or on the network, depending on the capabilities of the daemon. The goal of TACACS+ is to provide a method for managing multiple network access points from a single management service. Your switch can be a network access server along with other Cisco routers and access servers. A network access server provides connections to a single user, to a network or subnetwork, and to interconnected networks as shown in Figure 3-4. Figure 3-4 Typical TACACS+ Network Configuration ![]() TACACS+ administered through the AAA security services can provide these services: • The authentication facility can conduct a dialog with the user (such as, after a username and password are provided, to challenge a user with several questions such as home address, mother's maiden name, service type, and social security number). The TACACS+ authentication service can also send messages to user screens. For example, a message could notify users that their passwords must be changed because of the company's password aging policy. • • The TACACS+ protocol provides authentication between the switch and the TACACS+ daemon, and it ensures confidentiality because all protocol exchanges between the switch and the TACACS+ daemon are encrypted. You need a system running the TACACS+ daemon software to use TACACS+ on your switch. TACACS+ OperationWhen a user attempts a simple ASCII login by authenticating to a switch using TACACS+, this process occurs: 1. TACACS+ allows a conversation between the daemon and the user until the daemon receives enough information to authenticate the user. The daemon prompts for a username and password combination, but can include other items such as the user's mother's maiden name. 2. • • • • After authentication, the user undergoes an additional authorization phase if authorization has been enabled on the switch. Users must first successfully complete TACACS+ authentication before proceeding to TACACS+ authorization. 3. • • Configuring TACACS+This section describes how to configure your switch to support TACACS+. At a minimum, you must identify the host or hosts maintaining the TACACS+ daemon and define the method lists for TACACS+ authentication. You can optionally define method lists for TACACS+ authorization and accounting. A method list defines the sequence and methods used to authenticate, to authorize, or to keep accounts on a user. You can use method lists to designate one or more security protocols, ensuring a backup system if the initial method fails. The software uses the first method listed to authenticate, to authorize, or to keep accounts on users; if that method does not respond, the software selects the next method in the list. This process continues until there is successful communication with a listed method or the method list is exhausted. This section contains this configuration information: • • • • Default TACACS+ ConfigurationTACACS+ and AAA are disabled by default. To prevent a lapse in security, you cannot configure TACACS+ through a network management application. When enabled, TACACS+ can authenticate users accessing the switch through the CLI. ![]() Note Identifying the TACACS+ Server Host and Setting the Authentication KeyYou can configure the switch to use a single server or AAA server groups in order to group existing server hosts for authentication. You can group servers to select a subset of the configured server hosts and use them for a particular service. The server group is used with a global server-host list and contains the list of IP addresses of the selected server hosts. Beginning in privileged EXEC mode, follow these steps to identify the IP host or host maintaining TACACS+ server and optionally set the encryption key: To remove the specified TACACS+ server name or address, use the no tacacs-server host hostname global configuration command. To remove a server group from the configuration list, use the no aaa group server tacacs+ group-name global configuration command. To remove the IP address of a TACACS+ server, use the no server ip-address server group subconfiguration command. Configuring TACACS+ Login AuthenticationTo configure AAA authentication, define a named list of authentication methods and then apply that list to various ports. The method list defines the types of authentication you intend to perform and the sequence in which you intend to perform them; you must apply the list to a specific port before you can perform any of the defined authentication methods. The only exception is the default method list (which, by coincidence, is named default). The default method list is automatically applied to all ports except those that have a named method list explicitly defined. A defined method list overrides the default method list. A method list describes the sequence and authentication methods that must be queried to authenticate a user. You can designate one or more security protocols for authentication, ensuring a backup system for authentication in case the initial method fails. The software uses the first method listed to authenticate users; if that method fails to respond, the software selects the next authentication method in the method list. This process continues until there is successful communication with a listed authentication method or until all defined methods are exhausted. If authentication fails at any point in this cycle—meaning that the security server or local username database responds by denying the user access—the authentication process stops, and no other authentication methods are attempted. Beginning in privileged EXEC mode, follow these steps to configure login authentication:
To disable AAA, use the no aaa new-model global configuration command. To disable AAA authentication, use the no aaa authentication login {default | list-name} method1 [method2...] global configuration command. To either disable TACACS+ authentication for logins or to return to the default value, use the no login authentication {default | list-name} line configuration command. Configuring TACACS+ Authorization for Privileged EXEC Access and Network ServicesAAA authorization limits the services available to a user. When AAA authorization is enabled, the switch uses information retrieved from the user's profile, which is located either in the local user database or on the security server, to configure the user's session. The user is granted access to a requested service only if the information in the user profile allows it. You can use the aaa authorization global configuration command with the tacacs+ keyword to set parameters that restrict a user's network access to privileged EXEC mode. The aaa authorization exec tacacs+ local command sets these authorization parameters: • • ![]() Note Beginning in privileged EXEC mode, follow these steps to specify TACACS+ authorization for privileged EXEC access and network services: To disable authorization, use the no aaa authorization {network | exec} method1 global configuration command. Starting TACACS+ AccountingThe AAA accounting feature tracks the services that users are accessing and the amount of network resources that they are consuming. When AAA accounting is enabled, the switch reports user activity to the TACACS+ security server in the form of accounting records. Each accounting record contains accounting attribute-value (AV) pairs and is stored on the security server. This data can then be analyzed for network management, client billing, or auditing. Beginning in privileged EXEC mode, follow these steps to enable TACACS+ accounting for each Cisco IOS privilege level and for network services: To disable accounting, use the no aaa accounting {network | exec} {start-stop} method1... global configuration command. Displaying the TACACS+ ConfigurationTo display TACACS+ server statistics, use the show tacacs privileged EXEC command. Encrypting PasswordsBecause protocol analyzers can examine packets (and read passwords), you can increase access security by configuring the Cisco IOS software to encrypt passwords. Encryption prevents the password from being readable in the configuration file. To configure the Cisco IOS software to encrypt passwords, perform this task: Encryption occurs when the current configuration is written or when a password is configured. Password encryption is applied to all passwords, including authentication key passwords, the privileged command password, console and virtual terminal line access passwords, and Border Gateway Protocol (BGP) neighbor passwords. The service password-encryption command keeps unauthorized individuals from viewing your password in your configuration file. ![]() Caution
Although you cannot recover a lost encrypted password (that is, you cannot get the original password back), you can regain control of the switch after having lost or forgotten the encrypted password. See the "Recovering a Lost Enable Password" section for more information. For information on how to display the password or access level configuration, see the "Displaying the Password, Access Level, and Privilege Level Configuration" section. Configuring Multiple Privilege LevelsBy default, Cisco IOS software has two modes of password security: user EXEC mode and privileged EXEC mode. You can configure up to 16 hierarchical levels of commands for each mode. By configuring multiple passwords, you can allow different sets of users to have access to specified commands. For example, if you want many users to have access to the clear line command, you can assign it level 2 security and distribute the level 2 password fairly widely. If you want more restricted access to the configure command, you can assign it level 3 security and distribute that password to fewer users. The procedures in the following sections describe how to configure additional levels of security: • • • • Setting the Privilege Level for a CommandTo set the privilege level for a command, perform this task: For information on how to display the password or access level configuration, see the "Displaying the Password, Access Level, and Privilege Level Configuration" section. Changing the Default Privilege Level for LinesTo change the default privilege level for a given line or a group of lines, perform this task:
For information on how to display the password or access level configuration, see the "Displaying the Password, Access Level, and Privilege Level Configuration" section. Logging In to a Privilege LevelTo log in at a specified privilege level, perform this task: Exiting a Privilege LevelTo exit to a specified privilege level, perform this task: Displaying the Password, Access Level, and Privilege Level ConfigurationTo display detailed password information, perform this task:
This example shows how to display the password and access level configuration: Switch# show running-config
Building configuration... Current configuration: ! version 12.0 service timestamps debug datetime localtime service timestamps log datetime localtime no service password-encryption ! hostname Switch ! boot system flash sup-bootflash enable password lab ! <...output truncated...> This example shows how to display the privilege level configuration: Switch# show privilege
Current privilege level is 15 Switch# Recovering a Lost Enable Password![]() Note Perform these steps to recover a lost enable password: Step 1 Step 2 Step 3 Step 4 Step 5 Step 6 Step 7 Step 8 Modifying the Supervisor Engine Startup ConfigurationThese sections describe how the startup configuration on the supervisor engine works and how to modify the BOOT variable and the configuration register: • • • • Understanding the Supervisor Engine Boot ConfigurationThe supervisor engine boot process involves two software images: ROM monitor and supervisor engine software. When the switch is booted or reset, the ROMMON code is executed. Depending on the NVRAM configuration, the supervisor engine either stays in ROMMON mode or loads the supervisor engine software. Two user-configurable parameters determine how the switch boots: the configuration register and the BOOT environment variable. The configuration register is described in the "Modifying the Boot Field and Using the boot Command" section. The BOOT environment variable is described in the "Specifying the Startup System Image" section. Understanding the ROM MonitorThe ROM monitor (ROMMON) is invoked at switch bootup, reset, or when a fatal exception occurs. The switch enters ROMMON mode if the switch does not find a valid software image, if the NVRAM configuration is corrupted, or if the configuration register is set to enter ROMMON mode. From ROMMON mode, you can manually load a software image from bootflash or a Flash disk, or you can boot up from the management interface. ROMMON mode loads a primary image from which you can configure a secondary image to boot up from a specified source either locally or through the network using the BOOTLDR environment variable. This variable is described in the "Switch#" section. You can also enter ROMMON mode by restarting the switch and then pressing Ctrl-C during the first five seconds of startup. If you are connected through a terminal server, you can escape to the Telnet prompt and enter the send break command to enter ROMMON mode. ![]() Note The ROM monitor has these features: • • • • Configuring the Software Configuration RegisterThe switch uses a 16-bit software configuration register, which allows you to set specific system parameters. Settings for the software configuration register are preconfigured in NVRAM. Here are some reasons why you might want to change the software configuration register settings: • • • • • • • ![]() Caution
Table 3-3 lists the meaning of each of the software configuration memory bits. Table 3-4 defines the boot field.
Modifying the Boot Field and Using the boot CommandThe configuration register boot field determines whether the switch loads an operating system image and, if so, where it obtains this system image. The following sections describe how to use and set the configuration register boot field and the procedures you must perform to modify the configuration register boot field. In ROMMON, you can use the confreg command to modify the configuration register and change boot settings. Bits 0 through 3 of the software configuration register contain the boot field. ![]() Note When the boot field is set to either 00 or 01 (0-0-0-0 or 0-0-0-1), the system ignores any boot instructions in the system configuration file and the following occurs: • • • ![]() Caution
You can enter the boot command only or enter the command and include additional boot instructions, such as the name of a file stored in Flash memory, or a file that you specify for booting from a network server. If you use the boot command without specifying a file or any other boot instructions, the system boots from the default Flash image (the first image in onboard Flash memory). Otherwise, you can instruct the system to boot up from a specific Flash image (using the boot system flash filename command). You can also use the boot command to boot up images stored in the compact Flash cards located in slot 0 on the supervisor engine. Modifying the Boot FieldModify the boot field from the software configuration register. To modify the software configuration register boot field, perform this task: To modify the configuration register while the switch is running Cisco IOS software, follow these steps: Step 1 Switch> enable Password: Switch# Step 2 Switch# configure terminal Enter configuration commands, one per line. End with CNTL/Z. Switch(config)# Step 3 Switch(config)# config-register 0x102 Set the contents of the configuration register by specifying the value command variable, where value is a hexadecimal number preceded by 0x (see Table 3-3). Step 4 Step 5 Configuration register is 0x141 (will be 0x102 at next reload) Step 6 Step 7 Verifying the Configuration Register SettingEnter the show version EXEC command to verify the current configuration register setting. In ROMMON mode, enter the show version command to verify the configuration register setting. To verify the configuration register setting for the switch, perform this task: In this example, the show version command indicates that the current configuration register is set so that the switch does not automatically load an operating system image. Instead, it enters ROMMON mode and waits for you to enter ROM monitor commands. Switch#show version Cisco Internetwork Operating System Software IOS (tm) Catalyst 4000 L3 Switch Software (cat4000-IS-M), Experimental Version 12.1(20010828:211314) [cisco 105] Copyright (c) 1986-2001 by cisco Systems, Inc. Compiled Thu 06-Sep-01 15:40 by Image text-base:0x00000000, data-base:0x00ADF444 ROM:1.15 Switch uptime is 10 minutes System returned to ROM by reload Running default software cisco Catalyst 4000 (MPC8240) processor (revision 3) with 262144K bytes of memory. Processor board ID Ask SN 12345 Last reset from Reload Bridging software. 49 FastEthernet/IEEE 802.3 interface(s) 20 Gigabit Ethernet/IEEE 802.3 interface(s) 271K bytes of non-volatile configuration memory. Configuration register is 0xEC60 Switch# Specifying the Startup System ImageYou can enter multiple boot commands in the startup configuration file or in the BOOT environment variable to provide backup methods for loading a system image. The BOOT environment variable is also described in the "Specify the Startup System Image in the Configuration File" section in the "Loading and Maintaining System Images and Microcode" chapter of the Cisco IOS Configuration Fundamentals Configuration Guide. Use the following sections to configure your switch to boot from Flash memory. Flash memory can be either single in-line memory modules (SIMMs) or Flash disks. Check the appropriate hardware installation and maintenance guide for information about types of Flash memory. Flash Memory FeaturesFlash memory allows you to do the following: • • • • • For more information on Flash Memory, see this URL: http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/hardware/configuration/notes/OL_2788.html Security PrecautionsNote the following security precaution when loading from Flash memory: ![]() Caution
Configuring Flash MemoryTo configure your switch to boot from Flash memory, perform the following procedure. (Refer to the appropriate hardware installation and maintenance publication for complete instructions on installing the hardware.) Step 1 Step 2 Step 3 Step 4 Controlling Environment VariablesAlthough the ROM monitor controls environment variables, you can create, modify, or view them with certain commands. To create or modify the BOOT and BOOTLDR variables, use the boot system and boot bootldr global configuration commands, respectively. Refer to the "Specify the Startup System Image in the Configuration File" section in the "Loading and Maintaining System Images and Microcode" chapter of the Configuration Fundamentals Configuration Guide for details on setting the BOOT environment variable. ![]() Note You can view the contents of the BOOT and BOOTLDR variables using the show bootvar command. This command displays the settings for these variables as they exist in the startup configuration and in the running configuration if a running configuration setting differs from a startup configuration setting. This example shows how to check the BOOT and BOOTLDR variables on the switch: Switch# show bootvar
BOOTLDR variable = bootflash:cat4000-is-mz,1; Configuration register is 0x0 Switch# Resetting a Switch to Factory Default SettingsManufacturing and repair centers can use the erase /all non-default command to do the following: • • For example, entering this command can generate the following output: Switch# erase /all non-default
Erase and format operation will destroy all data in non-volatile storage. Continue? [confirm] Formatting bootflash: ... Format of bootflash complete Erasing nvram: Erasing cat4000_flash: Clearing crashinfo:data Clearing the last power failure timestamp Clearing all ROMMON variables Setting default ROMMON variables: ConfigReg=0x2101 PS1=rommon ! > EnableAutoConfig=1 Setting vtp mode to transparent %WARNING! Please reboot the system for the changes to take effect Switch# 00:01:48: %SYS-7-NV_BLOCK_INIT: Initialized the geometry of nvram Switch# If the Catalyst 4500 series switch is accessible to an tftp server, you can copy an image to the bootflash memory with the tftp command: Switch# copy tftp://192.20.3.123/tftpboot/abc/cat4500-entservices-mz.bin bootflash:
When the copying completes, you can reboot the just-copied Catalyst 4500 series switch image to the image stored in the bootflash memory with the reload command: Switch# reload
System configuration has been modified. Save? [yes/no]: no
Proceed with reload? [confirm] 00:06:17: %SYS-5-RELOAD: Reload requested by console. Reload Reason: Reload Command. To see details about the default parameters set by the erase /all non-default command, see the usage guidelines for the erase command page in the Catalyst 4500 Series Switch Command Reference. de, 12.2(50)SG | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Configuring the Switch for the First Time ![]() | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Feedback: Help us help you
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
|
Premier7K>
Premier7K> setenv epasswd yes
Premier7K> run bootcmd
[Alteon Password Recovery 방법]
1. Console로 L4 Switch에 접속 (Telnet으로는 Recovery 안됨)
2. "Enter Password:"가 나오면 " forgetMe! " 입력
3. /c/sys/user에 들어가서.. Password를 변경함.
When creating a loop-free logical topology, Spanning Tree always uses the same
four-step decision sequence:
Step 1.
Lowest Root BID
Step 2.
Lowest Path Cost to Root Bridge
Step 3.
Lowest Sender BID
Step 4.
Lowest Port ID
Bridges pass Spanning Tree information between themselves using special frames
known as bridge protocol data units (BPDUs). A bridge uses this four-step decision
sequence to save a copy of the best BPDU seen on every port. When making this
evaluation, it considers all of the BPDUs received on the port as well as the BPDU
that would be sent on that port. As every BPDU arrives, it is checked against this
four-step sequence to see if it is more attractive (that is, lower in value) than the
existing BPDU saved for that port. If the new BPDU (or the locally generated BPDU)
is more attractive, the old value is replaced.
|
MIB-2 Interface : 1.3.6.1.2.1.2.2.1 : interface table
Storages : 1.3.6.1.2.1.25.2.2.0 : system memory
CPU
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Net-SNMP Load 1.3.6.1.4.1.2021.10.1 : load table
Memory 1.3.6.1.4.1.2021.4 : memory table
CPU 1.3.6.1.4.1.2021.11 : CPU table
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Cisco Generic for routers and switch Cisco CPU load (5min %) : 1.3.6.1.4.1.9.2.1.58.0 Memory : 1.3.6.1.4.1.9.9.48.1 : cisco memory pool .1 : type 1.3.6.1.4.1.9.9.109.1.1.1.1 : cpmCPUTotalEntry | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Checkpoint FW : 1.3.6.1.4.1.2620.1.1 HA : 1.3.6.1.4.1.2620.1.5.13.1 : table status
1.3.6.1.4.1.2620.1.7.5.0 : "active" : mgmt state 1.3.6.1.4.1.2620.1.7.7.0 : mgmt table clients : | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hewlett Packard
HP Procurve switch memory check |
High CPU Utilization on Cisco IOS Software-Based Catalyst 4500 Switches

Introduction
Prerequisites
Requirements
Components Used
Conventions
Background Information
Understand the Catalyst 4500 CPU Packet-Handling Architecture
Identify the Reason for High CPU Utilization on Catalyst 4500
Baseline the CPU Usage
Understand the show processes cpu Command on the Catalyst 4500 Switches
Understand the show platform health Command on the Catalyst 4500 Switches
Troubleshoot Common High CPU Utilization Problems
High CPU Utilization Due to Process-Switched Packets
Other Causes of High CPU Utilization
Troubleshooting Tools to Analyze the Traffic Destined to the CPU
Tool 1: Monitor the CPU Traffic with SPAN—Cisco IOS Software Release 12.1(19)EW and Later
Tool 2: In-Built CPU Sniffer—Cisco IOS Software Release 12.2(20)EW and Later
Tool 3: Identify the Interface That Sends Traffic to the CPU—Cisco IOS Software Release 12.2(20)EW and Later
Summary
NetPro Discussion Forums - Featured Conversations
Related Information
Introduction
The Catalyst 4500 series switches, which includes the Catalyst 4948 switches, has a sophisticated packet-handling methodology for CPU-bound traffic. A commonly perceived problem is high CPU utilization on these switches. This document provides details about the CPU packet-handling architecture and shows you how to identify the causes of high CPU utilization on these switches. The document also lists some common network or configuration scenarios that cause high CPU utilization on the Catalyst 4500 series.
Note: If you run Catalyst OS (CatOS)-based Catalyst 4500/4000 series switches, refer to the document CPU Utilization on Catalyst 4500/4000, 2948G, 2980G, and 4912G Switches That Run CatOS Software.
Prerequisites
Requirements
There are no specific requirements for this document.
Components Used
The information in this document is based on these software and hardware versions:
-
Catalyst 4500 series switches
-
Catalyst 4948 series switches
Note: This document applies only to Cisco IOS® Software-based switches and not CatOS-based switches.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Conventions
Refer to Cisco Technical Tips Conventions for more information on document conventions.
Background Information
Before you look at the CPU packet-handling architecture and troubleshoot high CPU utilization, you must understand the different ways in which hardware-based forwarding switches and Cisco IOS® Software-based routers use the CPU. The common misconception is that high CPU utilization indicates the depletion of resources on a device and the threat of a crash. A capacity issue is one of the symptoms of high CPU utilization on Cisco IOS routers. However, a capacity issue is almost never a symptom of high CPU utilization with hardware-based forwarding switches like the Catalyst 4500. The Catalyst 4500 is designed to forward packets in the hardware application-specific integrated circuit (ASIC) and reach traffic-forwarding speeds of up to 102 million packets per second (Mpps).
The Catalyst 4500 CPU performs these functions:
-
Manages configured software protocols, for example:
-
Spanning Tree Protocol (STP)
-
Routing protocol
-
Cisco Discovery Protocol (CDP)
-
Port Aggregation Protocol (PAgP)
-
VLAN Trunk Protocol (VTP)
-
Dynamic Trunking Protocol (DTP)
-
-
Programs configuration/dynamic entries to the hardware ASICs, for example:
-
Access control lists (ACLs)
-
CEF entries
-
-
Internally manages various components, for example:
-
Power over Ethernet (PoE) line cards
-
Power supplies
-
Fan tray
-
-
Manages access to the switch, for example:
-
Telnet
-
Console
-
Simple Network Management Protocol (SNMP)
-
-
Forwards packets via the software path, for example:
-
Internetwork Packet Exchange (IPX)-routed packets, which are only supported in the software path
-
Maximum transmission unit (MTU) fragmentation
-
According to this list, high CPU utilization can result from the receipt or process of packets by the CPU. Some of the packets that are sent for process can be essential for the network operation. An example of these essential packets are bridge protocol data unit (BPDUs) for spanning-tree topology configurations. However, other packets can be software-forwarded data traffic. These scenarios require the switching ASICs to send packets to the CPU for processing:
-
Packets that are copied to the CPU, but the original packets are switched in hardware
An example is host MAC address learning.
-
Packets that are sent to the CPU for processing
Examples include:
-
Routing protocol updates
-
BPDUs
-
An intentional or unintentional flood of traffic
-
-
Packets that are sent to the CPU for forwarding
An example is packets that need IPX or AppleTalk routing.
Understand the Catalyst 4500 CPU Packet-Handling Architecture
The Catalyst 4500 has an in-built quality of service (QoS) mechanism in order to differentiate between types of traffic that are destined to the CPU. The mechanism makes the differentiation on the basis of the Layer 2 (L2)/Layer 3 (L3)/ Layer 4 (L4) packet information. The Supervisor packet Engine has 16 queues in order to handle various types of packets or events. Figure 1 shows these queues. Table 1 lists the queues and the packet types that queue in each. The 16 queues allow the Catalyst 4500 to queue the packets on the basis of the packet type or priority.
Figure 1 – Catalyst 4500 Uses Multiple CPU Queues
|
Queue Number |
Queue Name |
Packets Queued |
|---|---|---|
|
0 |
Esmp |
ESMP1 packets (internal management packets) for the line card ASICs or other component management |
|
1 |
Control |
L2 control plane packets, such as STP, CDP, PAgP, LACP2, or UDLD3 |
|
2 |
Host Learning |
Frames with unknown source MAC addresses that are copied to the CPU in order to build the L2 forwarding table |
|
3, 4, 5 |
L3 Fwd Highest, L3 Fwd High/Medium, L3 Fwd Low |
Packets that must be forwarded in software, such as GRE4 tunnels If the ARP5 is unresolved for the destination IP address, packets are sent to this queue. |
|
6, 7, 8 |
L2 Fwd Highest, L2 Fwd High/Medium, L2 Fwd Low |
Packets that are forwarded as a result of bridging
|
|
9, 10 |
L3 Rx High, L3 Rx Low |
L3 control plane traffic, for example, routing protocols, that is destined for CPU IP addresses Examples include Telnet, SNMP, and SSH8. |
|
11 |
RPF Failure |
Multicast packets that failed the RPF9 check |
|
12 |
ACL fwd(snooping) |
Packets that are processed by the DHCP10 snooping, dynamic ARP inspection, or IGMP11 snooping features |
|
13 |
ACL log, unreach |
Packets that hit an ACE12 with the log keyword or packets that were dropped due to a deny in an output ACL or the lack of a route to the destination These packets require the generation of ICMP unreachable messages. |
|
14 |
ACL sw processing |
Packets that are punted to the CPU due to a lack of additional ACL hardware resources, such as TCAM13, for security ACL |
|
15 |
MTU Fail/Invalid |
Packets that need to be fragmented because the output interface MTU size is smaller than the size of the packet |
1 ESMP = Even Simple Management Protocol.
2 LACP = Link Aggregation Control Protocol.
3 UDLD = UniDirectional Link Detection.
4 GRE = generic routing encapsulation.
5 ARP = Address Resolution Protocol.
6 SVI = switched virtual interface.
7 TTL = Time to Live.
8 SSH = Secure Shell Protocol.
9 RPF = Reverse Path Forwarding
10 DHCP = Dynamic Host Configuration Protocol.
11 IGMP = Internet Group Management Protocol.
12 ACE = access control entry.
13 TCAM = ternary content addressable memory.
These queues are separate queues:
-
L2 Fwd Highest or L3 Fwd Highest
-
L2 Fwd High/Medium or L3 Fwd High/Medium
-
L2 Fwd Low or L3 Fwd Low
-
L3 Rx High or L3 Rx Low
Packets are queued into these queues on the basis of the QoS label, which is the differentiated services code point (DSCP) value from the IP type of service (ToS). For example, packets with a DSCP of 63 are queued to the L3 Fwd Highest queue. You can see the packets that are received and dropped for these 16 queues in the output of the show platform cpu packet statistics all command. The output of this command is very long. Issue the show platform cpu packet statistics command in order to show only the nonzero events. An alternate command is the show platform cpuport command. Only use the show platform cpuport command if you run Cisco IOS Software Release 12.1(11)EW or earlier. This command has since been deprecated. However, this older command was a part of the show tech-support command in Cisco IOS Software releases earlier than Cisco IOS Software Release 12.2(20)EWA.
Use the show platform cpu packet statistics command for all troubleshooting.
Switch#show platform cpu packet statistics all !--- Output suppressed. Total packet queues 16 Packets Received by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Esmp 0 0 0 0 0 Control 48 0 0 0 0 Host Learning 0 0 0 0 0 L3 Fwd High 0 0 0 0 0 L3 Fwd Medium 0 0 0 0 0 L3 Fwd Low 0 0 0 0 0 L2 Fwd High 0 0 0 0 0 L2 Fwd Medium 0 0 0 0 0 L2 Fwd Low 0 0 0 0 0 L3 Rx High 0 0 0 0 0 L3 Rx Low 0 0 0 0 0 RPF Failure 0 0 0 0 0 ACL fwd(snooping) 0 0 0 0 0 ACL log, unreach 0 0 0 0 0 ACL sw processing 0 0 0 0 0 MTU Fail/Invalid 0 0 0 0 0 Packets Dropped by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Esmp 0 0 0 0 0 Control 0 0 0 0 0 Host Learning 0 0 0 0 0 L3 Fwd High 0 0 0 0 0 L3 Fwd Medium 0 0 0 0 0 L3 Fwd Low 0 0 0 0 0 L2 Fwd High 0 0 0 0 0 L2 Fwd Medium 0 0 0 0 0 L2 Fwd Low 0 0 0 0 0 L3 Rx High 0 0 0 0 0 L3 Rx Low 0 0 0 0 0 RPF Failure 0 0 0 0 0 ACL fwd(snooping) 0 0 0 0 0 ACL log, unreach 0 0 0 0 0 ACL sw processing 0 0 0 0 0 MTU Fail/Invalid 0 0 0 0 0
The Catalyst 4500 CPU assigns weights to the various queues that Table 1 lists. The CPU assigns the weights on the basis of importance, or type, and on the basis of traffic priority, or DSCP. The CPU services the queue on the basis of the relative weights of the queue. For example, if both a control packet, such as a BPDU, and an ICMP echo request are pending, the CPU services the control packet first. An excessive amount of low-priority or less-important traffic does not starve the CPU of the ability to process or manage the system. This mechanism guarantees that the network is stable even under high utilization of the CPU. This ability of the network to remain stable is critical information that you must understand.
There is another very important implementation detail of Catalyst 4500 CPU packet handling. If the CPU has already serviced high-priority packets or processes but has more spare CPU cycles for a particular time period, the CPU services the low-priority queue packets or performs background processes of lower priority. High CPU utilization as a result of low-priority packet processing or background processes is considered normal because the CPU constantly tries to use all the time available. In this way, the CPU strives for maximum performance of the switch and network without a compromise of the stability of the switch. The Catalyst 4500 considers the CPU underutilized unless the CPU is used at 100 percent for a single time slot.
Cisco IOS Software Release 12.2(25)EWA2 and later have enhanced the CPU packet- and process-handling mechanism and accounting. Therefore, use these releases on your Catalyst 4500 deployments.
Identify the Reason for High CPU Utilization on Catalyst 4500
Now that you understand the Catalyst 4500 CPU packet-handling architecture and design, you may still wish to identify why your Catalyst 4500 CPU utilization is high. The Catalyst 4500 has the commands and tools that are necessary to identify the root cause of the high CPU utilization. After you identify the reason, the administrators can perform either of these actions:
-
Corrective Action—This can include configuration or network changes, or the creation of a Cisco Technical Support service request for further analysis.
-
No action—The Catalyst 4500 performs according to the expectation. The CPU exhibits high CPU utilization because the Supervisor Engine maximizes the CPU cycles in order to perform all the necessary software packet forwarding and background jobs.
Be sure to identify the reason for high CPU utilization even though corrective action is not necessary in all cases. High CPU utilization can be just a symptom of an issue in the network. A resolution of the root cause of that problem can be necessary in order to lower the CPU utilization.
Figure 2 shows the troubleshooting methodology to use in order to identify the root cause of the Catalyst 4500 high CPU utilization.
Figure 2 – High CPU Utilization Troubleshooting Methodology on Catalyst 4500 Switches
The general troubleshooting steps are:
-
Issue the show processes cpu command in order to identify the Cisco IOS processes that consume CPU cycles.
-
Issue the show platform health command in order to further identify the platform-specific processes.
-
If the highly active process is K2CpuMan Review , issue the show platform cpu packet statistics command in order to identity the type of traffic that hits the CPU.
If the activity is not due to the K2CpuMan Review process, skip Step 4 and go on to Step 5.
-
Identify the packets that hit the CPU with use of the Troubleshooting Tools to Analyze the Traffic Destined to the CPU, if necessary.
An example of the troubleshooting tools to use is the CPU Switched Port Analyzer (SPAN).
-
Review this document and the section Troubleshoot Common High CPU Utilization Problems for common causes.
If you still cannot identify the root cause, contact Cisco Technical Support.
Baseline the CPU Usage
The important first step is to know the CPU utilization of your switch for your configuration and network setup. Use the show processes cpu command in order to identify the CPU utilization on the Catalyst 4500 switch. The continual update of baseline CPU utilization can be necessary as you add more configuration to the network setup or as your network traffic pattern changes. Figure 2 indicates this requirement.
This output is from a fully loaded Catalyst 4507R. The steady-state CPU is about 32 to 38 percent, which is necessary in order to perform the management functions for this switch:
Switch#show processes cpu CPU utilization for five seconds: 38%/1%; one minute: 32%; five minutes: 32% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 0 63 0 0.00% 0.00% 0.00% 0 Chunk Manager 2 60 50074 1 0.00% 0.00% 0.00% 0 Load Meter 3 0 1 0 0.00% 0.00% 0.00% 0 Deferred Events !--- Output suppressed. 27 524 250268 2 0.00% 0.00% 0.00% 0 TTY Background 28 816 254843 3 0.00% 0.00% 0.00% 0 Per-Second Jobs 29 101100 5053 20007 0.00% 0.01% 0.00% 0 Per-minute Jobs 30 26057260 26720902 975 12.07% 11.41% 11.36% 0 Cat4k Mgmt HiPri 31 19482908 29413060 662 24.07% 19.32% 19.20% 0 Cat4k Mgmt LoPri 32 4468 162748 27 0.00% 0.00% 0.00% 0 Galios Reschedul 33 0 1 0 0.00% 0.00% 0.00% 0 IOS ACL Helper 34 0 2 0 0.00% 0.00% 0.00% 0 NAM Manager
Five-second CPU utilization is expressed as:
x%/y%
The x% represents total CPU utilization, and y% represents the CPU that is spent at the interrupt level. When you troubleshoot Catalyst 4500 switches, focus only on the total CPU utilization.
Understand the show processes cpu Command on the Catalyst 4500 Switches
This show processes cpu output shows that there are two processes that use the CPU— Cat4k Mgmt HiPri and Cat4k Mgmt LoPri . These two processes aggregate multiple platform-specific processes which perform the essential management functions on the Catalyst 4500. These processes process control plane as well as data packets that need to be software-switched or processed.
In order to see which of the platform-specific processes use the CPU under the context of Cat4k Mgmt HiPri and Cat4k Mgmt LoPri , issue the show platform health command.
Each of the platform-specific processes has a target/expected utilization of the CPU. When that process is within the target, the CPU executes the process in the high-priority context. The show processes cpu command output counts that utilization under Cat4k Mgmt HiPri . If a process exceeds the target/expected utilization, that process runs under the low-priority context. The show processes cpu command output counts that additional utilization under Cat4k Mgmt LoPri . This Cat4k Mgmt LoPri is also used to run background and other low-priority processes, such as consistency check and reading interface counters. This mechanism allows the CPU to run high-priority processes when necessary, and the idle CPU cycles that remain are used for the low-priority processes. To exceed the target CPU utilization by a small amount, or a momentary spike in utilization, is not an indication of a problem that needs investigation.
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU Lj-poll 1.00 0.02 2 1 100 500 0 0 0 1:09 GalChassisVp-review 3.00 0.29 10 3 100 500 0 0 0 11:15 S2w-JobEventSchedule 10.00 0.32 10 7 100 500 0 0 0 10:14 Stub-JobEventSchedul 10.00 12.09 10 6 100 500 14 13 9 396:35 StatValueMan Update 1.00 0.22 1 0 100 500 0 0 0 6:28 Pim-review 0.10 0.00 1 0 100 500 0 0 0 0:22 Ebm-host-review 1.00 0.00 8 0 100 500 0 0 0 0:05 Ebm-port-review 0.10 0.00 1 0 100 500 0 0 0 0:01 Protocol-aging-revie 0.20 0.00 2 0 100 500 0 0 0 0:00 Acl-Flattener e 1.00 0.00 10 0 100 500 0 0 0 0:00 KxAclPathMan create/ 1.00 0.00 10 5 100 500 0 0 0 0:39 KxAclPathMan update 2.00 0.00 10 0 100 500 0 0 0 0:00 KxAclPathMan reprogr 1.00 0.00 2 0 100 500 0 0 0 0:00 TagMan-RecreateMtegR 1.00 0.00 10 0 100 500 0 0 0 0:00 K2CpuMan Review 30.00 10.19 30 28 100 500 14 13 9 397:11 K2AccelPacketMan: Tx 10.00 2.20 20 0 100 500 2 2 1 82:06 K2AccelPacketMan: Au 0.10 0.00 0 0 100 500 0 0 0 0:00 K2AclMan-taggedFlatA 1.00 0.00 10 0 100 500 0 0 0 0:00 K2AclCamMan stale en 1.00 0.00 10 0 100 500 0 0 0 0:00 K2AclCamMan hw stats 3.00 1.04 10 5 100 500 1 1 0 39:36 K2AclCamMan kx stats 1.00 0.00 10 5 100 500 0 0 0 13:40 K2AclCamMan Audit re 1.00 0.00 10 5 100 500 0 0 0 13:10 K2AclPolicerTableMan 1.00 0.00 10 1 100 500 0 0 0 0:38 K2L2 Address Table R 2.00 0.00 12 5 100 500 0 0 0 0:00 K2L2 New Static Addr 2.00 0.00 10 1 100 500 0 0 0 0:00 K2L2 New Multicast A 2.00 0.00 10 5 100 500 0 0 0 0:01 K2L2 Dynamic Address 2.00 0.00 10 0 100 500 0 0 0 0:00 K2L2 Vlan Table Revi 2.00 0.00 12 9 100 500 0 0 0 0:01 K2 L2 Destination Ca 2.00 0.00 10 0 100 500 0 0 0 0:00 K2PortMan Review 2.00 0.72 15 11 100 500 1 1 0 37:22 Gigaport65535 Review 0.40 0.07 4 2 100 500 0 0 0 3:38 Gigaport65535 Review 0.40 0.08 4 2 100 500 0 0 0 3:39 K2Fib cam usage revi 2.00 0.00 15 0 100 500 0 0 0 0:00 K2Fib IrmFib Review 2.00 0.00 15 0 100 500 0 0 0 0:00 K2Fib Vrf Default Ro 2.00 0.00 15 0 100 500 0 0 0 0:00 K2Fib AdjRepop Revie 2.00 0.00 15 0 100 500 0 0 0 0:00 K2Fib Vrf Unpunt Rev 2.00 0.01 15 0 100 500 0 0 0 0:23 K2Fib Consistency Ch 1.00 0.00 5 2 100 500 0 0 0 29:25 K2FibAdjMan Stats Re 2.00 0.30 10 4 100 500 0 0 0 6:21 K2FibAdjMan Host Mov 2.00 0.00 10 4 100 500 0 0 0 0:00 K2FibAdjMan Adj Chan 2.00 0.00 10 0 100 500 0 0 0 0:00 K2FibMulticast Signa 2.00 0.01 10 2 100 500 0 0 0 2:04 K2FibMulticast Entry 2.00 0.00 10 7 100 500 0 0 0 0:00 K2FibMulticast Irm M 2.00 0.00 10 7 100 500 0 0 0 0:00 K2FibFastDropMan Rev 2.00 0.00 7 0 100 500 0 0 0 0:00 K2FibPbr route map r 2.00 0.06 20 5 100 500 0 0 0 16:42 K2FibPbr flat acl pr 2.00 0.07 20 2 100 500 0 0 0 3:24 K2FibPbr consolidati 2.00 0.01 10 0 100 500 0 0 0 0:24 K2FibPerVlanPuntMan 2.00 0.00 15 4 100 500 0 0 0 0:00 K2FibFlowCache flow 2.00 0.01 10 0 100 500 0 0 0 0:23 K2FibFlowCache flow 2.00 0.00 10 0 100 500 0 0 0 0:00 K2FibFlowCache adj r 2.00 0.01 10 0 100 500 0 0 0 0:20 K2FibFlowCache flow 2.00 0.00 10 0 100 500 0 0 0 0:06 K2MetStatsMan Review 2.00 0.14 5 2 100 500 0 0 0 23:40 K2FibMulticast MET S 2.00 0.00 10 0 100 500 0 0 0 0:00 K2QosDblMan Rate DBL 2.00 0.12 7 0 100 500 0 0 0 4:52 IrmFibThrottler Thro 2.00 0.01 7 0 100 500 0 0 0 0:21 K2 VlanStatsMan Revi 2.00 1.46 15 7 100 500 2 2 1 64:44 K2 Packet Memory Dia 2.00 0.00 15 8 100 500 0 1 1 45:46 K2 L2 Aging Table Re 2.00 0.12 20 3 100 500 0 0 0 7:22 RkiosPortMan Port Re 2.00 0.73 12 7 100 500 1 1 1 52:36 Rkios Module State R 4.00 0.02 40 1 100 500 0 0 0 1:28 Rkios Online Diag Re 4.00 0.02 40 0 100 500 0 0 0 1:15 RkiosIpPbr IrmPort R 2.00 0.02 10 3 100 500 0 0 0 2:44 RkiosAclMan Review 3.00 0.06 30 0 100 500 0 0 0 2:35 MatMan Review 0.50 0.00 4 0 100 500 0 0 0 0:00 Slot 3 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:00 Slot 3 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:00 Slot 4 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:00 Slot 4 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:00 Slot 5 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:00 Slot 5 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:00 Slot 6 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:00 Slot 6 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:00 Slot 7 ILC Manager R 3.00 0.00 10 0 100 500 0 0 0 0:00 Slot 7 ILC S2wMan Re 3.00 0.00 10 0 100 500 0 0 0 0:00 EthHoleLinecardMan(1 1.66 0.04 10 0 100 500 0 0 0 1:18 EthHoleLinecardMan(2 1.66 0.02 10 0 100 500 0 0 0 1:18 EthHoleLinecardMan(6 1.66 0.17 10 6 100 500 0 0 0 6:38 ------------- %CPU Totals 212.80 35.63
Understand the show platform health Command on the Catalyst 4500 Switches
The show platform health command provides a lot of information that is relevant only for a development engineer. In order to troubleshoot high CPU utilization, look for a higher number in the %CPU actual column in the output. Also, be sure to glance at the right side of that row in order to verify the CPU usage of that process in the 1 minute and 1 hour average %CPU columns. Sometimes, processes momentarily peak but do not hold the CPU for a long time. Some of the momentarily high CPU utilization happens during hardware programming or optimization of the programming. For example, a spike of CPU utilization is normal during the hardware programming of a large ACL in the TCAM.
In the show platform health command output in the section Understand the show processes cpu Command on the Catalyst 4500 Switches, the Stub-JobEventSchedul and the K2CpuMan Review processes use a higher number of CPU cycles. Table 2 provides some basic information about the common platform-specific processes that appear in the output of the show platform health command.
|
Platform-Specific Process Name |
Description |
|---|---|
|
Pim-review |
Chassis/line card state management |
|
Ebm |
Ethernet bridge module, such as aging and monitoring |
|
Acl-Flattener / K2AclMan |
ACL merging process |
|
KxAclPathMan - Path TagMan-Review |
ACL state management and maintenance |
|
K2CpuMan Review |
The process that performs software packet forwarding If you see high CPU utilization due to this process, investigate the packets that hit the CPU with use of the show platform cpu packet statistics command. |
|
K2AccelPacketMan |
The driver that interacts with the packet engine in order to send packets that are destined from the CPU |
|
K2AclCamMan |
Manages the input and output TCAM hardware for QoS and security features |
|
K2AclPolicerTableMan |
Manages the input and output policers |
|
K2L2 |
Represents the L2 forwarding subsystem of the Catalyst 4500 Cisco IOS Software These processes are responsible for maintenance of the various L2 tables. |
|
K2PortMan Review |
Manages the various port-related programming functions |
|
K2Fib |
FIB1 management |
|
K2FibFlowCache |
PBR2 cache management |
|
K2FibAdjMan |
FIB adjacency table management |
|
K2FibMulticast |
Manages multicast FIB entries |
|
K2MetStatsMan Review |
Manages MET3 statistics |
|
K2QosDblMan Review |
Manages QoS DBL4 |
|
IrmFibThrottler Thro |
IP routing module |
|
K2 L2 Aging Table Re |
Manages the L2 aging function |
|
GalChassisVp-review |
Chassis state monitoring |
|
S2w-JobEventSchedule |
Manages the S2W5 protocols to monitor line cards state |
|
Stub-JobEventSchedul |
Stub ASIC-based line card monitoring and maintenance |
|
RkiosPortMan Port Re |
Port state monitoring and maintenance |
|
Rkios Module State R |
Line card monitoring and maintenance |
|
EthHoleLinecardMan |
Manages GBICs6 in each of the line cards |
1 FIB = Forwarding Information Base.
2 PBR = policy-based routing.
3 MET = Multicast Expansion Table.
4 DBL = Dynamic Buffer Limiting.
5 S2W = serial-to-wire.
6 GBIC = Gigabit Interface Converter.
Troubleshoot Common High CPU Utilization Problems
This section covers some of the common high CPU utilization problems on the Catalyst 4500 switches.
High CPU Utilization Due to Process-Switched Packets
One of the common reasons for high CPU utilization is that the Catalyst 4500 CPU is busy with the process of packets for software-forwarded packets or control packets. Examples of software-forwarded packets are IPX or control packets, such as BPDUs. A small number of these packets is typically sent to the CPU. However, a consistently large number of packets can indicate a configuration error or a network event. You must identify the cause of events that lead to the forward of packets to the CPU for processing. This identification enables you to debug the high CPU utilization problems.
Some of the common reasons for high CPU utilization due to process-switched packets are:
Other reasons for the switch of packets to the CPU are:
-
MTU fragmentation—Be sure that all interfaces along the path of the packet have the same MTU.
-
ACL with TCP flags other than established
-
IP version 6 (IPv6) routing—This is supported only via the software-switching path.
-
GRE—This is supported only via the software-switching path.
-
Denial of traffic in the input or output router ACL (RACL)
Note: This is rate-limited in Cisco IOS Software Release 12.1(13)EW1 and later.
Issue the no ip unreachables command under the interface of the ACL.
-
Excessive ARP and DHCP traffic hits the CPU for processing due to a large number of directly connected hosts
If you suspect a DHCP attack, use DCHP snooping to rate-limit DHCP traffic from any specific host port.
-
Excessive SNMP polling by a legitimate or misbehaving end station
A High Number of Spanning-Tree Port Instances
The Catalyst 4500 supports 3000 spanning-tree port instances or active ports in the Per VLAN Spanning Tree+ (PVST+) mode. The support is on all Supervisor Engines, except the Supervisor Engine II+ and II+TS, and the Catalyst 4948. The Supervisor Engine II+ and II+TS and the Catalyst 4948 support up to 1500 port instances. If you exceed these STP-instance recommendations, the switch exhibits high CPU utilization.
This diagram shows a Catalyst 4500 with three trunk ports that each carry VLANs 1 through 100. This equates to 300 spanning-tree port instances. In general, you can calculate spanning-tree port instances with this formula:
Total number of STP instances = Number of access ports + Sum of all VLANs that are carried in each of the trunks
In the diagram, there are no access ports, but the three trunks carry VLANs 1 through 100:
Total number of STP instances = 0 + 100 + 100 + 100 = 300
Step 1: Check for the Cisco IOS process with the show processes cpu command.
This section reviews the commands that an administrator uses in order to narrow down the problem of high CPU utilization. If you issue the show processes cpu command, you can see that two main processes, Cat4k Mgmt LoPri and Spanning Tree , primarily use the CPU. With only this information, you know that the spanning tree process consumes a sizable portion of the CPU cycles.
Switch#show processes cpu CPU utilization for five seconds: 74%/1%; one minute: 73%; five minutes: 50% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 4 198 20 0.00% 0.00% 0.00% 0 Chunk Manager 2 4 290 13 0.00% 0.00% 0.00% 0 Load Meter !--- Output suppressed. 25 488 33 14787 0.00% 0.02% 0.00% 0 Per-minute Jobs 26 90656 223674 405 6.79% 6.90% 7.22% 0 Cat4k Mgmt HiPri 27 158796 59219 2681 32.55% 33.80% 21.43% 0 Cat4k Mgmt LoPri 28 20 1693 11 0.00% 0.00% 0.00% 0 Galios Reschedul 29 0 1 0 0.00% 0.00% 0.00% 0 IOS ACL Helper 30 0 2 0 0.00% 0.00% 0.00% 0 NAM Manager !--- Output suppressed. 41 0 1 0 0.00% 0.00% 0.00% 0 SFF8472 42 0 2 0 0.00% 0.00% 0.00% 0 AAA Dictionary R 43 78564 20723 3791 32.63% 30.03% 17.35% 0 Spanning Tree 44 112 999 112 0.00% 0.00% 0.00% 0 DTP Protocol 45 0 147 0 0.00% 0.00% 0.00% 0 Ethchnl
Step 2: Check for the Catalyst 4500-specific process with the show platform health command.
In order to understand which platform-specific process consumes the CPU, issue the show platform health command. From this output, you can see that the K2CpuMan Review process, a job to handle CPU-bound packets, uses up the CPU:
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU !--- Output suppressed. TagMan-RecreateMtegR 1.00 0.00 10 0 100 500 0 0 0 0:00 K2CpuMan Review 30.00 37.62 30 53 100 500 41 33 1 2:12 K2AccelPacketMan: Tx 10.00 4.95 20 0 100 500 5 4 0 0:36 K2AccelPacketMan: Au 0.10 0.00 0 0 100 500 0 0 0 0:00 K2AclMan-taggedFlatA 1.00 0.00 10 0 100 500 0 0 0 0:00
Step 3: Check the CPU queue that receives traffic in order to identify the type of CPU-bound traffic.
Issue the show platform cpu packet statistics command in order to check which CPU queue receives the CPU-bound packet. The output in this section shows that the control queue receives a lot of packets. Use the information in Table 1 and the conclusion that you drew in Step 1. You can determine that the packets that the CPU processes and the reason for the high CPU utilization is BPDU processing.
Switch#show platform cpu packet statistics !--- Output suppressed. Total packet queues 16 Packets Received by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Esmp 202760 196 173 128 28 Control 388623 2121 1740 598 16 Packets Dropped by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Control 17918 0 19 24 3
Step 4: Identify the root cause.
Issue the show spanning-tree summary command. You can check if the receipt of BPDUs is because of a high number of spanning-tree port instances. The output clearly identifies the root cause:
Switch#show spanning-tree summary Switch is in pvst mode Root bridge for: none Extended system ID is enabled Portfast Default is disabled PortFast BPDU Guard Default is disabled Portfast BPDU Filter Default is disabled Loopguard Default is disabled EtherChannel misconfig guard is enabled UplinkFast is disabled BackboneFast is disabled Configured Pathcost method used is short !--- Output suppressed. Name Blocking Listening Learning Forwarding STP Active ---------------------- -------- --------- -------- ---------- ---------- 2994 vlans 0 0 0 5999 5999
There are a large number of VLANs with the PVST+ mode configuration. In order to resolve the issue, change the STP mode to Multiple Spanning Tree (MST). In some cases, the number of STP instances is high because a high number of VLANs are forwarded on all trunk ports. In this case, manually prune the VLANs that are not necessary from the trunk in order to drop the number of STP active ports to well below the recommended value.
Tip: Be sure that you do not configure IP phone ports as trunk ports. This is a common misconfiguration. Configure IP phone ports with a voice VLAN configuration. This configuration creates a pseudo trunk, but does not require you to manually prune the unnecessary VLANs. For more information on how to configure voice ports, refer to the Configuring Voice Interfaces software configuration guide. Non-Cisco IP phones do not support this voice VLAN or auxiliary VLAN configuration. You must manually prune the ports with non-Cisco IP phones.
ICMP Redirects; Routing Packets on the Same Interface
Routing packets on the same interface, or traffic ingress and egress on the same L3 interface, can result in an ICMP redirect by the switch. If the switch knows that the next hop device to the ultimate destination is in the same subnet as the sending device, the switch generates ICMP redirect to the source. The redirect messages indicate to the source to send the packet directly to the next hop device. The messages indicate that the next hop device has a better route to the destination, a route of one less hop than this switch.
In the diagram in this section, PC A communicates with the web server. The default gateway of PC A points to the VLAN 100 interface IP address. However, the next hop router that enables the Catalyst 4500 to reach the destination is in the same subnet as PC A. The best path in this case is to send directly to "Router". Catalyst 4500 sends an ICMP redirect message to PC A. The message instructs PC A to send the packets destined to the web server via Router, instead of via Catalyst 4500. However, in most cases, the end devices do not respond to the ICMP redirect. The lack of response causes the Catalyst 4500 to spend a lot of CPU cycles on the generation of these ICMP redirects for all the packets that the Catalyst forwards via the same interface as the ingress packets.
By default, ICMP redirect is enabled. In order to disable it, use the no ip icmp redirects command. Issue the command under the relevant SVI or L3 interface.
Note: Since ip icmp redirects is a default command, it is not visible in the show running-configuration command output.
Step 1: Check for the Cisco IOS process with the show processes cpu command.
Issue the show processes cpu command. You can see that two main processes, Cat4k Mgmt LoPri and IP Input , primarily use the CPU. With only this information, you know that the process of IP packets expends a sizable portion of the CPU.
Switch#show processes cpu CPU utilization for five seconds: 38%/1%; one minute: 32%; five minutes: 32% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 0 63 0 0.00% 0.00% 0.00% 0 Chunk Manager 2 60 50074 1 0.00% 0.00% 0.00% 0 Load Meter 3 0 1 0 0.00% 0.00% 0.00% 0 Deferred Events !--- Output suppressed. 27 524 250268 2 0.00% 0.00% 0.00% 0 TTY Background 28 816 254843 3 0.00% 0.00% 0.00% 0 Per-Second Jobs 29 101100 5053 20007 0.00% 0.01% 0.00% 0 Per-minute Jobs 30 26057260 26720902 975 5.81% 6.78% 5.76% 0 Cat4k Mgmt HiPri 31 19482908 29413060 662 19.64% 18.20% 20.48% 0 Cat4k Mgmt LoPri !--- Output suppressed. 35 60 902 0 0.00% 0.00% 0.00% 0 DHCP Snooping 36 504625304 645491491 781 72.40% 72.63% 73.82% 0 IP Input
Step 2: Check for the Catalyst 4500-specific process with the show platform health command.
The output of the show platform health command confirms the use of the CPU in order to process CPU-bound packets.
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU --- Output suppressed. TagMan-RecreateMtegR 1.00 0.00 10 0 100 500 0 0 0 0:00 K2CpuMan Review 330.00 19.18 150 79 25 500 20 19 18 5794:08 K2AccelPacketMan: Tx 10.00 4.95 20 0 100 500 5 4 0 0:36 K2AccelPacketMan: Au 0.10 0.00 0 0 100 500 0 0 0 0:00 K2AclMan-taggedFlatA 1.00 0.00 10 0 100 500 0 0 0 0:00
Step 3: Check the CPU queue that receives traffic in order to identify the type of CPU-bound traffic.
Issue the show platform cpu packet statistics command in order to check which CPU queue receives the CPU-bound packet. You can see that the L3 Fwd Low queue receives quite a lot of traffic.
Switch#show platform cpu packet statistics !--- Output suppressed. Packets Received by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Esmp 48613268 38 39 38 39 Control 142166648 74 74 73 73 Host Learning 1845568 2 2 2 2 L3 Fwd High 17 0 0 0 0 L3 Fwd Medium 2626 0 0 0 0 L3 Fwd Low 4717094264 3841 3879 3873 3547 L2 Fwd Medium 1 0 0 0 0 L3 Rx High 257147 0 0 0 0 L3 Rx Low 5325772 10 19 13 7 RPF Failure 155 0 0 0 0 ACL fwd(snooping) 65604591 53 54 54 53 ACL log, unreach 11013420 9 8 8 8
Step 4: Identify the root cause.
In this case, use the CPU SPAN in order to determine the traffic that hits the CPU. For information about the CPU SPAN, see the Tool 1: Monitor the CPU Traffic with SPAN—Cisco IOS Software Release 12.1(19)EW and Later section of this document. Complete an analysis of the traffic and a configuration with use of the show running-configuration command. In this case, a packet is routed through the same interface, which leads to the issue of an ICMP redirect for each packet. This root cause is one of the common reasons for high CPU utilization on the Catalyst 4500.
You may expect the sourcing device to act on the ICMP redirect that the Catalyst 4500 sends and change the next hop for the destination. However, not all devices respond to an ICMP redirect. If the device does not respond, the Catalyst 4500 must send redirects for every packet that the switch receives from the sending device. These redirects can consume a great deal of CPU resources. The solution is to disable ICMP redirect. Issue the no ip redirects command under the interfaces.
This scenario can occur when you also have configured secondary IP addresses. When you enable the secondary IP addresses, IP redirect is automatically disabled. Be sure you do not manually enable the IP redirects.
As this ICMP Redirects; Routing Packets on the Same Interface section has indicated, most end devices do not respond to ICMP redirects. Therefore, as a general practice, disable this feature.
IPX or AppleTalk Routing
The Catalyst 4500 supports IPX and AppleTalk routing via software-forwarding path only. With the configuration of such protocols, a higher CPU utilization is normal.
Note: The switching of IPX and AppleTalk traffic in the same VLAN does not require process switching. Only packets that need to be routed require software path forwarding.
Step 1: Check for the Cisco IOS process with the show processes cpu command.
Issue the show processes cpu command in order to check which Cisco IOS process consumes the CPU. In this command output, notice that the top process is the Cat4k Mgmt LoPri :
witch#show processes cpu CPU utilization for five seconds: 87%/10%; one minute: 86%; five minutes: 87% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 4 53 75 0.00% 0.00% 0.00% 0 Chunk Manager !--- Output suppressed. 25 8008 1329154 6 0.00% 0.00% 0.00% 0 Per-Second Jobs 26 413128 38493 10732 0.00% 0.02% 0.00% 0 Per-minute Jobs 27 148288424 354390017 418 2.60% 2.42% 2.77% 0 Cat4k Mgmt HiPri 28 285796820 720618753 396 50.15% 59.72% 61.31% 0 Cat4k Mgmt LoPri
Step 2: Check for the Catalyst 4500-specific process with the show platform health command.
The output of the show platform health command confirms the use of the CPU in order to process CPU-bound packets.
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU !--- Output suppressed. TagMan-RecreateMtegR 1.00 0.00 10 4 100 500 0 0 0 0:00 K2CpuMan Review 30.00 27.39 30 53 100 500 42 47 42 4841: K2AccelPacketMan: Tx 10.00 8.03 20 0 100 500 21 29 26 270:4
Step 3: Check the CPU queue that receives traffic in order to identify the type of CPU-bound traffic.
In order to determine the type of traffic that hits the CPU, issue the show platform cpu packet statistics command.
Switch#show platform cpu packet statistics !--- Output suppressed. Packets Received by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Esmp 48613268 38 39 38 39 Control 142166648 74 74 73 73 Host Learning 1845568 2 2 2 2 L3 Fwd High 17 0 0 0 0 L3 Fwd Medium 2626 0 0 0 0 L3 Fwd Low 1582414 1 1 1 1 L2 Fwd Medium 1 0 0 0 0 L2 Fwd Low 576905398 1837 1697 1938 1515 L3 Rx High 257147 0 0 0 0 L3 Rx Low 5325772 10 19 13 7 RPF Failure 155 0 0 0 0 ACL fwd(snooping) 65604591 53 54 54 53 ACL log, unreach 11013420 9 8 8 8
Step 4: Identify the root cause.
Since the administrator has configured IPX or AppleTalk routing, identification of the root cause should be straightforward. But in order to confirm, SPAN the CPU traffic and be sure that the traffic that you see is the expected traffic. For information about the CPU SPAN, see the Tool 1: Monitor the CPU Traffic with SPAN—Cisco IOS Software Release 12.1(19)EW and Later section of this document.
In this case, the administrator must update the baseline CPU to the current value. The Catalyst 4500 CPU behaves as expected when the CPU processes software-switched packets.
Host Learning
The Catalyst 4500 learns the MAC addresses of various hosts, if the MAC address is not already in the MAC address table. The switching engine forwards a copy of the packet with the new MAC address to the CPU.
All the VLAN interfaces (layer 3) use the chassis base hardware address as their MAC address. As a result, there is not an entry in the MAC address table, and the packets destined to these VLAN interfaces are not sent to the CPU for processing.
If there is an excessive number of new MAC addresses for the switch to learn, high CPU utilization can result.
Step 1: Check for the Cisco IOS process with the show processes cpu command.
Issue the show processes cpu command in order to check which Cisco IOS process consumes the CPU. In this command output, notice that the top process is the Cat4k Mgmt LoPri :
Switch#show processes cpu CPU utilization for five seconds: 89%/1%; one minute: 74%; five minutes: 71% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 4 53 75 0.00% 0.00% 0.00% 0 Chunk Manager !--- Output suppressed. 25 8008 1329154 6 0.00% 0.00% 0.00% 0 Per-Second Jobs 26 413128 38493 10732 0.00% 0.02% 0.00% 0 Per-minute Jobs 27 148288424 354390017 418 26.47% 10.28% 10.11% 0 Cat4k Mgmt HiPri 28 285796820 720618753 396 52.71% 56.79% 55.70% 0 Cat4k Mgmt LoPri
Step 2: Check for the Catalyst 4500-specific process with the show platform health command.
The output of the show platform health command confirms the use of the CPU in order to process CPU-bound packets.
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU !--- Output suppressed. TagMan-RecreateMtegR 1.00 0.00 10 4 100 500 0 0 0 0:00 K2CpuMan Review 30.00 46.88 30 47 100 500 30 29 21 265:01 K2AccelPacketMan: Tx 10.00 8.03 20 0 100 500 21 29 26 270:4
Step 3: Check the CPU queue that receives traffic in order to identify the type of CPU-bound traffic.
In order to determine the type of traffic that hits the CPU, issue the show platform cpu packet statistics command.
Switch#show platform cpu packet statistics !--- Output suppressed. Packets Received by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Esmp 48613268 38 39 38 39 Control 142166648 74 74 73 73 Host Learning 1845568 1328 1808 1393 1309 L3 Fwd High 17 0 0 0 0 L3 Fwd Medium 2626 0 0 0 0 L3 Fwd Low 1582414 1 1 1 1 L2 Fwd Medium 1 0 0 0 0 L2 Fwd Low 576905398 37 7 8 5 L3 Rx High 257147 0 0 0 0 L3 Rx Low 5325772 10 19 13 7 RPF Failure 155 0 0 0 0 ACL fwd(snooping) 65604591 53 54 54 53 ACL log, unreach 11013420 9 8 8 8
Step 4: Identify the root cause.
The output of the show platform health command shows you that the CPU sees a lot of new MAC addresses. This situation is often the result of network topology instability. For example, if the spanning-tree topology changes, the switch generates Topology Change Notifications (TCNs). The issue of TCNs reduces the aging time to 15 seconds in PVST+ mode. MAC address entries are flushed if the addresses are not learned back within the time period. In the case of Rapid STP (RSTP) (IEEE 802.1w) or MST (IEEE 802.1s), the entries immediately age out if the TCN comes from another switch. This age out causes MAC addresses to be learned anew. This is not a major issue if the topology changes are rare. But there can be an excessive number of topology changes because of a flapping link, faulty switch, or host ports that are not enabled for PortFast. A large number of MAC table flushes and subsequent relearning can result. The next step in root cause identification is to troubleshoot the network. The switch works as expected and sends the packets to the CPU for host address learning. Identify and fix the faulty device that results in excessive TCNs.
Your network can have a lot of devices that send traffic in bursts, which causes MAC addresses to be aged out and subsequently relearned on the switch. In this case, increase the MAC address table aging time in order to provide some relief. With a longer aging time, the switches retain the device MAC addresses in the table for a longer period of time before the age out.
Caution: Make this age-out change only after careful consideration. The change can lead to a traffic black hole if you have devices in your network which are mobile.
Out of Hardware Resources (TCAM) for Security ACL
The Catalyst 4500 programs the configured ACLs with use of the Cisco TCAM. TCAM allows for the application of the ACLs in the hardware-forwarding path. There is no impact on performance of the switch, with or without ACLs in the forwarding path. Performance is constant despite the size of the ACL because performance of the ACL lookups is at line rate. However, TCAM is a finite resource. Therefore, if you configure an excessive number of ACL entries, you exceed the TCAM capacity. Table 3 shows the number of TCAM resources available on each of the Catalyst 4500 Supervisor Engines and switches.
|
Product |
Feature TCAM (per Direction) |
QoS TCAM (per Direction) |
|---|---|---|
|
Supervisor Engine II+/II+TS |
8192 entries with 1024 masks |
8192 entries with 1024 masks |
|
Supervisor Engine III/IV/V and Catalyst 4948 |
16,384 entries with 2048 masks |
16,384 entries with 2048 masks |
|
Supervisor Engine V-10GE and Catalyst 4948-10GE |
16,384 entries with 16,384 masks |
16,384 entries with 16,384 masks |
The switch uses the feature TCAM in order to program the security ACL, such as RACL and VLAN ACL (VACL). The switch also uses the feature TCAM for security features like IP Source Guard (IPSG) for dynamic ACLs. The switch uses the QoS TCAM in order to program classification and policer ACLs.
When the Catalyst 4500 runs out of TCAM resources during the programming of a security ACL, a partial application of the ACL occurs via the software path. The packets that hit those ACEs are processed in software, which causes high CPU utilization. ACL is programmed from the top down. In other words, if the ACL does not fit into the TCAM, the ACE at the bottom portion of the ACL likely is not programmed in the TCAM.
This warning message appears when a TCAM overflow happens:
%C4K_HWACLMAN-4-ACLHWPROGERRREASON: (Suppressed 1times) Input(null, 12/Normal) Security: 140 - insufficient hardware TCAM masks. %C4K_HWACLMAN-4-ACLHWPROGERR: (Suppressed 4 times) Input Security: 140 - hardware TCAM limit, some packet processing will be software switched.
You can see this error message in the show logging command output. The message conclusively indicates that some software processing will take place and, consequently, there can be high CPU utilization.
Note: If you change a large ACL, you can see this message briefly before the changed ACL is programmed again in the TCAM.
Step 1: Check for the Cisco IOS process with the show processes cpu command.
Issue the show processes cpu command. You can see that the CPU utilization is high because the Cat4k Mgmt LoPri process takes up most of the CPU cycles.
Switch#show processes cpu CPU utilization for five seconds: 99%/0%; one minute: 99%; five minutes: 99% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 0 11 0 0.00% 0.00% 0.00% 0 Chunk Manager 2 9716 632814 15 0.00% 0.00% 0.00% 0 Load Meter 3 780 302 2582 0.00% 0.00% 0.00% 0 SpanTree Helper !--- Output suppressed. 23 18208 3154201 5 0.00% 0.00% 0.00% 0 TTY Background 24 37208 3942818 9 0.00% 0.00% 0.00% 0 Per-Second Jobs 25 1046448 110711 9452 0.00% 0.03% 0.00% 0 Per-minute Jobs 26 175803612 339500656 517 4.12% 4.31% 4.48% 0 Cat4k Mgmt HiPri 27 835809548 339138782 2464 86.81% 89.20% 89.76% 0 Cat4k Mgmt LoPri 28 28668 2058810 13 0.00% 0.00% 0.00% 0 Galios Reschedul
Step 2: Check for the Catalyst 4500-specific process with the show platform health command.
Issue the show platform health command. You can see that the K2CpuMan Review, a job to handle CPU-bound packets, uses the CPU.
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU Lj-poll 1.00 0.01 2 0 100 500 0 0 0 13:45 GalChassisVp-review 3.00 0.20 10 16 100 500 0 0 0 88:44 S2w-JobEventSchedule 10.00 0.57 10 7 100 500 1 0 0 404:22 Stub-JobEventSchedul 10.00 0.00 10 0 100 500 0 0 0 0:00 StatValueMan Update 1.00 0.09 1 0 100 500 0 0 0 91:33 Pim-review 0.10 0.00 1 0 100 500 0 0 0 4:46 Ebm-host-review 1.00 0.00 8 4 100 500 0 0 0 14:01 Ebm-port-review 0.10 0.00 1 0 100 500 0 0 0 0:20 Protocol-aging-revie 0.20 0.00 2 0 100 500 0 0 0 0:01 Acl-Flattener 1.00 0.00 10 5 100 500 0 0 0 0:04 KxAclPathMan create/ 1.00 0.00 10 5 100 500 0 0 0 0:21 KxAclPathMan update 2.00 0.00 10 6 100 500 0 0 0 0:05 KxAclPathMan reprogr 1.00 0.00 2 1 100 500 0 0 0 0:00 TagMan-InformMtegRev 1.00 0.00 5 0 100 500 0 0 0 0:00 TagMan-RecreateMtegR 1.00 0.00 10 14 100 500 0 0 0 0:18 K2CpuMan Review 30.00 91.31 30 92 100 500 128 119 84 13039:02 K2AccelPacketMan: Tx 10.00 2.30 20 0 100 500 2 2 2 1345:30 K2AccelPacketMan: Au 0.10 0.00 0 0 100 500 0 0 0 0:00
Step 3: Check the CPU queue that receives traffic in order to identify the type of CPU-bound traffic.
You need to further understand which CPU queue and, therefore, what type of traffic hits the CPU queue. Issue the show platform cpu packet statistics command. You can see that the ACL sw processing queue receives a high number of packets. Therefore, TCAM overflow is the cause of this high CPU utilization issue.
Switch#show platform cpu packet statistics !--- Output suppressed. Packets Received by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Control 57902635 22 16 12 3 Host Learning 464678 0 0 0 0 L3 Fwd Low 623229 0 0 0 0 L2 Fwd Low 11267182 7 4 6 1 L3 Rx High 508 0 0 0 0 L3 Rx Low 1275695 10 1 0 0 ACL fwd(snooping) 2645752 0 0 0 0 ACL log, unreach 51443268 9 4 5 5 ACL sw processing 842889240 1453 1532 1267 1179 Packets Dropped by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- L2 Fwd Low 3270 0 0 0 0 ACL sw processing 12636 0 0 0 0
Step 4: Resolve the issue.
In Step 3, you determined the root cause in this scenario. Remove the ACL which caused the overflow or minimize the ACL to avoid overflow. Also, review the Configuring Network Security with ACLs configuration guideline in order to optimize the ACL configuration and programming in the hardware.
The log Keyword in ACL
The Catalyst 4500 supports logging of packets detail that hit any specific ACL entry, but excessive logging can cause high CPU utilization. Avoid the use of log keywords, except during the traffic discovery stage. During the traffic discovery stage, you identify the traffic that flows through your network for which you have not explicitly configured ACEs. Do not use the log keyword in order to gather statistics. In Cisco IOS Software Release 12.1(13)EW and later, the log messages are rate-limited. If you use log messages in order to count the number of packets that match the ACL, the count is not accurate. Instead, use the show access-list command for accurate statistics. Identification of this root cause is easier because a review of the configuration or log messages can indicate the use of the ACL logging feature.
Step 1: Check for the Cisco IOS process with the show processes cpu command.
Issue the show processes cpu in order to check which Cisco IOS process consumes the CPU. In this command output, you find that the top process is the Cat4k Mgmt LoPri :
Switch#show processes cpu CPU utilization for five seconds: 99%/0%; one minute: 99%; five minutes: 99% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 0 11 0 0.00% 0.00% 0.00% 0 Chunk Manager 2 9716 632814 15 0.00% 0.00% 0.00% 0 Load Meter !--- Output suppressed. 26 175803612 339500656 517 4.12% 4.31% 4.48% 0 Cat4k Mgmt HiPri 27 835809548 339138782 2464 86.81% 89.20% 89.76% 0 Cat4k Mgmt LoPri 28 28668 2058810 13 0.00% 0.00% 0.00% 0 Galios Reschedul
Step 2: Check for the Catalyst 4500-specific process with the show platform health command.
Check the platform-specific process that uses the CPU. Issue the show platform health command. In the output, notice that the K2CpuMan Review process uses most of the CPU cycles. This activity indicates that the CPU is busy as it processes packets destined to it.
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU Lj-poll 1.00 0.01 2 0 100 500 0 0 0 13:45 GalChassisVp-review 3.00 0.20 10 16 100 500 0 0 0 88:44 S2w-JobEventSchedule 10.00 0.57 10 7 100 500 1 0 0 404:22 Stub-JobEventSchedul 10.00 0.00 10 0 100 500 0 0 0 0:00 StatValueMan Update 1.00 0.09 1 0 100 500 0 0 0 91:33 Pim-review 0.10 0.00 1 0 100 500 0 0 0 4:46 Ebm-host-review 1.00 0.00 8 4 100 500 0 0 0 14:01 Ebm-port-review 0.10 0.00 1 0 100 500 0 0 0 0:20 Protocol-aging-revie 0.20 0.00 2 0 100 500 0 0 0 0:01 Acl-Flattener 1.00 0.00 10 5 100 500 0 0 0 0:04 KxAclPathMan create/ 1.00 0.00 10 5 100 500 0 0 0 0:21 KxAclPathMan update 2.00 0.00 10 6 100 500 0 0 0 0:05 KxAclPathMan reprogr 1.00 0.00 2 1 100 500 0 0 0 0:00 TagMan-InformMtegRev 1.00 0.00 5 0 100 500 0 0 0 0:00 TagMan-RecreateMtegR 1.00 0.00 10 14 100 500 0 0 0 0:18 K2CpuMan Review 30.00 91.31 30 92 100 500 128 119 84 13039:02 K2AccelPacketMan: Tx 10.00 2.30 20 0 100 500 2 2 2 1345:30 K2AccelPacketMan: Au 0.10 0.00 0 0 100 500 0 0 0 0:00
Step 3: Check the CPU queue that receives traffic in order to identify the type of CPU-bound traffic.
In order to determine the type of traffic that hits the CPU, issue the show platform cpu packet statistics command. In this command output, you can see that the receipt of packets is due to the ACL log keyword:
Switch#show platform cpu packet statistics !--- Output suppressed. Total packet queues 16 Packets Received by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ----------------- -------------------- --------- --------- --------- ---------- Control 1198701435 35 35 34 35 Host Learning 874391 0 0 0 0 L3 Fwd High 428 0 0 0 0 L3 Fwd Medium 12745 0 0 0 0 L3 Fwd Low 2420401 0 0 0 0 L2 Fwd High 26855 0 0 0 0 L2 Fwd Medium 116587 0 0 0 0 L2 Fwd Low 317829151 53 41 31 31 L3 Rx High 2371 0 0 0 0 L3 Rx Low 32333361 7 1 2 0 RPF Failure 4127 0 0 0 0 ACL fwd (snooping) 107743299 4 4 4 4 ACL log, unreach 1209056404 1987 2125 2139 2089 Packets Dropped by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ----------------- -------------------- --------- --------- --------- ---------- ACL log, unreach 193094788 509 362 437 394
Step 4: Resolve the issue.
In Step 3, you determined the root cause in this scenario. In order to prevent this problem, remove the log keyword from the ACLs. In Cisco IOS Software Release 12.1(13)EW1 and later, the packets are rate-limited so that CPU utilization does not get too high. Use the access list counters as a way to keep track of ACL hits. You can see the access list counters in the show access-list acl_id command output.
Layer 2 forwarding loops
Layer 2 forwarding loops can be caused by poor implementation of Spanning Tree Protocol (STP) and various issues that can affect STP.
Step 1: Check for the Cisco IOS process with the show processes cpu command
This section reviews the commands that an administrator uses in order to narrow down the problem of high CPU utilization. If you issue the show processes cpu command, you can see that two main processes, Cat4k Mgmt LoPri and Spanning Tree , primarily use the CPU. With only this information, you know that the spanning tree process consumes a sizable portion of the CPU cycles.
Switch#show processes cpu CPU utilization for five seconds: 74%/1%; one minute: 73%; five minutes: 50% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 4 198 20 0.00% 0.00% 0.00% 0 Chunk Manager 2 4 290 13 0.00% 0.00% 0.00% 0 Load Meter !--- Output suppressed. 25 488 33 14787 0.00% 0.02% 0.00% 0 Per-minute Jobs 26 90656 223674 405 6.79% 6.90% 7.22% 0 Cat4k Mgmt HiPri 27 158796 59219 2681 32.55% 33.80% 21.43% 0 Cat4k Mgmt LoPri 28 20 1693 11 0.00% 0.00% 0.00% 0 Galios Reschedul 29 0 1 0 0.00% 0.00% 0.00% 0 IOS ACL Helper 30 0 2 0 0.00% 0.00% 0.00% 0 NAM Manager !--- Output suppressed. 41 0 1 0 0.00% 0.00% 0.00% 0 SFF8472 42 0 2 0 0.00% 0.00% 0.00% 0 AAA Dictionary R 43 78564 20723 3791 32.63% 30.03% 17.35% 0 Spanning Tree 44 112 999 112 0.00% 0.00% 0.00% 0 DTP Protocol 45 0 147 0 0.00% 0.00% 0.00% 0 Ethchnl
Step 2: Check for the Catalyst 4500-specific process with the show platform health command
In order to understand which platform-specific process consumes the CPU, issue the show platform health command. From this output, you can see that the K2CpuMan Review process, a job to handle CPU-bound packets, uses up the CPU:
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU !--- Output suppressed. TagMan-RecreateMtegR 1.00 0.00 10 0 100 500 0 0 0 0:00 K2CpuMan Review 30.00 37.62 30 53 100 500 41 33 1 2:12 K2AccelPacketMan: Tx 10.00 4.95 20 0 100 500 5 4 0 0:36 K2AccelPacketMan: Au 0.10 0.00 0 0 100 500 0 0 0 0:00 K2AclMan-taggedFlatA 1.00 0.00 10 0 100 500 0 0 0 0:00
Step 3: Check the CPU queue that receives traffic in order to identify the type of CPU-bound traffic
Issue the show platform cpu packet statistics command in order to check which CPU queue receives the CPU-bound packet. The output in this section shows that the control queue receives a lot of packets. Use the information in Table 1 and the conclusion that you drew in Step 1. You can determine that the packets that the CPU processes and the reason for the high CPU utilization is BPDU processing.
Switch#show platform cpu packet statistics !--- Output suppressed. Total packet queues 16 Packets Received by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Esmp 202760 196 173 128 28 Control 388623 2121 1740 598 16 Packets Dropped by Packet Queue Queue Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Control 17918 0 19 24 3
Step 4: Identify the root cause and fix the issue
Generally, you can complete these steps in order to troubleshoot (depending on the situation, some steps are not be necessary):
-
Identify the loop.
-
Discover the scope of the loop.
-
Break the loop.
-
Fix the cause for the loop.
-
Restore redunancy.
Each of the steps are explained in detail at Troubleshooting Forwarding Loops - Troubleshooting STP on Catalyst Switches Running Cisco IOS System Software.
Step 5: Implement advanced STP features
-
BDPU Guard—Secures STP from unauthorized network devices connected to portfast enabled ports. Refer to Spanning Tree PortFast BPDU Guard Enhancement for more information.
-
Loop Guard—Increases the stability of layer 2 networks. Refer to Spanning-Tree Protocol Enhancements using Loop Guard and BPDU Skew Detection Features for more information.
-
Root Guard—Enforces root bridge placement in the network. Refer to Spanning Tree Protocol Root Guard Enhancement for more information.
-
UDLD—Detects unidirectional links and prevents forwarding loops. Refer to Understanding and Configuring the Unidirectional Link Detection Protocol Feature for more information.
Other Causes of High CPU Utilization
These are some other known causes of high CPU utilization:
-
High CPU utilization in the RkiosPortMan Port Review process
-
High CPU utilization when connected to an IP phone with the use of trunk ports
-
Spike during large ACL programming
The spike in CPU utilization occurs during application or removal of a large ACL from an interface.
Excessive Link Flaps
The Catalyst 4500 exhibits high CPU utilization when one or more of the attached links starts to flap excessively. This situation occurs in Cisco IOS Software releases earlier than Cisco IOS Software Release 12.2(20)EWA.
Step 1: Check for the Cisco IOS process with the show processes cpu command.
Issue the show processes cpu command in order to check which Cisco IOS process consumes the CPU. In this command output, notice that the top process is the Cat4k Mgmt LoPri :
Switch#show processes cpu CPU utilization for five seconds: 96%/0%; one minute: 76%; five minutes: 68% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 0 4 0 0.00% 0.00% 0.00% 0 Chunk Manager 2 9840 463370 21 0.00% 0.00% 0.00% 0 Load Meter 3 0 2 0 0.00% 0.00% 0.00% 0 SNMP Timers !--- Output suppressed. 27 232385144 530644966 437 13.98% 12.65% 12.16% 0 Cat4k Mgmt HiPri 28 564756724 156627753 3605 64.74% 60.71% 54.75% 0 Cat4k Mgmt LoPri 29 9716 1806301 5 0.00% 0.00% 0.00% 0 Galios Reschedul
Step 2: Check for the Catalyst 4500-specific process with the show platform health command.
The output of the show platform health command indicates that the KxAclPathMan create process uses up the CPU. This process is for internal path creation.
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU Lj-poll 1.00 0.03 2 0 100 500 0 0 0 9:49 GalChassisVp-review 3.00 1.11 10 62 100 500 0 0 0 37:39 S2w-JobEventSchedule 10.00 2.85 10 8 100 500 2 2 2 90:00 Stub-JobEventSchedul 10.00 5.27 10 9 100 500 4 4 4 186:2 Pim-review 0.10 0.00 1 0 100 500 0 0 0 2:51 Ebm-host-review 1.00 0.00 8 4 100 500 0 0 0 8:06 Ebm-port-review 0.10 0.00 1 0 100 500 0 0 0 0:14 Protocol-aging-revie 0.20 0.00 2 0 100 500 0 0 0 0:00 Acl-Flattener 1.00 0.00 10 5 100 500 0 0 0 0:00 KxAclPathMan create/ 1.00 69.11 10 5 100 500 42 53 22 715:0 KxAclPathMan update 2.00 0.76 10 6 100 500 0 0 0 86:00 KxAclPathMan reprogr 1.00 0.00 2 1 100 500 0 0 0 0:00 TagMan-InformMtegRev 1.00 0.00 5 0 100 500 0 0 0 0:00 TagMan-RecreateMtegR 1.00 0.00 10 227 100 500 0 0 0 0:00 K2CpuMan Review 30.00 8.05 30 57 100 500 6 5 5 215:0 K2AccelPacketMan: Tx 10.00 6.86 20 0 100 500 5 5 4 78:42
Step 3: Identify the root cause.
Enable logging for link up/down messages. This logging is not enabled by default. The enablement helps you to narrow down the offending links very quickly. Issue the logging event link-status command under all the interfaces. You can use the interface range command in order to conveniently enable on a range of interfaces, as this example shows:
Switch#configure terminal Enter configuration commands, one per line. End with CNTL/Z. Switch(config)#interface range gigabitethernet 5/1 - 48 Switch(config-if-range)#logging event link-status Switch(config--if-range)#end
Switch#show logging !--- Output suppressed. 3w5d: %LINK-3-UPDOWN: Interface GigabitEthernet5/24, changed state to down 3w5d: %LINK-3-UPDOWN: Interface GigabitEthernet5/24, changed state to up 3w5d: %LINK-3-UPDOWN: Interface GigabitEthernet5/24, changed state to down 3w5d: %LINK-3-UPDOWN: Interface GigabitEthernet5/24, changed state to up 3w5d: %LINK-3-UPDOWN: Interface GigabitEthernet5/24, changed state to down 3w5d: %LINK-3-UPDOWN: Interface GigabitEthernet5/24, changed state to up
After you have identified the faulty or flapping interface, shut down the interface in order to resolve the high CPU utilization issue. Cisco IOS Software Release 12.2(20)EWA and later have improved the Catalyst 4500 behavior for this flapping-links condition. Therefore, the impact on the CPU is not as great as before the improvement. Remember that this process is a background process. High CPU utilization because of this issue does not cause adverse effects on the Catalyst 4500 switches.
Spikes in CPU Utilization Due to FIB Consistency Check
The Catalyst 4500 can show momentary spikes in the CPU utilization during a FIB table consistency check. The FIB table is the L3 forwarding table that the CEF process creates. The consistency check maintains consistency between the Cisco IOS Software FIB table and the hardware entries. This consistency ensures that packets are not misrouted. The check occurs every 2 seconds and runs as a low-priority background process. This process is normal behavior and does not interfere with other high-priority processes or packets.
The output of the show platform health command shows that K2Fib Consistency Ch consumes most of the CPU.
Note: The average CPU utilization for this process is insignificant over a minute or an hour, which confirms that the check is a short periodic review. This background process only uses the idle CPU cycles.
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU Lj-poll 1.00 0.02 2 1 100 500 0 0 0 1:09 GalChassisVp-review 3.00 0.29 10 3 100 500 0 0 0 11:15 !--- Output suppressed. K2Fib cam usage revi 2.00 0.00 15 0 100 500 0 0 0 0:00 K2Fib IrmFib Review 2.00 0.00 15 0 100 500 0 0 0 0:00 K2Fib Vrf Default Ro 2.00 0.00 15 0 100 500 0 0 0 0:00 K2Fib AdjRepop Revie 2.00 0.00 15 0 100 500 0 0 0 0:00 K2Fib Vrf Unpunt Rev 2.00 0.01 15 0 100 500 0 0 0 0:23 K2Fib Consistency Ch 1.00 60.40 5 2 100 500 0 0 0 100:23 K2FibAdjMan Stats Re 2.00 0.30 10 4 100 500 0 0 0 6:21 K2FibAdjMan Host Mov 2.00 0.00 10 4 100 500 0 0 0 0:00 K2FibAdjMan Adj Chan 2.00 0.00 10 0 100 500 0 0 0 0:00 K2FibMulticast Signa 2.00 0.01 10 2 100 500 0 0 0 2:04
High CPU Utilization in the K2FibAdjMan Host Move Process
The Catalyst 4500 can display high CPU utilization in the K2FibAdjMan Host Move process. This high utilization appears in the output of the show platform health command. Many MAC addresses frequently expire or are learned on new ports, which causes this high CPU utilization. The workaround for this issue is to increase the MAC address aging time. Or, you can engineer the network in order to avoid the high number of MAC address moves. Cisco IOS Software Release 12.2(18)EW and later have enhanced this process behavior in order to consume less CPU. Refer to Cisco bug ID CSCed15021
( registered customers only) .
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU Lj-poll 1.00 0.02 2 1 100 500 0 0 0 1:09 GalChassisVp-review 3.00 0.29 10 3 100 500 0 0 0 11:15 S2w-JobEventSchedule 10.00 0.32 10 7 100 500 0 0 0 10:14 !--- Output suppressed. K2FibAdjMan Stats Re 2.00 0.30 10 4 100 500 0 0 0 6:21 K2FibAdjMan Host Mov 2.00 18.68 10 4 100 500 25 29 28 2134:39 K2FibAdjMan Adj Chan 2.00 0.00 10 0 100 500 0 0 0 0:00 K2FibMulticast Signa 2.00 0.01 10 2 100 500 0 0 0 2:04 K2FibMulticast Entry 2.00 0.00 10 7 100 500 0 0 0 0:00
High CPU Utilization in the RkiosPortMan Port Review Process
The Catalyst 4500 can display high CPU utilization in the RkiosPortMan Port Review process in the output of the show platform health command in Cisco IOS Software Release 12.2(25)EWA and 12.2(25)EWA1. Cisco bug ID CSCeh08768
( registered customers only) causes the high utilization, which Cisco IOS Software Release 12.2(25)EWA2 resolves. This process is a background process and does not affect the stability of the Catalyst 4500 switches.
Switch#show platform health %CPU %CPU RunTimeMax Priority Average %CPU Total Target Actual Target Actual Fg Bg 5Sec Min Hour CPU Lj-poll 1.00 0.02 2 1 100 500 0 0 0 1:09 GalChassisVp-review 3.00 0.29 10 3 100 500 0 0 0 11:15 S2w-JobEventSchedule 10.00 0.32 10 7 100 500 0 0 0 10:14 !--- Output suppressed. K2 Packet Memory Dia 2.00 0.00 15 8 100 500 0 1 1 45:46 K2 L2 Aging Table Re 2.00 0.12 20 3 100 500 0 0 0 7:22 RkiosPortMan Port Re 2.00 87.92 12 7 100 500 99 99 89 1052:36 Rkios Module State R 4.00 0.02 40 1 100 500 0 0 0 1:28 Rkios Online Diag Re 4.00 0.02 40 0 100 500 0 0 0 1:15
High CPU Utilization When Connected to an IP Phone with the Use of Trunk Ports
If a port is configured for both the voice VLAN option and the access VLAN option, the port acts as a multi-VLAN access port. The advantage is that only those VLANs that are configured for the voice and access VLAN options are trunked.
The VLANs that are trunked to the phone increase the number of STP instances. The switch manages the STP instances. Management of the increase in STP instances also increases the STP CPU utilization.
The trunking of all the VLANs also causes unnecessary broadcast, multicast, and unknown unicast traffic to hit the phone link.
Switch#show processes cpu CPU utilization for five seconds: 69%/0%; one minute: 72%; five minutes: 73% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process 1 4 165 24 0.00% 0.00% 0.00% 0 Chunk Manager 2 29012 739091 39 0.00% 0.00% 0.00% 0 Load Meter 3 67080 13762 4874 0.00% 0.00% 0.00% 0 SpanTree Helper 4 0 1 0 0.00% 0.00% 0.00% 0 Deferred Events 5 0 2 0 0.00% 0.00% 0.00% 0 IpSecMibTopN 6 4980144 570766 8725 0.00% 0.09% 0.11% 0 Check heaps 26 539173952 530982442 1015 13.09% 13.05% 13.20% 0 Cat4k Mgmt HiPri 27 716335120 180543127 3967 17.61% 18.19% 18.41% 0 Cat4k Mgmt LoPri 33 1073728 61623 17424 0.00% 0.03% 0.00% 0 Per-minute Jobs 34 1366717824 231584970 5901 38.99% 38.90% 38.92% 0 Spanning Tree 35 2218424 18349158 120 0.00% 0.03% 0.02% 0 DTP Protocol 36 5160 369525 13 0.00% 0.00% 0.00% 0 Ethchnl 37 271016 2308022 117 0.00% 0.00% 0.00% 0 VLAN Manager 38 958084 3965585 241 0.00% 0.01% 0.01% 0 UDLD 39 1436 51011 28 0.00% 0.00% 0.00% 0 DHCP Snooping 40 780 61658 12 0.00% 0.00% 0.00% 0 Port-Security 41 1355308 12210934 110 0.00% 0.01% 0.00% 0 IP Input
Troubleshooting Tools to Analyze the Traffic Destined to the CPU
As this document has shown, traffic that is destined to the CPU is one of the major causes of high CPU utilization on the Catalyst 4500. The CPU-destined traffic can be either intentional because of the configuration, or unintentional because of misconfiguration or a denial-of-service attack. The CPU has an in-built QoS mechanism to prevent any adverse network effects because of this traffic. However, identify the root cause of CPU-bound traffic and eliminate the traffic if it is undesirable.
Tool 1: Monitor the CPU Traffic with SPAN—Cisco IOS Software Release 12.1(19)EW and Later
The Catalyst 4500 allows for the monitor of the CPU-bound traffic, either ingress or egress, with the use of the standard SPAN function. The destination interface connects to a packet monitor or an administrator laptop that runs packet sniffer software. This tool helps to quickly and accurately analyze the traffic that the CPU processes. The tool provides the ability to monitor individual queues that are bound to the CPU packet engine.
Note: The switching engine has 32 queues for the CPU traffic, and the CPU packet engine has 16 queues.
Switch(config)#monitor session 1 source cpu ? both Monitor received and transmitted traffic queue SPAN source CPU queue rx Monitor received traffic only tx Monitor transmitted traffic only <cr> Switch(config)#monitor session 1 source cpu queue ? <1-32> SPAN source CPU queue numbers acl Input and output ACL [13-20] adj-same-if Packets routed to the incoming interface [7] all All queues [1-32] bridged L2/bridged packets [29-32] control-packet Layer 2 Control Packets [5] mtu-exceeded Output interface MTU exceeded [9] nfl Packets sent to CPU by netflow (unused) [8] routed L3/routed packets [21-28] rpf-failure Multicast RPF Failures [6] span SPAN to CPU (unused) [11] unknown-sa Packets with missing source address [10] Switch(config)#monitor session 1 source cpu queue all rx Switch(config)#monitor session 1 destination interface gigabitethernet 1/3 Switch(config)#end 4w6d: %SYS-5-CONFIG_I: Configured from console by console Switch#show monitor session 1 Session 1 --------- Type : Local Session Source Ports : RX Only : CPU Destination Ports : Gi1/3 Encapsulation : Native Ingress : Disabled Learning : Disabled
If you connect a PC that runs a sniffer program, you can quickly analyze the traffic. In the output that appears in the window in this section, you can see that the cause of the high CPU utilization is an excessive number of STP BPDUs.
Note: STP BPDUs in the CPU sniffer is normal. But if you see more than you expect, you may have exceeded the recommended limits for your Supervisor Engine. See the A High Number of Spanning-Tree Port Instances section of this document for more information.
Tool 2: In-Built CPU Sniffer—Cisco IOS Software Release 12.2(20)EW and Later
The Catalyst 4500 provides an in-built CPU sniffer and decoder to quickly identify the traffic that hits the CPU. You can enable this facility with the debug command, as the example in this section shows. This features implements a circular buffer that can retain 1024 packets at a time. As new packets arrive, they overwrite the older packets. This feature is safe to use when you troubleshoot high CPU utilization issues.
Switch#debug platform packet all receive buffer platform packet debugging is on Switch#show platform cpu packet buffered Total Received Packets Buffered: 36 ------------------------------------- Index 0: 7 days 23:6:32:37214 - RxVlan: 99, RxPort: Gi4/48 Priority: Crucial, Tag: Dot1Q Tag, Event: Control Packet, Flags: 0x40, Size: 68 Eth: Src 00-0F-F7-AC-EE-4F Dst 01-00-0C-CC-CC-CD Type/Len 0x0032 Remaining data: 0: 0xAA 0xAA 0x3 0x0 0x0 0xC 0x1 0xB 0x0 0x0 10: 0x0 0x0 0x0 0x80 0x0 0x0 0x2 0x16 0x63 0x28 20: 0x62 0x0 0x0 0x0 0x0 0x80 0x0 0x0 0x2 0x16 30: 0x63 0x28 0x62 0x80 0xF0 0x0 0x0 0x14 0x0 0x2 40: 0x0 0xF 0x0 0x0 0x0 0x0 0x0 0x2 0x0 0x63 Index 1: 7 days 23:6:33:180863 - RxVlan: 1, RxPort: Gi4/48 Priority: Crucial, Tag: Dot1Q Tag, Event: Control Packet, Flags: 0x40, Size: 68 Eth: Src 00-0F-F7-AC-EE-4F Dst 01-00-0C-CC-CC-CD Type/Len 0x0032 Remaining data: 0: 0xAA 0xAA 0x3 0x0 0x0 0xC 0x1 0xB 0x0 0x0 10: 0x0 0x0 0x0 0x80 0x0 0x0 0x2 0x16 0x63 0x28 20: 0x62 0x0 0x0 0x0 0x0 0x80 0x0 0x0 0x2 0x16 30: 0x63 0x28 0x62 0x80 0xF0 0x0 0x0 0x14 0x0 0x2 40: 0x0 0xF 0x0 0x0 0x0 0x0 0x0 0x2 0x0 0x63
Note: The CPU utilization when you issue a debug command is always almost 100%. It is normal to have high CPU utilization when you issue a debug command.
Tool 3: Identify the Interface That Sends Traffic to the CPU—Cisco IOS Software Release 12.2(20)EW and Later
Catalyst 4500 provides another useful tool to identify the top interfaces that send traffic/packets for CPU processing. This tool helps you quickly identify an errand device that sends a high number of broadcast or other denial-of-service attacks to the CPU. This feature is also safe to use when you troubleshoot high CPU utilization issues.
Switch#debug platform packet all count platform packet debugging is on Switch#show platform cpu packet statistics !--- Output suppressed. Packets Transmitted from CPU per Output Interface Interface Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Gi4/47 1150 1 5 10 0 Gi4/48 50 1 0 0 0 Packets Received at CPU per Input Interface Interface Total 5 sec avg 1 min avg 5 min avg 1 hour avg ---------------------- --------------- --------- --------- --------- ---------- Gi4/47 23130 5 10 50 20 Gi4/48 50 1 0 0 0
Note: The CPU utilization when you issue a debug command is always almost 100%. It is normal to have high CPU utilization when you issue a debug command.
Summary
The Catalyst 4500 switches handle a high rate of IP version 4 (IPv4) packet forwarding in hardware. Some of the features or exceptions can cause the forward of some packets via the CPU process path. The Catalyst 4500 uses a sophisticated QoS mechanism to handle CPU-bound packets. This mechanism ensures reliability and stability of the switches and, at the same time, maximizes the CPU for the software forwarding of packets. Cisco IOS Software Release 12.2(25)EWA2 and later provide additional enhancements for packet/process handling as well as accounting. The Catalyst 4500 also has sufficient commands and powerful tools to aid in the identification of the root cause of high CPU-utilization scenarios. But, in most cases, high CPU utilization on the Catalyst 4500 is not a cause of network instability nor a cause for concern.









Prev
Rss Feed