Introduction
This document describes how to understand errors from devices connected to Multigigabit Ethernet (mGig) ports on Catalyst 9000 Series Switches.
Prerequisites
Requirements
There are no specific requirements for this document.
Components Used
The information in this document is based on these platforms: Catalyst 9000 series switches with mGig capable ports.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Background Information
This document describes why you can encounter frame check sequence (FCS) errors, input errors, or packet loss with devices that connect to Multigigabit Ethernet (mGig) ports on Catalyst 9000 series switches due to interpacket gap (IPG) or interframe gap (IFG) tolerance.
In networking, a pause could be required between network packets or network frames. This time between packets is known as the IPG or IFG. This pause is necessary to allow for receiver clock recovery, which permits the receiver to prepare for another packet. The IFG/IPG standard value for Gigabit Ethernet is 12 bytes. However, from IEEE Standard 802.3, the minimum value for the IFG can be as low as 8 bytes or 64 BT (bit times). For reference, this is documented in 802.3-2000 - IEEE Standard for Information Technology - LAN/MAN - Specific Requirements.
Problem Summary
Multigigabit Ethernet technology is implemented on 10Gig PHYs on Cat9000 architecture. For example, when a connection is established through an mGig port at 1Gbps, if traffic bursts higher than the bandwidth of the interface, the C9600 utilizes port buffers to accommodate that excess traffic and dynamically decreases the IFG/IPG size to avoid any impact and ensure traffic throughput and switch performance. The issue arises when some peer devices are unable to handle the smaller IFG/IPG sizes and no longer recognize legitimate packets and drop this traffic, which results in input errors on their NIC or PHY, such as Cyclic Redundancy Check (CRC) or FCS errors. In certain scenarios the local mGig port (an interface from the mGig linecard C9600-LC-48TX) can also experience the same type of loss in the form of input errors (CRC, FCS) on the interface.
As shown in the table, the structure of an Ethernet packet, which includes the IPG/IFG field:
|
Layer
|
Preamble
|
Start Frame Delimeter
|
Destination MAC
|
Source MAC
|
802.1Q Tag
|
Ethertype (Ethernet II) or length (IEEE 802.3)
|
Payload
|
Frame Check Sequence (32 bit CRC)
|
IPG/IFG
|
|
7 octets
|
1 octet
|
6 octets
|
6 octets
|
4 octets
|
2 octets
|
46-1500 octets
|
4 octets
|
≥ 8 octets
|
|
Layer 2 Ethernet Frame
|
|
|
64-1522 octets
|
|
|
Layer 1 Bits
|
72-1530 octets
|
≥ 8 octets
|
Software Changes
Cisco has made changes to software for mGig capable Catalyst switches to accommodate devices that do not tolerate variance in the IPG/IFG. These changes are documented in various Cisco bug IDs.
|
Platform(s) Affected
|
Bug ID and Resolution Status
|
|
C9200L
|
Fully resolved, see 'Cisco bug ID CSCvy72944' for more information.
|
|
C9300-48UN
|
Fully resolved, see Cisco bug ID CSCvw65866 for more information.
|
|
C9300-48UXM
|
Fully resolved, see 'Cisco bug ID CSCvr95643' for more information.
|
|
C9300-48UXM
|
Fully resolved, see 'Cisco bug ID CSCvr13950 'for more information.
|
|
C9600-LC-48TX
|
Fully resolved, see 'Cisco bug ID CSCvz67689' and/or 'Cisco bug ID CSCwb31319' for more information.
|
Note: Only registered Cisco clients can access the bugs listed in this document.
Workarounds
In some cases, these interoperability issues can be mitigated through hard-coding the mGig port to a lower speed (100Mbps vs 1Gbps), utilization of a different speed (100Mbps or 10Gbps vs 1Gbps), or the affected device is moved to a non-mGig capable port.
Feedback