Introduction

This document describes the Cisco NX-OS configuration limits for Cisco Nexus 9000 Series switches.

The values provided in this guide should not be interpreted as theoretical system limits for Cisco Nexus 9000 Series hardware or Cisco NX-OS software. These limits refer to values that have been validated by Cisco. They can increase over time as more testing and validation is done.

Verified Scalability Limits - Unidimensional

The tables in this section list the verified scalability limits for the Cisco Nexus 9000 Series switches for Cisco NX-OS Release 9.3(2).

These limits are validated with a unidimensional configuration. The values provided in these tables focus on the scalability of one particular feature at a time.

Each number is the absolute maximum currently supported by this Cisco NX-OS release for the corresponding feature. If the hardware is capable of a higher scale, future software releases might increase this verified maximum limit. Results might differ from the values listed in this guide when you try to achieve maximum scalability with multiple features enabled.


Note

  1. If only one number is provided, the verified limit applies to all supported platforms and line cards.

  2. Verified limits are provided only for supported platforms.

  3. If a feature is not supported for a particular platform, the verified limit is not provided.

Table 1. Cisco Nexus 2000 Series Fabric Extenders (FEX) Straight Through Mode Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

Fabric Extenders1 and Fabric Extender server interfaces

Nexus 9300, 9300-EX, and -FX switches

16 and 768

Nexus 9500 switches

32 and 1536

VLANs across all Fabric Extenders

Nexus 9300 and 9500 switches

2,000

Nexus 9300-EX and -FX switches

562

VLANs per Fabric Extender server interface2

Nexus 9300, 9300-EX, 9300-FX, and 9500 switches

75

Port channels

Nexus 9300 switches

256

Nexus 9300-EX and 9300-FX/FX2 switches + FEX

511

Nexus 9500 switches

426

Unique Fabric Extenders per Cisco Nexus 9500 Series supported line card

Nexus 9500 switches

12

1 When FEX configured using "AA" mode, then the maximum number of 6 FEX on NFE base ToR and 16 FEX for LSE base ToR are supported.
2 For FEX HIF port channels, Cisco recommends that you enable STP port type edge using the spanning tree port type edge [trunk] command.
Table 2. FCoE Verified Scalability Limits (Unidimensional)

Feature 3

Supported Platforms

Verified Limits

FLOGI per port

Nexus 93180YC-FX

256

FLOGI per switch

Nexus 93180YC-FX

1,000

Port channels

Nexus 93180YC-FX switches

84

Maximum number of member ports in a port channel

Nexus 93180YC-FX switches

16

VFCs

Nexus 93180YC-FX switches

544

VSANs

Nexus 93180YC-FX switches

32

3 Feature Verified Limit is 5.
4 The number of SAN port channels and virtual FC port channels, together, can be only 8 on the Cisco Nexus 9000 Series switch.

Note
For a list of platforms on which FCoE is supported, see the Cisco Nexus 9000 Series NX-OS FCoE Configuration Guide.
Table 3. FC Verified Scalability Limits (Unidimensional)

Feature 5

Supported Platforms

Verified Limits

FLOGI per port

Nexus 93180YC-FX switches

256

FLOGI per switch

Nexus 93180YC-FX switches

1,000

Port channels

Nexus 93180YC-FX switches

8 6

Maximum number of member ports in a port channel

Nexus 93180YC-FX switches

16

FC ports

Nexus 93180YC-FX switches

48

VSANs

Nexus 93180YC-FX switches

32
5 Feature Verified Limit is 6.
6 The number of SAN port channels and virtual FC port channels, together, can be only 8 on the Cisco Nexus 9000 Series switch.
Table 4. Intelligent Traffic Director Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

Nodes per device group

Nexus 9200, 9300, and 9500 switches

32

Nodes across all device groups

Nexus 9200, 9300, and 9500 switches

256

Device groups per switch

Nexus 9200, 9300, and 9500 switches

48

ITD services per switch

Nexus 9200, 9300, and 9500 switches

64

Ingress interfaces per ITD service

Nexus 9200, 9300, and 9500 switches

8

Virtual IP addresses per ITD service

Nexus 9200, 9300, and 9500 switches

255

Device groups per ITD service

Nexus 9200, 9300, and 9500 switches

48


Note
For a list of platforms on which ITD is supported, see the Cisco Nexus 9000 Series NX-OS Intelligent Traffic Director Configuration Guide.
Table 5. Interfaces Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

DHCP clients per switch

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, 9500 switches and the X9700-EX line cards

10 (IPv4) + 10 (IPv6)

Flex link

Nexus 9300-EX, 9300-FX/FX2, and 9364C switches

One pair consists of one each of active and backup interface. The active and backup interface can be either a physical port or port channel.

IP DHCP relay addresses (helper addresses) per L3 interface

32 (IPv4) + 32 (IPv6) - (Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, 9500 switches and the X9700-EX line cards

32 (IPv4) + 32 (IPv6)

Generic routing encapsulation (GRE) tunnels

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, 9500 switches and the X9700-EX line cards

8

LACP rate fast support during system switchover

Nexus 9500 switches with X9700-EX and X9700-FX line cards

606 ports in total on the Nexus 9516 switches with 16 line cards

MAC address - table limit per port

Nexus 9500 switches with N9K-X9636C-RX line cards

2,000

MAC address - table system, VLAN limit

Nexus 9500 switches with N9K-X9636C-RX line cards

2,000

Port channel links

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, 9500 switches and the X9700-EX, X9636C-R/RX, X9636Q-R line cards)

32

SVIs

Nexus 9200 switches

490

Nexus 9300-EX and 9300-FX/FX2 switches

1000 (with HSRP)

1000 HSRP groups

Nexus 9300 switches

450 (with HSRP)

Nexus 9500 switches and the X9700-EX line cards

490 (with HSRP), 1500 (without HSRP)

X9636C-R/RX and X9636Q-R line cards

3,967

Selective QinQ with Multiprovider tag

Nexus 93180YC-EX, 93180YC-FX, and 9336C-FX2 switches

4,000 mappings, 10 provider VLANs;System wide: 48,000 mappings, 512 Provider VLANs

SVI Unnumbered

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX line cards

Primary (50); Secondary (450), 1 primary SVI can have a maximum of 50 secondary SVIs

vPCs

Nexus 9200, Nexus 9300, and Nexus 9300-FX/FX2 switches

80

Nexus 9300-EX switches

48

Nexus 9500 switches and the X9700-EX line cards

300

X9636C-R/RX and X9636Q-R line cards

255

Static network address translation (NAT)

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9300-FX2 switches

1,023

Dynamic network address translation (NAT)

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9300-FX2 switches

1023

Static twice network address translation (NAT)

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9300-FX2 switches

768

Dynamic twice network address translation (NAT)

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9300-FX2 switches

1,023

Table 6. Label Switching Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

Forwarding Equivalence Classes (FECs) (Node /Prefix /Adj / Binding SID)

Nexus 9200 switches

MPLS Heavy Template: 512; Default: 128

Nexus 9300 and 9500 switches

128

Nexus 9300-EX, and 9300-FX switches and the X9700-EX/FX line cards

MPLS Heavy Template: 4096; Default: 1024

X9636C-R/RX and X9636Q-R line cards

1,000

Equal-cost multipaths (ECMPs)

Nexus 9200, 9300, and 9500 switches

16

Nexus 9300-EX and 9300-FX switches, and the X9700-EX/FX line cards

32

X9636C-R/RX and X9636Q-R line cards

8-way

Equal-cost multipaths Groups (ECMPs)

Nexus 9300-EX and 9300-FX switches, and the X9700-EX/FX line cards

MPLS Heavy Template: 4096; Default: 1024

N9K-X9636C-RX line cards

24K ECMP Groups 2 paths per ECMP

FECs * ECMPs

Nexus 9200, 9300, and 9500 switches

1,000

X9636C-R/RX and X9636Q-R line cards

8,000

Flex counters for segment-routing in ingress direction

Nexus 9300 and 9500 switches

4,000 (includes ingress and egress)

Nexus 9200 switches

4,000 (includes ingress and egress) (MPLS Heavy Template)

Nexus 9300-EX/FX switches and the X9700-EX/FX line cards)

Total ingress label stats: 4000; VRF ingress label stats: 1,000; (MPLS Heavy Template)

Flex counters for segment-routing in Egress direction

Nexus 9300 and 9500 switches

4,000 (includes ingress and egress)

Nexus 2000 switches

4000 (includes ingress and egress)

(MPLS Heavy Template)

Nexus 9300-EX/FX switches and the X9700-EX/FX line cards

Total ingress label stats: 48K

(MPLS Heavy Template)

Egress Peer Engineering

Nexus 9200, 9300, 9300-EX, and 9300-FX switches and the X9700-EX/FX line cards

64

IAS option B labels

X9636C-R/RX and X9636Q-R line cards

450,000

Label-switched paths (LSPs) for label stack imposition7

Nexus 9300 and 9500 switches

128 (with 4-way ECMP and 3 label stack push)

Nexus 9200 switches

256 (with 32-way ECMP and 5 label stack push)

Nexus 9300-EX/FX switches and the X9700-EX/FX line cards

256 (with 32-way ECMP and 5 label stack push)

Layer 3 VPN routes

X9636C-R/RX and X9636Q-R line cards

450,000

Layer 3 EVPN Labels

Nexus 9300 and 9500 switches

128

Nexus 9200 switches

128 (With MPLS Heavy Template)

Nexus 9300-EX/FX switches and the X9700-EX/FX line cards

1,000 (With MPLS Heavy Template))

LDP session

N9K-X9636C-RX line card

64+ sessions

Node Sid/Prefix SID

Nexus 9200, 9300, and 9500 switches

128

Nexus 9300-EX/FX switches and the X9700-EX/FX line cards

4,000

Adjacency SID

Nexus 9200, 9300, and 9500 switches

128

Nexus 9300-EX/FX switches and the X9700-EX/FX line cards

600

Binding SID

Nexus 9200, 9300, and 9500 switches

50

Nexus 9300-EX/FX switches and the X9700-EX/FX line cards

1,000

Private VLANs (PVLANs)

Primary VLANs 8

Nexus 9300, 9300-FX2, and 9500 switches and the X9700-EX/FX line cards

16

Nexus 9300-EX and 9300-FX switches

400

Secondary VLANs 9

Nexus 9300, 9300-FX2, and 9500 switches and the X9700-EX/FX line cards

20

Nexus 9300-EX and 9300-FX switches

400

Ports in Community host mode

Nexus 9300, Nexus 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

40

Ports in isolated host mode

Nexus 9500 switches

20

Nexus 9300, Nexus 9300-EX, and 9300-FX/FX2 switches and the X9700-EX/FX line cards

40)

Ports in isolated trunk host mode

Nexus 9500 switches

22

Nexus 9300, Nexus 9300-EX, 9300-FX/FX2 switches and the X9700-EX/FX line cards

40

Ports in promiscuous mode

Nexus 9300-EX and 9300-FX switches

10

Nexus 9500 switches

48

Nexus 9300 and 9300-FX2 switches and the X9700-EX/FX line cards

5

Ports in promiscuous trunk mode

Nexus 9300-EX and 9300-FX switches

10

Nexus 9500 switches

80

Nexus 9300 and 9300-FX2 switches and the X9700-EX/FX line cards

5

PVLANs allowed on a PVLAN port 10

Nexus 9300-EX and 9300-FX switches

400

Nexus 9300, 9300-FX2, and 9500 switches and the X9700-EX/FX line cards

16
7 For Cisco Nexus 9300 and 9500 Series switches, LSPs *ECMP* label stack push cannot exceed 1500.
8 The 400 PVLAN mapping scale per PVLAN port is only applicable when port is configured as promiscuous trunk port
9 The 400 PVLAN mapping scale per PVLAN port is only applicable when port is configured as promiscuous trunk port
10 The 400 PVLAN mapping scale per PVLAN port is only applicable when port is configured as promiscuous trunk port

Note
For network scalability, Cisco recommends using a hierarchical routing design with multi-hop BGP for advertising the attached prefixes from a top-of-rack (ToR) or border leaf switch.
Table 7. Layer 2 Switching Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

MAC addresses

Nexus 9200 and 9300-EX, 9300-FX/FX2 switches and the X9700-EX/FX line cards

92,000

Nexus 9300 and 9500 switches

90,000

X9636C-R/RX and X9636Q-R line cards

192,000

N9K-C9264PQ, N9K-C9272Q, N9K-C9236C, N9K-C92300YC, N9K-C92304QC, N9K-C92300YC, and 9300-EX switches

200,000 11

MST instances

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2 switches and the X9700-EX/FX, X9636C-R/RX, and X9636Q-R line cards

64

MST PV count with single instances 0

Nexus 9200, 9300-FX, and 9300-FX switches

190,000

MST virtual ports with more than 1 MST instance

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches

48,000

Nexus 9500 switches and the X9700-EX/FX line cards

85,000

X9636C-R/RX and X9636Q-R line cards)

236,000

RPVST virtual ports

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches

48,000

Nexus 9500 switches and the X9700-EX/FX line cards

22,000

X9636C-R/RX and X9636Q-R line cards

13,750

VLANs

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX, X9636C-R/RX, and X9636Q-R line cards

3,967 (the remaining 127 VLANs are reserved)

Nexus 92348GC-X switches

4.096

VLANs in RPVST mode

Nexus 9200 and 9300-EX/FX/FX2/FX3/GX switches

3,967

Nexus 9300 and 9500 switches

500

3,967 - (X9700-EX/FX line cards)

3,96712

X9636C-R/RX and X9636Q-R line cards

250

Total number of VLANs × ports with switchport isolated (3967 VLANs x 48 ports)

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches, and the X9700-EX/FX line cards

190,000

Private VLANs (PVLANs)

Primary VLANs

Nexus 9300, 9300-EX/FX/FX2, 9500 switches and the Nexus 9700-EX line cards

16

Secondary VLANs

Nexus 9300, 9300-EX/FX/FX2, 9500 switches and the Nexus 9700-EX line cards

20

Ports in Community host mode

Nexus 9300, 9300-EX/FX/FX2, 9500 switches and the Nexus 9700-EX line cards

40

Ports in isolated host mode

Nexus 9500 switches

20

Nexus 9300, 9300-EX/FX/FX2 switches and the Nexus 9700-EX line cards

40

Ports in isolated trunk host mode

Nexus 9500 switches

22

Nexus 9300, 9300-EX/FX/FX2 switches and the Nexus 9700-EX line cards

40

Ports in promiscuous mode

Nexus 9500 switches

48

Nexus 9300, 9300-EX/FX/FX2 switches and the Nexus 9700-EX line cards

5

Ports in promiscuous trunk mode

Nexus 9500 switches

80

Nexus 9300, 9300-EX/FX/FX2 switches and the Nexus 9700-EX line cards

5

PVLANs allowed on a PVLAN port

Nexus 9300, 9300-EX/FX/FX2, 9500 switches and the Nexus 9700-EX line cards

16
11 Layer 2 unidimensional scale only. SVI, Layer 3 interface, and VXLAN VLANs are not supported. 200K MAC is enabled only when " system routing template-l2-heavy" is configured and the system is reloaded.
12 On EOR, support is for 12000 PV count with 3967 vlans and RPVST with default timers. If 22000 PV count is needed with 3968 vlans and RPVST, recommended hello timer value is 4 or higher. It is also recommended to tune forward delay and max age accordingly

Note

  • The number of supported VLANs per vPC should be within the MST or RPVST virtual port count specified in this table, depending on the topology.

  • The number of supported STP VLAN port instances for Fabric Extender host interface ports should be less than 13,000.

Table 8. Multicast Routing Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

IPv4 multicast routes

Note

 
The limits are for a combination of IPv4 and IPv6 multicast routes. Layer 2 multicast routes are a part of the total 120K limits. For example 110,000 IPv4 + 2,000 IPv6 multicast routes.

Nexus 9200 switches

8,000 (Layer 2 + Layer 3)

Note

 
Cisco Nexus 9200 switches do not support the system routing template -Ipm - heavy mode for IPv4 multicast routes. Make sure to reset the LPM maximum limit to 0.

Nexus 9300 switches

8,192 (Layer 2 + Layer 3)

Nexus 9300-EX switches

8,000 (Layer 2 + Layer 3); 32,000 (layer 2 + Layer 3 with system routing template -multicast -heavy mode)

Nexus 9300-FX/FX2 switches

8,000 (Layer 2 + Layer 3); 32,000 (layer 2 + Layer 3 with system routing template -multicast -heavy mode)

Nexus 9500 switches

32,000 (Layer 2 + Layer 3)

X9700-EX line cards

8,000 (Layer 2 + Layer 3); 32,000 (layer 2 + Layer 3 with system routing template -multicast -heavy mode); 8000 (with system routing template -lpm - heavy mode)

Nexus 9300 switches and the X9700-FX line cards

8,000 (Layer 2 + Layer 3); 32,000 (layer 2 + Layer 3 with system routing template -multicast -heavy mode); 128,000 (with system routing template -multicast - ext - heavy mode) 13

X9636C-R/RX and X9636Q-R line cards

32, 000 (Layer 3)

IPv6 multicast routes

Nexus 9300-FX2 and 9364C switches

2048 (Layer 3 with system routing template -multicast -heavy mode)

Outgoing interfaces (OIFs)

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

40 (SVI + physical Layer 3) or 256 (physical Layer 3)

X9636C-R/RX and X9636Q-R line cards

16 OIFs for 32K mroutes or 287 OIFs for 1000 mroutes

IGMP snooping groups

Nexus 9200, 9300, 9300-EX, and 9500 switches and the X9700-EX line cards

8,000

Nexus 9600-R and 9600-RX line cards

8000

Nexus 9300-FX/FX2 switches and the X9700-FX line cards

16,000

PIM neighbors

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches

250

Nexus 9500 switches and the X9700-EX/FX,X9636C-R/RX and X9636Q-R line cards

500
13 All line cards must have the FX type.

Note

  • The IPv4 multicast routes and the IPv4/IPv6 host routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.

  • High availability (graceful restart and stateful switchover) is not supported when unicast or multicast aggressive timers are configured at any scale.

Table 9. IP Fabric for Media Solution Verified Scalability Limits (Unidimensional)

Feature

Verified Limits

Nodes

29 ( 2spine and 27 leafs)

Routes

32,000

Host Policy

Sender

8,000

Receiver

8,000

PIM

512

FlowPolicy

2,000

ASM group-range

20

NBM Static Flows

Per switch maximum (receiver leaf where the static OIF will be programmed) mroutes

1,500

Per fabric maximum mroutes

8,000

VRFs

16

RTP Flow Monitoring with ACL

ACL

128 IPv4 ACL entries or 64 IPv6 ACL entries (total 128 TCAM spaces)

Note

 
With combined IPv4 and IPv6 ACL entries, the scale limit cannot exceed 128 TCAM space.

Note
For a list of platforms on which IP Fabric for Media is supported, see Cisco Nexus 9000 Series NX-OS IP Fabric for Media Solution Guide.
Table 10. Programmability Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

OpenFlow ports

Nexus 9300 switches

96

OpenFlow Layer 2 flows

Nexus 9300 switches

32,000

OpenFlow Layer 3 flows

Nexus 9300 switches

3,000

OpenFlow IPv6 Layer 3 flows

Nexus 9300 switches

1,500

Table 11. Security Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

Egress ACLs

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches and Nexus 9500 platform switches with the -R and -RX line cards

20,000

System ACLs

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches and Nexus 9500 platform switches with the -R and -RX line cards

4000 TCAM entries in internal TCAM

64000 TCAM entries in external TCAM

ACLs

Nexus 92348GC-X switches

Ingress - 3072 IPv4, 1792 IPv6

DHCP snooping bindings

(Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX line cards

2,048

IPv4 ingress access control entries (ACEs)

X9636C-R/RX and X9636Q-R line cards

  • RACL on LC N9K-X9636C-RX: 100,000

  • PACL on LC N9K-X9636C-RX: 12,000

  • RACL-2048, PACL-1024 (without TCAM Carving) IPv4 52640 ACEs per system

  • PACL IPv4: 1024 TCAM entries in internal TCAM

  • PACL MAC: 2048 TCAM entries in internal TCAM

  • RACL IPv4: 2048 TCAM entries in internal TCAM

IPv6 ingress access control entries (ACEs)

For X9636C-R/RX and X9636Q-R line cards

  • RACL-1024, PACL-1024 (without TCAM Carving) IPv6 25200 ACEs per system

  • PACL IPv6: 1024 TCAM entries in internal TCAM

  • RACL IPv6: 1024 TCAM entries in internal TCAM

IPv4 ingress TCAM entries

Nexus 9200, 9300-EX, and 9300-FX switches and the X9700-EX line cards

3,582 (per slice of the forwarding engine)

Nexus 9300-FX2 switches

3,582

(Nexus 9300 and 9500 switches

3,072 (per network forwarding engine)

IPv4 egress TCAM entries

Nexus 9200, 9300-EX, 9300-FX/FX2 switches and the X9700-EX line cards

1,792 (per slice of the forwarding engine)

Nexus 9300 and 9500 switches

768 (per network forwarding engine)

IPv6 ingress TCAM entries

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches and the X9700-EX line cards

1,792 (per slice of the forwarding engine)

Nexus 9300 and 9500 switches

1,536 (per network forwarding engine)

IPv6 egress TCAM entries

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches and the X9700-EX line cards

896 (per slice of the forwarding engine)

Nexus 9300 and 9500 switches

256 (per network forwarding engine)


Note

  • The TCAM entries scalability limits also apply to policy-based TCAM entries (PBACLs).

  • Only 62 unique ACLs can be configured. Each ACL takes one label. If the same ACL is configured on multiple interfaces, the same label is shared. If each ACL has unique entries, the ACL labels are not shared, and the label limit is 62.

Table 12. System Management Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

MPLS Stripping

Labels

Nexus 9300 and 9500 switches

12,000

Ingress interfaces

Nexus 9200 and 9300 switches

48

Nexus 9500 switches

400

Egress interfaces

Nexus 9200 and 9300 switches

16

Nexus 9500 switches

64

PTP

PTP ports

Nexus 9200, 9300, 9300-EX, 9300-FX, 9300-FX2, and 9500 switches

64

N9K-C92160YC-X

44

X9700-EX line cards

1305

9508-R line cards

64 per line card

300 per chassis

Note

 
PTP Offload is supported on 9508-R line cards.

PTP offload

14

N9K-X9564PX line cards

32 PTP sessions per line card.

N9K-X9636PQ line cards

24 PTP session per line card.

sFlow

sFlow ports

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches

64

Nexus 9500 switches and the X9700-EX line cards

256

SPAN and ERSPAN

Configurable SPAN or ERSPAN sessions

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches and the X9636C-R/RX and X9636Q-R line cards

32

Active SPAN or ERSPAN sessions15

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX line cards

4

Nexus 9500 switches and the X9700-EX line cards

4 to 32, based on the number of line cards and the session configuration

X9636C-R/RX and X9636Q-R line cards

32

Active localized SPAN or ERSPAN sessions per line card16

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX line cards

4

X9636C-R/RX and X9636Q-R line card)

32 sessions across ports on single line card

Active localized SPAN or ERSPAN session (Rx and Tx, Rx, or Tx)

X9636C-R/RX and X9636Q-R line cards

32 sessions, 128 sources and 1 destination

Source interfaces per SPAN or ERSPAN session (Rx and Tx, Rx, or Tx)

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX line cards

48

Destination interfaces per SPAN session

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX, X9636C-R/RX, and X9636Q-R line cards

1 (physical/PO interface)

Source VLANs per SPAN or ERSPAN session

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX, X9636C-R/RX, and X9636Q-R line cards

32

Tap Aggregation

Redirect interfaces in the redirect port list

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches

12

Redirect port lists (or fan outs) per system

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches

12

Nexus 9300 and 9500 switches

100

NetFlow

Flow monitors

Nexus 9300-EX switches and Nexus 9500 switches with 9700-EX and FM-E fabric line cards

2 exporters and 2 flow monitors per type (2 IPv4 flow monitors and 2 IPv6 flow monitors)

Nexus 9300-FX/FX2 switches

2 exporters and 32 flow monitors per type (32 Layer 2 flow monitors, 32 IPv4 flow monitors, and 32 IPv6 flow monitors)

Layer 3 interfaces (Layer 3 ports, port channels, and SVIs) to which IPv4 flow monitors can be applied

Nexus 9300-EX switches

Number of Layer 3 interfaces (Layer 3 ports, port channels, and SVIs) to which IPv4 flow monitors can be applied. You can use the show interface hardware-mappings command to check if the interface belongs to ASIC slice 0 or slice 1.

Layer 3 interfaces (Layer 3 ports, port channels, and SVIs) to which IPv6 flow monitors can be applied

Nexus 9300-EX switches

252 (with members on just one ASIC slice) or 126 (with members on both ASIC slices). You can use the show interface hardware-mappings command to check if the interface belongs to ASIC slice 0 or slice 1.

14 An EPLD upgrade is necessary before you use PTP offload.
15 A single forwarding engine instance supports four SPAN or ERSPAN sessions. For Cisco Nexus 9300 Series switches, if the first three sessions have bidirectional sources, the fourth session has hardware resources only for Rx sources. This limitation might also apply to Cisco Nexus 9500 Series switches, depending on the SPAN or ERSPAN source's forwarding engine instance mappings.
16 The number of SPAN or ERSPAN sessions per line card reduces to two if the same interface is configured as the bidirectional source in more than one session.

Note
PTP is supported for all Cisco Nexus 9000 Series hardware except for the 100G 9408PC line card and the 100G M4PC generic expansion module (GEM).
Table 13. Unicast Routing Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

Unicast Routing

BFD sessions (echo mode)

Nexus 9200, 9300, 9364C, 9300-EX, and 9300-FX/FX2 switches

128

Note

 
Cisco Nexus 9300 (EX, FX, FX2, and 9364C) platform switches support up to 512 BFD sessions, when the BFD intervals are relaxed to 300 ms.

Nexus 9500 switches

512

X9700-EX /FX line cards

51217

X9636C-R/RX and X9636Q-R line cards

288

BGP neighbors

Nexus 9200 switches

512 (IPv4), 512 (IPv6), or 256 (IPv4 + IPv6)

Nexus 92348GC-X switches

141

Nexus 9300, 9364C, 9300-EX, and 9300-FX/FX2 switches

1024 (9346C, 9300-EX, 9300-FX/FX2)

Nexus 9500 switches and the X9700-EX/FX line cards

2,000

X9636C-R/RX and X9636Q-R line cards

960

EIGRP routes

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX /FX line cards

20, 000

EIGRP neighbors

Nexus 9200, 9300, 9364C, 9300-EX, and 9300-FX/FX2 switches

256

Nexus 9500 switches and the X9700-EX/FX line cards

512

HSRP groups

Nexus 9200, 9300, 9364C, 9300-EX, and 9500 switches and the X9700-EX /FX, X9636C-R/RX, and X9636Q-R line cards

490

Nexus 9300-EX and FX2 switches

1000 18

IPv4 ARP

Nexus 9200 and 9364C switches

32,000

Nexus 9300 and 9500 switches and the X9636C-R/RX and X9636Q-R line cards

48,000

Nexus 9300-EX and 9300-FX switches

Nexus 9300-EX and 9300-FX switches

Nexus 9300-FX2 switches

64,000 / 32,000 (with out/with urpf enabled) (in default routing mode, Hash Table: Shared between IPv6 ND, IPv4 ARP)

X9700-EX /FX line cards

X9700-EX /FX line cards

IPv4 host routes 19

Nexus 9200 switches

96,000 (hash table and there will be more collisions after 80%)

Nexus 9300 and 9500 switches

Default System Routing Mode: 208,000 (hash table and there will be more collisions after 80%)

Nexus 9364C switches

96,000

Nexus 9300-EX switches

458,000 (default); 786,000/720,000 (with
system routing
template -lpm - heavy
mode)

Nexus 9300-FX switches

471,000 / 419,000 (with out/with urpf enabled) (default); 786,000/734,000 (with
system routing
template -lpm -heavy
mode)

Nexus 9300-FX2 switches

471,000 (default); 786,000/734,000 (with out/with urpf enabled) (with
system routing
template -lpm -heavy
mode)

X9700-EX line card

589,000 (default);
786,000 (with
system routing
template -lpm -heavy
mode)

X9636C-R

750,000

X9636C-RX

1,000,000

IPv6 host routes 20

Nexus 9200 switches

48,000 (hash table and there will be more collisions after 80%)

Nexus 9300 and 9500 switches

Default System Routing Mode: 104,000 (hash table and there will be more collisions after 80%); ALPM Routing Mode: 16000 with host Routes Programmed in the LPM Table.

Nexus 9364C switches

48,000

Nexus 9300-EX switches

24000 / 16,000 (with out/with urpf enabled)

Nexus 9300-FX switches

265,000 / 235,000 (with out/with urpf enabled) (default) 442,000 / 412,000 (with out/with urpf enabled) (with system routing template -lpm - heavy mode)

Nexus 9300-FX2 switches

265,000 (default) 442,000 / 412,000 (with out/with urpf enabled) (with system routing template -lpm - heavy mode)

X9700-EX/FX line cards

32,000 (FM-E), 235000 (FM-E2)

X9636C-RX

256,000

IPv6 ND

Nexus 9200, 9364C, 9300-EX, and 9300-FX2 switches

32,000 (default), 16,000 (lpm heavy)

Nexus 9300 and 9500 switches

48,000

Nexus 9300-FX

32,000 (in default routing mode, Hash Table: Shared between IPv6 ND, IPv4 ARP)

X9700-EX/FX, X9636C-R/RX, and X9636Q-R line cards

32,000

IPv4 unicast routes (LPM)*

Nexus 9200 switches

  • Default values: 6000 (IPv4), 1900 (IPv6), and 2000 (multicast)

  • With hardware profile multicast max-limit lpm-entries 0 configured: 8000 (IPv4), 1900 (IPv6), and 0 (multicast)

  • With hardware profile ipv6 lpm-entries maximum 0 configured: 14,000 (IPv4), 0 (IPv6), and 2000 (multicast)

  • With hardware profile ipv6 lpm-entries maximum 4096 and hardware profile multicast max - limit lpm - entries 0 configured: 0 (IPv4), 4096 (IPv6), and 0 (multicast)

  • When you allocate the entire table for IPv4 or IPv6 LPM unicast routes, the other address family cannot be used

Nexus 9300 switches

12,000 (default system routing mode); 128,000 (ALPM routing mode)

Nexus 9364C switches

  • Default values: 8000 (IPv4), 1900 (IPv6), and 2000 (multicast)

  • With hardware profile multicast max-limit lpm-entries 0 configured: 10000 (IPv4), 1900 (IPv6), and 0 (multicast)

  • With hardware profile ipv6 lpm-entries maximum 0 configured: 14,000 (IPv4), 0 (IPv6), and 2000 (multicast)

  • With hardware profile ipv6 lpm-entries maximum 4096 and hardware profile multicast max - limit lpm - entries 0 configured: 4000 (IPv4), 4096 (IPv6), and 0 (multicast)

  • When you allocate the entire table for IPv4 or IPv6 LPM unicast routes, the other address family cannot be used.

Nexus 9300-EX switches

458,000 (default)

Nexus 9300-FX switches

471,000 / 419,000 (with out / with urpf enabled) (default)

Nexus 9300-FX2 switches

471,000 (default)

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2 switches and the X9636Q-R line cards

128,000 (default system routing mode); 16,000 (max-host routing mode)

Nexus 9500 switches

128,000 with no IPv6 routes (64-bit ALPM routing mode)

X9700-EX /FX line cards

589,000 (default)

N9K-X9636Q-R, N9K-9636C-R, and N9K-93180YC-R line cards

192,000 (Default system routing template)

X9636C-RX

1,000,000 (Default system routing template)

IPv6 unicast routes (LPM)*

Nexus 9200 switches

  • Default values: 6000 (IPv4), 1900 (IPv6), and 2000 (multicast)

  • With hardware profile multicast max-limit lpm-entries 0 configured: 8000 (IPv4), 1900 (IPv6), and 0 (multicast)

  • With hardware profile ipv6 lpm-entries maximum 0 configured: 14,000 (IPv4), 0 (IPv6), and 2000 (multicast)

  • With hardware profile ipv6 lpm-entries maximum 4096 and hardware profile multicast max - limit lpm - entries 0 configured: 0 (IPv4), 4096 (IPv6), and 0 (multicast)

  • When you allocate the entire table for IPv4 or IPv6 LPM unicast routes, the other address family cannot be used.

Nexus 9300 switches

7000 (6000 routes < /64, 1000 routes > /64) (default system routing mode); 20,000 (ALPM routing mode)

Nexus 9364C switches

  • Default values: 8000 (IPv4), 1900 (IPv6), and 2000 (multicast)

  • With hardware profile multicast max-limit lpm-entries 0 configured: 10000 (IPv4), 1900 (IPv6), and 0 (multicast)

  • With hardware profile ipv6 lpm-entries maximum 0 configured: 14,000 (IPv4), 0 (IPv6), and 2000 (multicast)

  • With hardware profile ipv6 lpm-entries maximum 4096 and hardware profile multicast max - limit lpm - entries 0 configured: 4000 (IPv4), 4096 (IPv6), and 0 (multicast)

  • When you allocate the entire table for IPv4 or IPv6 LPM unicast routes, the other address family cannot be used

Nexus 9300-EX switches

206,000 (/64 prefix length); 1900 (non /64 prefix length)

Nexus 9300-FX switches

265,000 / 235,000 (with out/with urpf enabled) (default)

Nexus 9300-FX2 switches

265,000 (default)

Nexus 9500 switches

20,000 (default system routing mode)

4000 (max-host routing mode)

80,000 with no IPv4 routes (64-bit ALPM routing mode)

X9700-EX /FX line cards

176,000 (/64 prefix length); 3900 (non /64 prefix length) ( FM-E ) and 235,000 ( any prefix length) ( FM-E2)

X9636C-R, X9636Q-R, and X96136YC-R line cards

62,000 (Default system routing template)

X9636C-RX

256,000 (Default system routing template)

IPv4 and IPv6 unicast routes (LPM) in 64-bit ALPM routing mode

Nexus 9500 switches

128,000 (IPv4) and 80,000 (IPv6)

IPv4 host routes (LPM heavy mode)

Nexus 9236C, 9272Q, and 92304QC switches

262,000

Nexus 92160YC-X switches

650,000

Nexus 9364C switches

262,000

Nexus 9300-EX switches

786,000 / 720,000 (with out/with urpf enabled)

Nexus 9300-FX switches

786000 / 734000 (with out/with urpf enabled)

Nexus 9300-FX2 switches

786,000 / 734,000 (with out/with urpf enabled)

X9700-EX /FX line cards

786,000

IPv6 host routes (LPM heavy mode)

Nexus 9200 switches

16,000

Nexus 9364 switches

131,000

Nexus 9300-EX switches

24000 / 16,000 (with out/with urpf enabled) (protocol learned host)

Nexus 9300-FX/FX2 switches

442,000 / 412,000 (with out/with urpf enabled) (protocol learned host)

X9700-EX/FX line cards

32,000 (shared between IPv6 ND and protocol learned host) ( FM-E ) and 235,000 ( FM-E2)

IPv4 LPM routes (LPM heavy mode)

Nexus 9236C, 9272Q, and 92304QC switches

262,000

Nexus 92160YC-X switches

650,000

Nexus 9364C switches

262,000

Nexus 9300-EX switches

786,000 / 720,000 (with out/with urpf enabled)

Nexus 9300-FX switches

786,000 / 734,000 (with out/with urpf enabled)

Nexus 9300-FX2 switches

786,000 / 734,000 (with out/with urpf enabled)

X9700-EX/FX line cards

786,000

IPv6 LPM routes (LPM heavy mode)

Nexus 9236C, 9272Q, and 92304QC switches

131,000 (/64 prefix length); 1900 (non /64 LPM scale)

Nexus 92160YC-X switches

294,000 (/64 prefix length); 1900 (non /64 LPM scale)

Nexus 9364C switches

131,000

Nexus 9300-EX switches

353,000 / 324,000 (with out/with urpf enabled) (/64 prefix length); 1900 (non /64 prefix length)

Nexus 9300-FX/FX2 switches

442,000 / 412,000 (with out/with urpf enabled)

X9700-EX /FX line cards

235,000 (/64 prefix length); 3900 (non /64 prefix length) ( FM-E) and 235,000 (any prefix length) (FM-E2)

IPv4 host routes (dual-host mode)

Nexus 9200 and 9364C switches

163,000

Nexus 9300-EX and 9300-FX/FX2 switches

262,000

IPv6 host routes (dual-host mode)

Nexus 9200 and 9364C switches

81,000

Nexus 9300-EX and 9300-FX/FX2 switches

131,000

IPv4 LPM routes (dual-host mode)

Nexus 9200 and 9300-EX switches

6000

Nexus 9364C switches

8,000

Nexus 9300-FX/FX2 switches

7,000

IPv6 LPM routes (dual-host mode)

Nexus 9200, 9300, 9364C, 9300-EX, and 9300-FX/FX2 switches

1,900

IPv4 ARP (dual-host mode)

Nexus 9200, 9300, 9364C, 9300-EX, and 9300-FX/FX2 switches

64,000

IPv6 ND (dual-host mode)

Nexus 9200, 9300, 9364C, 9300-EX, and 9300-FX/FX2 switches

64,000

IPv4 host routes (internet-peering mode)

Nexus 9300-EX and 9300-FX/FX2 switches

1 Million (protocol learned host)

IPv6 host routes (internet-peering mode)

Nexus 9300-EX switches and the X9700-EX /FX line cards

16,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP, and protocol learned IPv6 host)

Nexus 9300-FX/FX2 switches

500,000

Routes (internet-peering mode)

Nexus 9500 switches with the N9K-X9636Q-R and N9K-9636C-R line cards

1 Million21

IPv4 routes (internet-peering mode)

Nexus 9500 switches with the N9K-X9636Q-R and N9K-9636C-R line cards

85200022

IPv6 routes (internet-peering mode)

Nexus 9500 switches with the N9K-X9636Q-R and N9K-9636C-R line cards

17500023

Routes (internet-peering mode)

Nexus 9500 switches with the N9K-96136YC-R line cards

852000

IPv4 routes (internet-peering mode)

Nexus 9500 switches with the N9K-96136YC-R line cards

781000

IPv6 routes (internet-peering mode)

Nexus 9500 switches with the N9K-96136YC-R line cards

71000

IPv4 ARP (internet peering mode)

Nexus 9300-EX switches and the X9700-EX/FX line cards

32,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP, and protocol learned IPv6 host)

Nexus 9300-FX/FX2 switches

32,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP)

IPv6 ND (internet-peering mode)

Nexus 9300-EX switches and the X9700-EX/FX line cards

16,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP, and protocol learned IPv6 host)

Nexus 9300-FX/FX2 switches

16,000 (Hash Table: Shared between IPv6 ND, IPv4 ARP)

IS-ISv4 adjacencies (either L1, L2, or sum of L1 and L2 with default timers)

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX /FX line cards

255

IS-ISv4 BFD sessions (with default timers)

Nexus 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

255

IS-ISv4 routes

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX /FX line cards

10,000

IS-ISv4 network type

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

Point to point, broadcast

Groups with default timers (3s/10s) and multiple group optimization. [There are 2 primary, one for IPv4 and the other for IPv6, and 7926 secondary]

X9636C-R/RX and X9636Q-R line cards

7,928

Groups with aggressive timers (1s/3s) and multiple groups optimization. [There are 2 primary, one for IPv4 and the other for IPv6, and 7926 secondary]24

X9636C-R/RX and X9636Q-R line cards

7,928

Groups per interface or I/) module

X9636C-R/RX and X9636Q-R line cards

Maximum 16 (Because 16 is the unique virtual MAC address limit)

OSPF/OSPFv3 LSA/LSDB size

X9636C-R/RX and X9636Q-R line cards

250, 000

OSPF/OSPFv3 areas

X9636C-R/RX and X9636Q-R line cards

15

OSPFv2 neighbors

Nexus 9500 switches and the X9700-EX /FX, X9636C-R/RX, and X9636Q-R line cards

1,000

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches

256

OSPFv3 neighbors

Nexus 9500 switches and the X9700-EX/FX, X9636C-R/RX, and X9636Q-R line cards

1000

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches

256

OSPF/OSPFv3 LSA/LSDB size

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

100,000

OSPF/OSPFv3 areas

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX, 9300-FX2, and 9500 switches and the X9700-EX/FX line cards

100

Static routes

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX, 9300-FX2, and 9500 switches and the X9700-EX/FX line cards

4,000

VRFs

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

1,000

X9636C-R/RX and X9636Q-R line cards

3,967

VRRP groups per interface or I/O module

Nexus 9200, 9300, 9364C, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

250

Policy-based routing (PBR)

Configured sequences per policy

Nexus 9200, 9300-EX, 9300-FX/FX2 switches and the X9700-EX/FX line cards

128

Nexus 9300 and 9500 switches

256

Next-hop addresses per policy

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

32

IPv4 ACEs (unidimensional)

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches and the X9700-EX /FX line cards

3582 (per network forwarding engine)

Nexus 9300 and 9500 switches

3072 (per network forwarding engine)

IPv6 ACEs (unidimensional)

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches

1792 (per network forwarding engine)

Nexus 9300 and 9500 switches

1536 (per network forwarding engine)

IPv4 and IPv6s ACEs

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches

1024 IPv4 + 128 IPv6

Nexus 9300 and 9500 switches

2048 IPv4 + 256 IPv6

X9700-EX/FX line cards

1024 IPv4

Interfaces with PBR policy

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches and the X9700-EX/FX line cards

512

Nexus 9300 and 9500 switches

256

VRRPv3

VRRPv3 groups per interface

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches and the X9700-EX /FX line cards

255

VRRPv3 groups with default timers (1 s)

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches and the X9700-EX/FX line cards

490

VRRPv3 groups with relaxed timers (3 s)

Nexus 9200, 9300-EX, and 9300-FX/FX2 and the X9700-EX /FX line cards

490

Pathways with one VRRPv3 group with default timer (1 s)

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches and the X9700-EX/FX line cards

489

VRRPv3 groups and pathways combined

Nexus 9200, 9300-EX, and 9300-FX/FX2 switches and the X9700-EX/FX line cards

490

ECMP

ECMP Paths

Nexus 9200, 9300-EX, and 9300-FX/FX2/FXP, and 9504 / 9508 switches with -R/ RX line cards

64
17 The limit of supported BFD sessions for each EoR line card is 75.
18 If you have more than 490 groups, then only one group per SVI. SVIs cannot have a user defined MAC or any VRRP group with it.
19 The hash table is subject to collisions. Depending on the host route pattern, collisions might occur.
20 The hash table is subject to collisions. Depending on the host route pattern, collisions might occur.
21 Contains internet peering profile with additional IPv4 and IPv6 routes.
22 Internet profile with additional IPv4 routes (total of 914K routes consisting of IPv4 and 62K of IPv6)
23 Internet profile with additional IPv6 routes (total of 871K routes consisting of IPv6 and 696K of IPv4)
24 If the user has Multi-protocol configuration, user should configure appropriate CoPP policies so as to avoid any control plane traffic drops.

For the Cisco Nexus 9200 Platform switches, the default value for LPM unicast routes is 6000 (IPv4) or 1900 (IPv6). You can use the hardware profile multicast max-limit lpm-entries 0 command to increase the number of IPv4 LPM unicast routes to 8000. The hardware profile ipv6 lpm-entries maximum 0 command reserves the entire LPM table for IPv4. With this configuration, the IPv4 LPM scale is 14,000 (with 2000 reserved for multicast by default). This value can be increased to 16,000 with the hardware profile multicast max-limit lpm-entries 0 command. The hardware profile ipv6 lpm-entries maximum 4096 command reserves the entire LPM table for IPv6. With this configuration, the IPv6 LPM scale is 3900. When you allocate the entire table for IPv4 or IPv6 LPM unicast routes, the other address family cannot be used.

The maximum number of PBR next-hops based on 4 FM-E supported is 192 per slice of the forwarding engine


Note

  • The IPv4/IPv6 host routes and the IPv4 multicast routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.

  • The IPv4 and IPv6 unicast routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.

  • High availability (graceful restart and stateful switchover) is not supported when unicast or multicast aggressive timers are configured at any scale.

Guidelines and Limitations for OSPF Verified Scalability Limits

  • To achieve the highest scale, we recommend that you use a single OSPF instance instead of multiple instances.

  • Each OSPFv2 and OSPFv3 scale value might vary when combined with other parameters.

  • The graceful restart timeout value might need to be increased in multi-dimensional scenarios.

Table 14. PVLAN VXLAN Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

Primary VLANs

Nexus 9300-EX and 9300-FX/FX2 switches

16

Secondary VLANs

Nexus 9300-EX and 9300-FX/FX2 switches

20

Ports in community host mode

Nexus 9300-EX and 9300-FX/FX2 switches

40

Port in Isolated host mode

Nexus 9300-EX and 9300-FX/FX2 switches

40

Ports in isolated trunk mode

Nexus 9300-EX and 9300-FX/FX2 switches

40

Ports in promiscuous mode

Nexus 9300-EX and 9300-FX/FX2 switches

5

PVLANs allowed on a PVLAN port

Nexus 9300-EX and 9300-FX/FX2 switches

16

Table 15. VXLAN Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

IGMP snooping over VXLAN

VXLAN VLANs

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 and the X9700-EX/FX line cards

1000

VTEP Peers25

Nexus 9200 and 9300 switches

256

Nexus 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX, X9636C-R/RX, and X9636Q-R line cards

512

Underlay multicast groups

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches and the X9700-EX/FX line cards

128

Multi-Site

26

Number of Sites

Nexus 9200 and 9300 switches

10

Nexus 9300-EX, 9300-FX/FX2, 9332C, 9364C, and 9500 switches and the X9700-EX/FX line cards

25

Number of BGWs per site with TRM enabled27

Nexus 9200, 9300, 9332C, 9364C, and 9500 switches and the X9700-EX/FX line cards

2 (Anycast), 2(vPC)

Nexus 9300-EX and 9300-FX/FX2 switches

6 (Anycast), 2(vPC)

Number of sites for TRM

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, 9332C, 9364C, and 9500 switches and the X9700-EX/FX line cards switches

15 sites

Number of BGWs for TRM

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, 9332C, 9364C, and 9500 switches and the X9700-EX/FX line cards switches

06 BGW

VTEPs per Site

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

256

Tenant Route Multicast L3 Mode with VXLAN BGP eVPN

VXLAN Layer 2 VNI

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 and the X9700-EX/FX line cards

1000

VXLAN Layer 3 VNI/VRFs

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 and the X9700-EX/FX line cards

250

VTEP Peers

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 and the X9700-EX/FX line cards

256

Underlay Multicast Group (PIM ASM Underlay)

Nexus 9200, 9300,9300-EX, and 9300-FX/FX2 and the X9700-EX/FX line cards

228

Overlay Multicast Group (PIM ASM & PIM SSM)

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 and the X9700-EX/FX line cards

720029

VXLAN Flood and Learn

Virtual network identifiers (VNIs) or VXLAN-mapped VLANs

Nexus 9200, 9300, 9300-EX,and 9300-FX/FX2 switches

2000

Nexus 9500 switches and the X9700-EX/FX line cards

1000

Underlay multicast groups

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

128

Overlay MAC addresses

Nexus 9200, 9300, and 9500 switches

64,000

Nexus 9300-EX and 9300-FX switches and the X9700-EX/FX line cards

90,000

Nexus 9300-FX2 switches

60,000

Remote VXLAN tunnel endpoints (VTEPs Multicast)

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

256

Ingress replication peers 30

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

256

Ingress replication Layer 2 VNIs

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

1000

MAC addresses for ingress replication

Nexus 9200, 9300, and 9500 switches

64,000

Nexus 9300-EX and 9300-FX/FX2 switches and the X9700-EX/FX line cards

90,000

Port VLAN translations under an interface

Nexus 9300 and 9500 switches and the X9700-EX/FX line cards

100

Nexus 9300-EX and 9300-FX switches

4,000

Nexus 9300-FX2 switches

500

Port VLAN translations in a switch

Nexus 9300 switches and the X9700-EX/FX line cards

2000

Nexus 9300-EX and 9300-FX switches

24000

Nexus 9300-FX2 switches

6,000

Nexus 9500 switches

200

Static MAC addresses pointing to a remote VTEP

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

1000

VXLAN VLAN logical port VP count

Nexus 9300 and 9500 switches

7,000

VXLAN VLANs per FEX port (host interface)

Nexus 9300 and 9500 switches

75

Nexus 93180YC-EX

7531

Layer 2 routed VNIs for vPC-centralized gateway

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

450

IGMP groups

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

8,192

VXLAN BGP eVPN

Layer 2 VNIs

Nexus 9200 and 9300 switches and the X9636C-R/RX and X9636Q-R line cards

2000

Nexus 9300-EX and 9300-FX/FX2 switches

2000, 4000 (with no Layer 3 VNIs)

Nexus 9500 switches and the X9700-EX/FX line cards

1000

Xconnect VLANs

Nexus 9300, 9332C, 9364C, 9300-EX, and 9300-FX/FX2 switches

40

SVI with Distributed Anycast Gateway; Layer 2 VNI extended

Nexus 9300-EX switches

2,00032

Nexus 9300-EX switches

200033

Nexus 9300 and 9300-FX/FX2 switches

2,000

Nexus 9500 switches and the X9700-EX/FX line cards

1000

Layer 3 VNIs / VRFs34

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches and the X9636C-R/RX and X9636Q-R line cards

900

Nexus 9500 switches and the X9700-EX/FX line cards

750

Underlay multicast groups

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

128

VTEPs

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2, and 9500 switches and the X9700-EX FX, X9636C-R/RX, and X9636Q-R line cards

512

MAC addresses

Nexus 9200, 9300, and 9500 switches

64,000

Nexus 9300-EX and 9300-FX/FX2 switches and the X9700-EX /FX line cards

90,000

Nexus 92348GC-X switches

97,000

IPv4 host routes

Nexus 9200, 9300, and 9500 switches

60,000

Nexus 92348GC-X switches

16,000

Nexus 9300-EX switches

458,000

Nexus 9300-FX/FX2 switches

471,000

X9700-EX/FX line cards

656,000

X9636C-R/RX and X9636Q-R line cards

128,000

IPv6 host routes

Nexus 9200, 9300, and 9500 switches)

7,000

Nexus 9300-EX switches

24,000

Nexus 9300-FX/FX2 switches

265,000

X9700-EX/FX line card

34,000

X9636C-R/RX and X9636Q-R line cards

32,000

Overlay IPv4 LPM routes

Nexus 9200 switches

8,000

Nexus 9300 and 9500 switches

12,000

Nexus 9300-EX switches

458,000

Nexus 9300-FX/FX2 switches

471,000

X9700-EX/FX line cards

656,000

Overlay IPv6 LPM routes

Nexus 9200 switches

2,000

Nexus 9300 and 9500 switches

7000

Nexus 9300-EX switches

206,00035

Nexus 9300-FC/FX2 switches

265,00036

X9700-EX/FX line cards

174,00037

VXLAN VLAN logical port VP count

Nexus 9300 switches

10,000

Nexus 9500 switches

7,000

VXLAN VLANs per FEX port (host interface)

Nexus 9500 switches

75

IGMP groups

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

8,192

VXLAN BGP eVPN Ingress Replication

Layer 2 VNIs

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches

2,000

Nexus 9500 switches and the X9700-EX/FX line cards

1,000

Xconnect VLANs

Nexus 9300, 9300-EX, and 9300-FX/FX2 switches

40

Selective Qinvni with multiprovider tag

Nexus 93180YC-EX, 93180YC-FX, and 9336C-FX2 switches

 -

4,000 mappings, 10 provider VLANs;System wide: 48,000 mappings, 512 Provider VLANs

SVI with Distributed Anycast Gateway; Layer 2 VNI extended

Nexus 9200 and 9300-EX switches

2,00038

Nexus 9300 and 9300-FX/FX2 switches

2,000

Nexus 9500 switches and the X9700-EX/FX line cards

1,000

Layer 3 VNIs / VRFs39

Nexus 9200, 9300, 9300-EX, and 9300-FX/FX2 switches

900

Nexus 9500 switches and the X9700-EX/FX line cards

750

VTEPs

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

256

MAC addresses

Nexus 9200, 9300, and 9500 switches

64,000

Nexus 9300-EX and 9300-FX/FX2 switches and the X9700-EX/FX line cards

90,000

IPv4 host routes

Nexus 9200, 9300, and 9500 switches

32,000

Nexus 9300-EX switches

458,000

Nexus 9300-FX/FX2 switches)

471,000

X9700-EX/FX line cards

656,000

IPv6 host routes

Nexus 9200, 9300, and 9500 switches

7,000

Nexus 9200, 9300, and 9500 switches

24,000

Nexus 9300-FX/FX2 switches

265,000

X9700-EX/FX line cards

34,000

Overlay IPv4 LPM routes

Nexus 9200 switches

8,000

Nexus 9300 and 9500 switches

12,000

Nexus 9300-EX switches

458,000

Nexus 9300-FX/FX2 switches

471,500

(X9700-EX/FX line cards

656,000

Overlay IPv6 LPM routes

Nexus 9200 switches

2,000

Nexus 9300 and 9500 switches

7,000

Nexus 9300-EX switches

206,00040

Nexus 9300-FX/FX2 switches

265,00041

X9700-EX/FX line cards

174,00042

VXLAN VLAN logical port VP count

Nexus 9300 and 9500 switches

7,500

VXLAN VLANs per FEX port (host interface)

Nexus 9300 and 9500 switches

75

IGMP groups

Nexus 9200, 9300, 9300-EX, 9300-FX/FX2, and 9500 switches and the X9700-EX/FX line cards

8,192
25 In case of IR, each VNI can have a max of 64 peers.
26 All the other BGW numbers (number of supported L2VNIs, L3VNIs, MAC addresses, IP addresses, and so on) match the values supported on a generic VXLAN EVPN VTEP node.
27 Multisite enabled with TRM supported number of L2VNIs - 1000 and L3VNIs – 100. Maximum supported multicast underlay and overlay route is 8000.
28 VXLAN underlay and overlay multicast routes shares the same hardware table. Maximum Multicast routes is 8000 in the default mode. If you want more overlay route scale, reduce the underlay multicast control group.
29 VXLAN underlay and overlay multicast routes shares the same hardware table. Maximum Multicast routes is 8000 in the default mode. If you want more overlay route scale, reduce the underlay multicast control group.
30 In case of IR, each VNI can have a max number of 64 peers
31 This is the limit for the Cisco Nexus 93180YC-EX and other fiber based switches. All copper based 9300-EX switches are not applicable.
32 Only 1900 SVI are supported if dual stack is used/IPv6 is used.
33 Only 1900 SVI are supported if dual stack is used/IPv6 is used.
34 ECMP objects are not shared across multiple VRFs.
35 All /64 routes + 4000 for non /64 routes.
36 All /64 routes + 4000 for non /64 routes.
37 All /64 routes + 4000 for non /64 routes.
38 Only 1900 SVI are supported if dual stack is used/IPv6 is used.
39 ECMP objects are not shared across multiple VRFs.
40 All /64 routes + 4000 for non /64 routes.
41 All /64 routes + 4000 for non /64 routes.
42 All /64 routes + 4000 for non /64 routes.
Table 16. Tetration Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limit

TCAM size

Nexus 92160YC-X, 9300-EX, and 9300-FX switches

1,024 entries

Nexus 92160YC-X switches

IPv4 – 4 entries per rule (TCP, UDP, ICMP, and IP)

Nexus 9300-EX, and 9300-FX switches

IPv4 – 2 entries per rule (ICMP and IP)

Nexus 92160YC-X switches

IPv6 – 8 entries per rule (4 entries per ICMP and IPv6 for a total of 8 entries)

Nexus 9300-EX and 9300-FX switches

IPv6 – 8 entries per rule (4 entries per ICMP and IPv6 for a total of 8 entries)

Nexus 92160YC-X, 9300-EX, and 9300-FX switches

24 entries out of 1000 is consumed for default

TCAM scale

Nexus 92160YC-X switches

250 (IPv4) or 62 (IPv6)

Nexus 9300-EX and 9300-FX switches

500 (IPv4) or 125 (IPv6)

The entire Cisco Tetration Analytics documentation set is available at the following URL: https://www.cisco.com/c/en/us/support/data-center-analytics/tetration-analytics/tsd-products-support-series-home.html

Verified Scalability Limits - Multidimensional

The tables in this section list the verified scalability limits for the Cisco Nexus 9508 switch with an X9636C-R, X9636C-RX, or X9636Q-R line card or a C9508-FM-R fabric module and Cisco Nexus 9504 with -R line cards for Cisco NX-OS Release 9.3(2). These limits are validated with a multidimensional configuration. The values provided in these tables focus on the scalability of all listed features at the same time.

Each number is the absolute maximum currently supported by this Cisco NX-OS release for the corresponding feature. If the hardware is capable of a higher scale, future software releases might increase this verified maximum limit. Results might differ from the values listed here when trying to achieve maximum scalability with multiple features enabled.


Attention
These numbers are not the maximum verified values if each feature is viewed in isolation. For these numbers, see the "Verified Scalability Limits" section.
Table 17. eBGP/IS-IS Profile Verified Scalability Limits (Multidimensional)

Feature

Verified Limits

Number of 100G ports

288

ECMP

16-way (Upstream)

BGP neighbors

960

BGP IPv4 /32 unicast routes

30,000

BGP IPv4 VLSM unicast routes

18,000

BGP IPv6 /128 unicast routes

16,000

BGP IPv6 VLSM unicast routes

1,000

IS-IS v2 neighbors

255

IS-IS v3 neighbors

255

IS-IS L2 adjacency

16

IS-IS IPv4 /32 unicast routes

20,000

IS-IS IPv4 VLSM unicast routes

1,000

IS-IS IPv6 /128 unicast routes

20,000

IS-IS IPv6 VLSM unicast routes

1,000

BFD sessions

272

PIM neighbors

256

ACL ACEs

15,000

500

Sub-interfaces

712

SPAN sessions

1 local SPAN session

Multicast SSM

20,000

Table 18. iBGP/OSPF Profile Verified Scalability Limits (Multidimensional)

Feature

Verified Limits

Number of 100G ports

180

Number of 40G ports

108

ECMP

8-way (Upstream)

BGP neighbors

8

BGP IPv4 VLSM unicast routes

40,000

BGP IPv6 VLSM unicast routes

10,000

OSPFv2 neighbors

108

OSPFv3 neighbors

30

OSPF IPv4 /32 unicast routes

100,000

OSPF IPv4 VLSM unicast routes

155,000

OSPFv3 IPv6 /128 unicast routes

1,000

OSPFv3 IPv6 VLSM unicast routes

9,000

BFD sessions

108

VRF

250

PIM neighbors

108

IPv4 (*,G) multicast routes

2,000

IPv4 (S,G) multicast routes

10,000

ACL ACEs

500 (IPv4)

500 (IPv6)

SPAN sessions

1 local SPAN session

Table 19. iBGP/EIGRP Profile Verified Scalability Limits (Multidimensional)

Feature

Verified Limits

Number of 100G ports

180

Number of 40G ports

108

ECMP

16-way (Upstream)

BGP neighbors

8

BGP IPv4 VLSM unicast routes

40,000

BGP IPv6 VLSM unicast routes

10,000

EIGRP v4 neighbors

276

EIGRP v6 neighbors

276

EIGRP IPv4 /32 unicast routes

30,000

EIGRP IPv4 VLSM unicast routes

1,000

EIGRP IPv6 /128 unicast routes

30,000

EIGRP IPv6 VLSM unicast routes

1,000

BFD sessions

276

VRF

250

PIM neighbors

276

IPv4 (*,G) multicast routes

6,000

IPv4 (S,G) multicast routes

16,000

ACL ACEs

500 (IPv4)

500 (IPv6)

SPAN sessions

1 local SPAN session

Table 20. MPLS Verified Scalability Limits (Multidimensional)

Feature

Verified Limits

MPLS L3VPN

3967

VPE

3967

PE nodes

3

PE routes

20,000

X9636C-RX line card: ACL - IPv4

95,000

X9636C-RX line card: ACL - IPv6

20,000

HSRP, HSRP VIP

3967 each for v4 and v6

vPC uRPF

3967

Strict uRPF

Yes

VRF

3967

SVI

3967

Layer 3 VPN routes IP ECMP

2,000

MPLS LSR ECMP

2,000

VPNv4 routes

400,000

VPNv6 routes

90,000

EBGP neighbors

750

Table 21. Layer 2/Layer 3 Boundary Verified Scalability Limits (Multidimensional)

Feature

Verified Limits

MAC addresses

19,000

Sub-interfaces

500

vPC Port channels

46

ECMP

16-way (Upstream)

OSPFv2 neighbors

47

OSPFv3 neighbors

47

OSPF IPv4 /32 unicast routes

45,000

OSPF IPv4 VLSM unicast routes

1,000

OSPF IPv6 /128 unicast routes

20,000

OSPF IPv6 VLSM unicast routes

1,000

BFD sessions

49

VRF

250

VLAN

3,750

SVI

3,750

VRRP v4 groups

1,996 VRRS / 4 VRRPv3

VRRP v6 groups

1,996 VRRS / 4 VRRPv3

HSRP IPv4

1,743 Secondary groups / 7 Primary groups

HSRP IPv6

1,743 Secondary groups / 7 Primary groups

PIM neighbors

396

IPv4 (*,G) multicast routes

3,080

IPv4 (S,G) multicast routes

26,600

IGMP snooping database entries

6,400

sFlow enabled interfaces

83

UDLD enabled interfaces

93

SPAN sessions

1 local SPAN session

Table 22. Segment Routing Verified Scalability Limits (Multidimensional)

Feature

Verified Limits

VLAN

100

SVI

100

MAC entries

10,000

ARP entries

70

HSRP v4 VIPs

100

HSRP v6 VIPs

100

LACP

11

LACP members

4

eBGP IPv6 neighbors

9

eBGP IPv4 LU neighbors

9

IPv4 (LU) routes

6888

IPv4 (LU) paths

17580

IPv6 routes

6,663

IPv4 (LU) routes

17,338

SR ECMP

18 (dual-homed)

MPLS HW entries

11,957

Table 23. VXLAN Profile Verified Scalability Limits (Multidimensional)

Feature

Verified Limits

Ports

16

ECMP

8-way (Upstream)

BGP neighbors

200

BGP EVPN Layer 2 VPN host routes

64,000

BGP IPv4 VLSM unicast routes or ospf

10,000

BGP IPv6 VLSM unicast routes or ospf

6,000

BFD sessions

20

PIM neighbors

20

IPv4 (*, G) multicast routes (co-existing)

4,000

IPv4 (S,G) multicast routes (co-existing)

2,000

Layer 3 VNI

900

Layer 2 VNI

2000

Local VTEP

1

Remote VTEPs

256

VLAN

3600

SVI

900

MAC

90,000

Deployment Case Studies

This section provides sample topologies for some common deployments. For each topology, the scalability numbers are the limits with all of the listed features enabled at the same time.


Attention
These numbers are not the maximum verified values if each feature is viewed in isolation. For these numbers, see the "Verified Scalability Limits" section.

Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)

This Layer 2/Layer 3 aggregation topology consists of Cisco Nexus 9508 switches as virtual port channel (vPC) aggregation pairs. These aggregation nodes are fully loaded with N9K-X9564TX, N9K-X9564PX, and N9K-X9636PQ line cards. The N9K-X9636PQ line cards are used in normal mode and breakout mode. Cisco Nexus 9396PX and 93128TX switches are used as top-of-rack units with Cisco Nexus 3000 Series switches to achieve the desired vPC scale.

The Cisco Nexus 9508 switch is also used as a core Layer 3 node that connects to a pair of vPC aggregation nodes. The focus of the topology is to test IPv4 ARP, IPv6 neighbor discovery (ND), and Layer 2 scalability and other routing, switching, and Layer 4 through Layer 7 features for management and operations. All Layer 3 interfaces are configured for dual stack, and the traffic is dual stack for all VLANs.

In the following table, the Verified Limit column lists the verified scaling capabilities with all listed features enabled at the same time. The scale numbers listed here exceed those used by most customers in their topologies. These numbers are not the maximum verified values if each feature is viewed in isolation.

Table 24. Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)

Feature

9508 Verified Limit (Max-Host Routing Mode)

Fully loaded chassis

1 N9K-X9636PQ, 1 N9K-X9564TX, 2 N9K-X9564PX, 1 N9K-X9432PQ, 1 N9K-X9536PQ

Physical interfaces enabled

276

Multicast S,G routes

653

Multicast *,G routes

500

IPv4 unicast routes (LPM)

5000

IPv6 unicast routes (LPM)

850

IPv4 ARP

65,000

IPv6 ND

40,000

MAC addresses

90,000

VLANs

490

vPCs*

200

OSPFv2 neighbors

20

OSPFv3 neighbors

4

BGP (IPv4) neighbors

65

BGP (IPv6) neighbors

65

SVIs

490

STP logical ports

2800 (RPVST)

HSRP VLANs (IPv4/IPv6)

490

Virtual ports

700

Port channel links

8

* The number of VLANs per vPC supported should be within the MST or RPVST virtual port count specified in this table, depending on the topology.

Layer 2/Layer 3 Aggregation Topology (Default Routing Mode)

This Layer 2/Layer 3 aggregation topology consists of Cisco Nexus 9516 switches as virtual port channel (vPC) aggregation pairs. These aggregation nodes are fully loaded with N9K-X9564TX, N9K-X9564PX, and N9K-X9536PQ line cards. The chassis is fully loaded with five line cards configured for breakout mode. The Cisco Nexus 9396PX and 93128TX switches are used as top-of-rack units with Cisco Nexus 3000 Series switches to achieve the desired vPC scale. The Cisco Nexus 9516 nodes are running in default routing mode. The Cisco Nexus 3164Q switch is also used as a core Layer 3 node that connects to a pair of vPC aggregation nodes.

The focus of the topology is to test IPv4 ARP, IPv6 neighbor discovery (ND), Layer 2 scalability, IPv4 and IPv6 LPM routing, Layer 2 and Layer 3 multicast routing for IPv4, and Layer 4 through Layer 7 features for management and operations. All Layer 3 interfaces are configured for dual stack, and the traffic is dual stack for all VLANs.

In the following table, the Verified Limit column lists the verified scaling capabilities with all listed features enabled at the same time. These numbers are not the maximum verified values if each feature is viewed in isolation.

Table 25. Layer 2/Layer 3 Aggregation Topology (Default Routing Mode)

Feature

9516 Switch Verified Limit (Default Routing Mode)

9300 Platform Verified Limit (Default Routing Mode)

Chassis configuration

5 N9K-X9432PQ line cards

4 N9K-X9464PX line cards

3 N9K-X9536PQ line cards

3 N9K-X9464TX line cards

1 N9K-X9564TX line card

9372

Physical ports

1335

50

vPCs

303

24

SVIs

450

450

VRFs

100

100

IPv4 ARP

40,000

40,000

IPv6 ND

10,000

10,000

STP logical ports

10,000

6000

BGP neighbors (IPv4 + IPv6)

502 + 502

502 + 502

IPv4 LPM routes

50,000

6000

IPv6 LPM routes

10,000

1000

BFD (IPv4 + IPv6)

300

102

IGP OSPFv2 neighbors

502

502

IGP OSPFv3 neighbors

502

502

HSRP (IPv4 + IPv6)

450 + 450

450 + 450

IGMP groups

2000

2000

Multicast *,G routes

2000

2000

Multicast S,G routes

8000

6000

Tracking objects

450

450

VLANs

500

500

PIM neighbors

502

502

MAC addresses

60,000

60,000

Network address translation (NAT)

Not applicable

756

sFlow

256

32

FEX System Topology

The FEX 9500 multi-dimensional scale topology consists of Cisco Nexus 9508 switches as virtual port channel (vPC) pairs. Each switch has multiple X9564PX line cards. Each switch has 32 FEX uplinks connected to them. The FEX 9300 multi-dimensional scale topology consists of two Cisco Nexus 9396PX switches used in vPC mode along with 16 FEX uplinks connected to each switch. Multiple FEXs of type Nexus 2248TP-E, 2232PP, 2248PQ, and 2348UPQ are used.

The switches are used at the Layer 2 and Layer 3 boundary and are also configured as VXLAN VTEPs. The FEX host ports are operating as Layer 2 ports. The switches are configured as gateways with the use of SVI interfaces.

In the following table, the Verified Limit column lists the verified scaling capabilities with all listed features enabled at the same time. The scale numbers listed here exceed those used by most customers in their topologies. These numbers are not the maximum verified values if each feature is viewed in isolation.

Table 26. FEX System Topology

Feature

9500 Platform Verified Limit

9300 Platform Verified Limit

Fabric Extenders

32

16

Up interfaces

1100

560

Port channels

426

256

vPC members

390

360

VLANs

744

416

PVLAN VLANs

56

56

Secondary VLANs per primary VLAN

25

25

MAC addresses

45,000

25,000

HSRP

365

365

ARP

12,000

10,000

Neighbor discovery (ND)

5000

5000

Multicast (*,G)

4000

4000

Multicast (S,G)

4000

4000

Multicast System Topology

Two Cisco Nexus 9508 switches are configured as vPC peers in one domain, and two Cisco Nexus 9372PX switches are configured as vPC peers in the other domain. The chassis are fully loaded with N9K-X9432PQ, N9K-X9464PX, N9K-X9536PQ, N9K-X9564PX, N9K-X9564TX, and N9K-X9636PQ line cards. eBGP routing is used to connect these two PIM domains. OSPF is used as IGP in one domain, and EIGRP is configured in the other domain. This setup is configured with multiple rendezvous points (RPs) to serve different multicast group ranges. BSR is used to advertise RP information in both of these PIM domains. PIM anycast is used in one domain, and MSDP anycast is used in the other domain for redundancy and load balancing. Static RP configuration is also used for a range of multicast groups.

The Cisco Nexus 9516 and Cisco Nexus 7000 Series switches are used as Layer 3 core routers in one domain. The Cisco Nexus 3164Q switches are used as Layer 3 core routers in the other domain. This topology also includes the Cisco Nexus 9396PX, Cisco Nexus 9372PX, and Cisco Nexus 3016/3064T switches in the access layer.

In addition to including Layer 2/Layer 3 IPv4 multicast routing, this topology also covers IPv4 and IPv6 host and LPM routing and Layer 2 unicast forwarding. All interfaces are configured for dual stack.

In the following table, the Verified Limit column lists the verified scaling capabilities with all listed features enabled at the same time. These numbers are not the maximum verified values if each feature is viewed in isolation.

Table 27. Multicast System Topology

Feature

9500 Platform Verified Limit

9300 Platform Verified Limit

Chassis configuration

N9K-X9636PQ, N9K-X9536PQ, N9K-X9564PX, N9K-X9564TX, N9K-X9432PQ, N9K-X9464PX, N9K-X9432PQ, C3164PQ

C9372PX, C9396PX, C3164PQ

Multicast S,G routes

17,500

5000

Multicast *,G routes

2500 (IGMP)

12500 (snooping)

500 (IGMP)

2500 (snooping)

Sources

2000, 200, 100, 40, 10, 3, 2, 1

2000, 200, 100, 40, 10, 3, 2, 1

Replications

40

20

ECMPs

16

8

SVIs

200

200

HSRP/VRRP

200 HSRP

100 VRRP

MAC addresses

40,000

10,000

ARP

20,000

4000

Unicast LPM IPv4 routes

20,000

4000

Unicast LPM IPv6 routes

10,000

1000

IPv4 ARP

18,000

4000

IPv6 ND

4000

2000

MSDP peers (fully mesh)

4

4

Anycast RPs (MSDP and PIM anycast) 43

2 MSDP

2 PIM anycast

IPv4 multicast routes with PIM bidirectional groups

8000

8000

43 This multicast system topology consists of two multicast PIM domains. The Multicast Source Discovery Protocol (MSDP) is used to exchange multicast source information between these two domains.

VXLAN BGP/eVPN iBGP Centric Topology

This VXLAN BGP/eVPN iBGP centric topology consists of Cisco Nexus 9300 and 9500 Platform switches acting as VXLAN vPC tunnel endpoints (VTEPs) and VXLAN non-vPC VTEPs. VXLAN VTEPs establish iBGP sessions to a Cisco Nexus 9508 switch (route reflector) acting as a spine node. VXLAN-distributed anycast gateway SVIs are configured for dual stack, and the traffic is dual stack.

The focus of this topology is to test VXLAN overlay network scale and underlay Layer 2 switching and other routing, multicast, and Layer 4 through Layer 7 features for management and operations. Underlay PIM neighbors and IS-IS adjacency were tested with the default timer and Bidirectional Forwarding Detection (BFD) enabled on all links.

In the following table, the Verified Limit column lists the verified scaling capabilities with all listed features enabled at the same time. These numbers are not the maximum verified values if each feature is viewed in isolation.

Table 28. VXLAN BGP/eVPN iBGP Centric Topology

Feature

Supported Platform

Verified Limit

System Routing Template

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9500 switches and the X9700-EX/FX line cards

default

Nexus 9364C switches

Not applicable

VXLAN VTEPs

Nexus 9200, 9300, 9300-EX, 9300-FX, 9364C and 9500 switches and the X9700-EX/FX line cards

128

VXLAN Layer 2 VNIs

Nexus 9200, 9300, 9300-EX, 9300-FX, 9364C and 9500 switches and the X9700-EX/FX line cards

2000

VXLAN Layer 3 VNIs/VRFs

Nexus 9200, 9300, 9300-EX, 9300-FX, 9364C and 9500 switches and the X9700-EX/FX line cards

500

VXLAN multicast groups

Nexus 9200, 9300, 9300-EX, 9300-FX, 9364C and 9500 switches and the X9700-EX/FX line cards

128
VXLAN overlay MAC addresses

Nexus 9200, 9300, 9300-EX, 9300-FX, 9364C and 9500 switches and the X9700-EX/FX line cards

64,000

VXLAN overlay IPv4 host routes

Nexus 9200, 9300, 9300-EX, 9300-FX, 9364C and 9500 switches and the X9700-EX/FX line cards

60,000

VXLAN overlay IPv6 host routes

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9500 switches and the X9700-EX/FX line cards

16,000

Nexus 9364C switches

Not applicable

VXLAN overlay IGMP Snooping groups

Nexus 9200, 9300, 9300-EX, 9300-FX, 9364C and 9500 switches and the X9700-EX/FX line cards

1000

VXLAN IPv4 LPM routes

Nexus 9200, 9300-EX, 9300-FX, and 9364C switches and the X9700-EX/FX line cards

5120

VXLAN IPv6 LPM routes

Nexus 9200, 9300-EX, and 9300-FX switches and the X9700-EX/FX line cards

1500

Nexus 9364C switches

Not applicable

VXLAN VLAN logical port VP count

Nexus 9364C switches

Not applicable

VLANs on VTEP node

Nexus 9200, 9300, 9300-EX, 9300-FX, 9364C, and 9500 switches and the X9700-EX/FX line cards

1700 (total VLANs)

1500 (VXLAN VLANs)

200 (non-VXLAN VLANs)

MST instances

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9500 switches and the X9700-EX/FX line cards

20

Nexus 9364C switches

Not applicable

STP logical ports

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9500 switches and the X9700-EX/FX line cards

3500

Nexus 9364C switches

Not applicable

vPC port channels

Nexus 9200, 9300, 9300-EX, and 9300-FX switches and the X9700-EX/FX line cards

40

Nexus 9364C switches

Not applicable

Underlay IS-IS neighbors

Nexus 9200, 9300, 9300-EX, and 9300-FX switches and the X9700-EX/FX line cards

32

Nexus 9364C switches

Not applicable

Underlay PIM neighbors

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9500 switches and the X9700-EX/FX line cards

12

Nexus 9364C switches

Not applicable

Underlay HSRP groups for regular VLANs

Nexus 9364C switches

Not applicable

Underlay vPC SVIs

Nexus 9200, 9300, 9300-EX, 9300-FX, and 9500 switches and the X9700-EX/FX line cards

200

Nexus 9364C switches

Not applicable