Configuring ToR Switches and Deploying Networks, Release 12.2.2

 
Updated April 8, 2024
PDF
Is this helpful? Feedback

New and Changed Information

The following table provides an overview of the significant changes up to this current release. The table does not provide an exhaustive list of all changes or of the new features up to this release.

Release Version Feature Description

NDFC release 12.2.2

Support for assigning a vPC/port-channel ID range for leaf-ToR pairings and for specifying a custom vPC/port-channel ID

With this feature, you can assign one virtual port channel (vPC)/port-channel ID range for leaf-ToR pairings and also specify a custom vPC/port-channel ID in a Data Center VXLAN EVPN fabric.

Beginning with NDFC 12.2.2, NDFC added an Action > Edit Pairing option on the TOR Pairing page for editing leaf and ToR vPC/port-channel IDs.

For more information, see Configuring a Specific vPC/Port-Channel ID Range for Leaf-ToR Pairing.

Configuring ToR Switches and Deploying Networks in Data Center VXLAN EVPN Fabrics

Overview

Layer 2 Top-of-Rack (ToRs) are considered as replacements for FEXs. In earlier NDFC releases, you can add the Layer 2 ToR switches in an external fabric, and connect them to the leaf switches in a Data Center VXLAN EVPN fabric. The network overlay attachments were managed from the VXLAN EVPN Multi-Site domain as both Data Center VXLAN EVPN fabrics with spine/leaf and external fabrics with ToRs were added to a VXLAN EVPN Multi-Site domain. From Cisco NDFC Release 12.1.1e, you can add Layer 2 ToR devices in the same fabric as in a spine/leaf Data Center VXLAN EVPN fabric. This allows a single configuration point for deploying and extending networks for a VXLAN fabric topology with Layer 2 ToRs.

note.svg

It is not recommended to have a combination of FEX and ToRs in leaf switches due to a scale limitation.

You can physically connect a Layer 2 ToR in one of the following ways:

  • Connected to a leaf through a port-channel

  • Connected to a vPC pair of leafs through a vPC

  • Connected to one of the leafs in a vPC pair through a port-channel

You can configure a pair of Layer 2 ToRs in a vPC. A ToR vPC pair can only be connected to a leaf vPC pair through back-to-back vPC (also known as a double-sided vPC).

Guidelines for Configuring ToR Switches in Data Center VXLAN EVPN Fabrics

You add ToR devices to a Data Center VXLAN EVPN fabric in the same way as all other devices.

  • ToR role must be set on ToR devices before you perform a Recalculate and Deploy operation.

  • Perform a Recalculate and Deploy after any change of ToR pairings/unpairings.

  • ToRs must be physically connected to the intended parent leaf switches.

  • vPC pairing should be done before a leaf-ToR pairing/unpairing.

  • ToR pairing/unpairing can be done on an individual leaf, or a leaf vPC pair.

  • Network-overlay association for ToR switches are managed from their parent leaf(s).

  • ToR ports are shown as additional ports under a leaf.

  • NDFC handles all intermediate configurations transparently.

  • Deletion of a leaf will also delete all associated child ToR devices.

  • A leaf can be connected to many ToRs, but a ToR can be connected to only one leaf or leaf-vPC pair.

Limitations for Configuring ToR Switches in Data Center VXLAN EVPN Fabrics

  • Interface groups on Layer 2 ToRs are not supported.

  • Brownfield import on Layer 2 ToRs is not supported.

Configuring ToR Switches in Data Center VXLAN EVPN Fabrics

On the Edit Fabric page, click the Advanced tab and specify the applicable fabric settings.

Field Description

Spanning Tree Root Bridge Protocol

Choose the protocol from the drop-down list for configuring a root bridge.

The available protocols are:

  • rpvst - Rapid Per-VLAN Spanning Tree

  • mst - Multiple Spanning Tree

  • unmanaged (default) - STP root is not managed by NDFC.

note.svg

It is recommended to use the mst protocol for a Layer 2 ToR.

Spanning Tree VLAN Range

Specify the VLAN range. The default value is 1 - 3967.

MST Instance Range

Specify the MST instance range. The default value is 0.

Spanning Tree Bridge Priority

Specify the bridge priority for the spanning tree in increments of 4096.

Supported Topologies for ToR Switches

The following topologies with ToR switches are supported:

note.svg

Only Cisco Nexus 9000 series switches are supported as ToR switches.

452254.jpg
Topology-1: ToR switch with port channel directly connected to the leaf switch
452255.jpg
Topology-2: ToR switch connected to leaf switches in a vPC pair
452256.jpg
Topology-3: ToR switches with port channels connected individually to leaf switches in a vPC pair
452257.jpg
Topology-4: ToR switches with back-to-back vPC connections with leaf and ToR switches in vPC pair

Unsupported Topology for ToR Switches

The following topology with ToR switches is not supported: ToR vPC pair connected to a single leaf switch.

452258.jpg
ToR Unsupported Topology

Configuring ToR Switches

Create a fabric with a Data Center VXLAN EVPN template and add switches to the fabric, including switches used as ToRs. For more information, see the section "Creating a VXLAN EVPN Fabric Using the Data Center VXLAN EVPN Template" in Data Center VXLAN EVPN. Based on the selection of topology, perform any of the following steps:

Create Leaf-ToR Pairings

Perform the following procedure to configure a ToR and a leaf switch as shown in the ToR Topology-1 and 3, where ToR switch(s) are connected to leaf switch(s) through a port channel. For Topology-1 and 3, there is only one ToR and one leaf. Add switches used as ToR switches. For all four topologies, NDFC only uses port channels in the ToR pairings.

  1. Add ToR switches to the Data Center VXLAN EVPN fabric and set the role as ToR.

  2. On the Fabric Overview > Switches page, choose a ToR switch.

  3. Click Actions > TOR Pairing.

    The TOR Pairing page displays the ToR switches on the top horizontal bar and a list of potential pairing leaf switches below the ToR switches. The status of the switches displays in the Details column.

    Only one ToR switch is connected to a single leaf switch or a leaf switch in a vPC pair.

  4. Click Save.

  5. On the Fabric Overview page, click Actions > Recalculate and Deploy.

  6. After the configuration deployment is completed on the Deploy Configuration page, click Close.

Perform the following procedure to configure a ToR and a leaf switch as shown in the ToR Topology-2 and 4. For Topology-2, a ToR switch is connected to leaf switches in a vPC pair. For Topology-4, vPC ToR switches are connected to the vPC leaf switches through back-to-back vPC connections.

  1. Select either of the vPC-paired leaf switches and click Actions > TOR Pairing. For more information on configuring vPC on ToR switches, see the "Creating a vPC Setup" section in External Connectivity Network.

    The TOR Pairing page appears with the list of ToR and vPC leaf switches.

  2. Click Edit Pairing under the Action column.

  3. Check the Enable <switch-name> Pairing as TOR Pairing check box.

  4. Click Save.

  5. On the Fabric Overview page, click on Actions > Recalculate and Deploy.

  6. After the configuration deployment is completed on the Deploy Configuration page, click Close.

Unpair Leaf-ToR Pairings

Perform the following procedure to unpair the ToR switch.

  1. Remove the overlay attachment before unpairing the ToR.

  2. On the Fabric Overview page, click the Switches tab.

  3. Select any leaf switch that has ToR pairing and click on Actions > TOR Pairing.

    The TOR Pairing page appears with the list of paired switches.

  4. Click Edit Pairing under the Action column.

  5. Uncheck the Enable <switch-name> Pairing as TOR Pairing check box.

  6. Click Save.

  7. On the Fabric Overview page, click Actions > Recalculate and Deploy.

  8. On the Deploy Configuration page, click Deploy.

  9. After the configuration deployment is completed on the Deploy Configuration page, click Close.

Configuring a Specific vPC/Port-Channel ID Range for Leaf-ToR Pairing

With this feature, you can:

  • Configure a specific vPC/port-channel ID range for leaf-ToR pairing by enabling the Use Specific vPC/Port-Channel ID Range field. NDFC then displays the vPC/Port-Channel ID Range field with the recommended vPC/port-channel ID range.

  • Edit a vPC/port-channel ID for paired switches by clicking the Action > Edit Pairing option on the TOR Pairing page.

Configure Fabric Settings for Specifying a vPC/Port-Channel ID Range for Leaf-ToR Pairing

  1. Create a fabric with the Data Center VXLAN EVPN fabric and add ToR switches to the fabric.

    For more information, see the section "Creating a VXLAN EVPN Fabric Using the Data Center VXLAN EVPN Template" in Data Center VXLAN EVPN and the section "Adding Switches to a Fabric" in Add Switches for LAN Operational Mode.

    note.svg

    You need to use the same version of Cisco NX-OS when configuring your vPC switches.

  2. On the Fabric Overview page, choose the Data Center VXLAN EVPN fabric that you want to edit, and click Actions > Edit Fabric.

  3. Click on the vPC tab.

  4. Check the Use Specific vPC/Port-Channel ID Range check box to use a specific vPC/port-channel ID range for leaf-ToR pairing.

    The vPC/Port-Channel ID Range field displays the recommended values.

    The recommended values are from 1-499.

    note.svg

    You can increase the existing range or add more ranges if the values are exhausted.

  5. Specify a range for the vPC/Port-Channel ID Range field if you do not want to use the recommended values.

  6. Click Save.

    The new range applies to the new pairing.

Create Leaf-ToR Pairings

For configuring leaf-ToR pairings, see Configuring ToR Switches and Deploying Networks in Data Center VXLAN EVPN Fabrics > Configuring ToR Switches.

Edit the Leaf and the ToR Port-Channel IDs

  1. On the Fabric Overview > Switches page, choose the leaf switch you want to edit and click Actions > TOR Pairing.

    The TOR Pairing page appears with a horizontal bar of the paired leaf switches.

  2. Click Edit Pairing under the Action column.

    The leaf page displays.

    The Enable <switch-name> Pairing as TOR Pairing check box is checked due to auto leaf-ToR pairing.

  3. Click the arrow on the right-hand corner of the page to view the fields.

  4. In the Leaf Port Channel ID field, add or edit the existing ID value.

  5. In the vPC ID field, add or edit the existing ID value.

  6. In the ToR Port Channel ID field, add or edit the existing ID value.

  7. Click Save.

    note.svg

    If you have overlays attached to the paired switches, you cannot change the vPC/port-channel IDs.

  8. Navigate to the Fabric Overview > Switches page and click Actions > Recalculate and Deploy.

    The Deploy Configuration page displays with the list of leaf switches.

    After successful deployment, the Fabric Status column displays as In-Sync.

Deploying Networks on ToR Switches

To deploy networks on ToR switches in the Data Center VXLAN EVPN fabric, perform the following steps:

  1. Choose Manage > Fabrics, then double-click on the Data Center VXLAN EVPN fabric.

  2. On the Networks page, select the networks that you want to deploy or create a new network. For information about creating a network, see the section "Creating Network for Standalone Fabrics" in Data Center VXLAN EVPN.

  3. On the Fabric Overview page, click Networks > Network Attachments.

  4. Select the leaf switches you want to edit and click on Actions > Edit.

    The Edit Network Attachment page appears.

  5. On the Edit Network Attachment page, choose Attach.

  6. (Optional) You can enter a value in the VLAN field when you use Actions > Create to create a network, but not when you edit a network using Actions > Edit.

  7. If a leaf is in a vPC pair, you can select interfaces/ports on a leaf switch and/or associated ToR(s), attach the ports, and click Save.

    Port channels that are used to connect the ToR(s) toward the leaf switch(es) or vPC pairs are automatically updated with the required VLAN deployed in the server interfaces of the ToR switch.

  8. Select the leaf switch and click on Actions > Deploy.

Configuring ToR Switches and Deploying Networks in External Fabrics

Overview

NDFC supports Top-of-Rack (ToR) switches. You can add the Layer 2 ToR switches in an external fabric, and the ToR switches can be connected to the leaf switches in a Data Center VXLAN EVPN fabric. Typically, you connect the leaf and ToR switches with a back-to-back vPC connection. For more information, see Supported Topologies for ToR Switches.

Supported Topologies for ToR Switches

The following topologies with ToR switches are supported in NDFC:

note.svg

Cisco Nexus 7000 Series Switches do not support the ToR switch role in Cisco NDFC.

441213.jpg
ToR switches with back-to-back vPC connection to the leaf switches
441214.jpg
ToR switches with port channels connected to both the leaf switches. The L1 and L2 switches are connected as a vPC pair.
441215.jpg
ToR switches with port channels directly connected to the leaf switches. The L1 and L2 switches are connected as a vPC pair.
441216.jpg
ToR switches with port channels directly connected to the leaf switches. vPC pairs are not configured for the leaf or ToR switches.

Unsupported Topologies for ToR Switches

The following topology with ToR switches is not supported in NDFC:

tor-unsupported.jpg

Configuring ToR Switches

Before you begin, make sure you have a Data Center VXLAN EVPN or create and deploy a new fabric. For more information, see the section "Creating a VXLAN EVPN Fabric Using the Data Center VXLAN EVPN Template" in Data Center VXLAN EVPN.

note.svg

NDFC supports the trunk_host policies for ToR switches. Make sure the ToR switch has a vPC and port channel policy attached on the interfaces connected to the leaf. These policies are used to connect the ToR switches in the external fabric to the leaf switches in the Data Center VXLAN EVPN fabric.

  1. Create a Data Center VXLAN fabric and add two ToR switches.

    The number of ToR switches can be more than two.

    This procedure shows how to configure ToR switches as shown in the ToR Topology-1, where ToR switches are connected using a vPC.

    The following are the different scenarios for connecting the ToR switches:

    • If a vPC is not configured on the ToR switches, then the vPC policy needs to be applied on the ToR-facing interfaces, if the uplinks for these ToR switches are connected to vPC leaf switches.

    • If ToR switches are connected to a leaf using a port channel, then port-channel policies need to be applied on the ToR interfaces connected to the leaf switches.

    • If ToR switches are connected to leaf switches as standalone, trunk policies need to be applied on the TOR interfaces.

      note.svg

      • While creating the Data Center VXLAN EVPN fabric, make sure that the Fabric Monitor Mode check box is not chosen.

      • The two ToR switches must be connected and have the same switch role.

    After adding the ToR switches, make sure that the role for the ToR switches is ToR.

  2. Select one of the ToR switches and click Actions > vPC Pairing.

  3. Choose the second ToR switch as a vPC peer.

  4. Under the vPC Pair Template, you can choose the No Policy option. Perform a Recalculate and Deploy to generate a vPC configuration for the ToR switches.

    For more information about all the fields and their descriptions, see the section "Creating a vPC Setup" in External Connectivity Network.

    note.svg

    Steps 2 and 3 are required since this example shows the ToR configuration for Topology-1. For Topology-2, -3, and -4, steps 2 and 3 are not required.

  5. On the Switch Overview page, click Actions > Recalculate and Deploy.

  6. After the configuration is completed on the Deploy Configuration page, click Close.

  7. Create a VXLAN EVPN Multi-Site fabric.

    While creating the VXLAN EVPN Multi-Site fabric, under the General tab, choose the ToR Auto-deploy Flag check box.

    This action enables automatic deployment of the networks and VRFs in the Data Center VXLAN EVPN fabric to the ToR switches in the VXLAN EVPN Multi-Site fabric when you click Recalculate and Deploy in the VXLAN EVPN Multi-Site fabric. For more information, see Deploying Networks on ToR Switches.

    For more information about the remaining tabs and fields, see the section "Creating a VXLAN EVPN Multi-Site and Associating Member Fabrics" in VXLAN EVPN Multi-Site.

  8. Open the VXLAN EVPN Multi-Site fabric. Navigate to Child Fabrics and click on Actions to move the fabric into a VXLAN EVPN Multi-Site fabric.

  9. Select the Data Center VXLAN EVPN fabric and click Actions > Interface.

  10. Choose vPC and enter all the relevant details and click Save.

    For more information about the fields on this page, see the section "Adding Interfaces" in Add Interfaces for LAN Operational Mode.

    After saving all the information, click Deploy.

    Follow the same steps to create a vPC on the ToR switch as well.

Deploying Networks on ToR Switches

To deploy networks on ToR switches in the external fabrics, you need to deploy them on the switches in the Data Center VXLAN EVPN through VXLAN EVPN Multi-Site. These switches should be connected to the ToR switches.

  1. Choose Manage > Fabrics, then double-click on the Data Center VXLAN EVPN fabric.

  2. On the Networks page, select the networks that you want to deploy or create a new network. For more information about creating a network, see the section "Creating Network for Standalone Fabrics" in Data Center VXLAN EVPN.

  3. Select the Network from the Network Attachment page. Click on Actions and Edit. Attach the network and select the appropriate interface/port-channels and then click on Save. These port channels connect the leaf switches to the ToR switches. The networks are deployed on these port channels.

  4. On the Fabric Overview page, click Actions > Recalculate and Deploy.

    Now the VLANs are deployed on the leaf switches.

  5. Navigate to the VXLAN EVPN Multi-Site fabric.

  6. On the Fabric Overview page, click Actions > Recalculate and Deploy.

    The networks created and deployed on the leaf switches in the Data Center VXLAN EVPN fabric are also deployed on the ToR switches in the external fabric. This step allows the same VLANs to be configured on the ToR switches that are deployed on the leaf switches in Step 4.

    note.svg

    If VLANs are created on the ToR switches manually using the freeform configs, they are not modified.

Copyright

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.

THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.

The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.

NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.

IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

© 2017-2024 Cisco Systems, Inc. All rights reserved.

Need help?