Configuring Infra for Cisco DCNM Sites

Prerequisites and Guidelines

The following sections describe the steps necessary to configure the general as well as site-specific fabric Infra settings.

Before you proceed with Infra configuration, you must have added the sites as described in previous sections.

In addition, keep in mind the following:

  • Adding or removing border gateway switches requires a Multi-Site Orchestrator fabric connectivity information refresh described in the Refreshing Site Connectivity Information as part of the general Infra configuration procedures.

Configuring Infra: General Settings

This section describes how to configure general Infra settings for all the sites.

Procedure

Step 1

Log in to the Cisco Multi-Site Orchestrator GUI.
Step 2

In the left navigation menu, select Infrastructure > Infra Configuration.

Step 3

In the main pane, click Configure Infra.

Step 4

In the left sidebar, select General Settings.

Step 5

Configure control plane BGP.

  1. Choose BGP Peering Type.

    • full-mesh—All border gateway switches in each site will establish peer connectivity with remote sites' border gateway switches.

    • route-server—The route-server option allows you to specify one or more control-plane nodes to which each site establishes MP-BGP EVPN sessions. The route-server nodes perform a function similar to traditional BGP route-reflectors, but for EBGP (and not iBGP) sessions. The use of route-server nodes avoids creating MP-BGP EVPN full mesh adjacencies between all the VXLAN EVPN sites managed by MSO.

  2. If you set the BGP Peering Type to route-server, click +Add Route Server to add one or more route servers.

In the Add Route Server window that opens:

  • From the Site dropdown, select the site you want to connect to the route server.

  • The ASN field will be auto-populated with the site's ASN.

  • From the Core Router Device dropdown, select the route server to which you want to connect.

  • From the Interface dropdown, select the interface on the core router device.

You can add up to 4 route servers. If you add multiple route servers, every site will establish MP-BGP EVPAN adjacencies to every route server.

  1. In the Keepalive Interval (Seconds) field, enter the keep alive interval seconds.

    We recommend keeping the default value.

  2. In the Hold Interval (Seconds) field, enter the hold interval seconds.

    We recommend keeping the default value.

  3. In the Stale Interval (Seconds) field, enter stale interval seconds.

    We recommend keeping the default value.

  4. Choose whether you want to turn on the Graceful Helper option.

  5. In the Maximum AS Limit field, enter the maximum AS limit.

  6. In the BGP TTL Between Peers field, enter the BGP TTL between peers.

Step 6

Configure DCNM settings.

  1. Provide the L2 VXLAN VNI Range.

  2. Provide the L3 VXLAN VNI Range.

  3. Provide the Multi-Site Routing Loopback IP Range.

    This field is used to auto-populate the Multi-Site TEP field for each fabric, which is described in Configuring Infra: DCNM Site Settings.

    For sites that were previously part of a Multi-Site Domain (MSD) in DCNM, this field will be pre-populated with the previously defined value.

  4. Provide the Anycast Gateway MAC.

Refreshing Site Connectivity Information

Infrastructure changes, such as adding and removing border gateway switches, require a Multi-Site Orchestrator fabric connectivity refresh. This section describes how to pull up-to-date connectivity information directly from each site's controller.

Procedure

Step 1

Log in to the Cisco Multi-Site Orchestrator GUI.
Step 2

In the left navigation menu, select Infrastructure > Infra Configuration.

Step 3

In the main pane, click Configure Infra.

Step 4

In the left sidebar, under Sites, select a specific site.

Step 5

In the main window, click the Refresh button to pull fabric information from the controller.

Step 6

(Optional) In the Confirmation dialog, check the box if you want to remove configuration for decommissioned border gateway switches.

If you choose to enable this checkbox, all configuration info for any currently decommissioned border gateway switches will be removed from the database.

Step 7

Finally, click Yes to confirm and load the connectivity information.

This will discover any new or removed spines and all site-related fabric connectivity will be re-imported from the site's controller.

Configuring Infra: DCNM Site Settings

This section describes how to configure site-specific Infra settings for on-premises sites.

Procedure

Step 1

Log in to the Cisco Multi-Site Orchestrator GUI.
Step 2

In the left navigation menu, select Infrastructure > Infra Configuration.

Step 3

In the main pane, click Configure Infra.

Step 4

In the left pane, under Sites, select a specific DCNM.

Step 5

In the right <Site> Settings sidebar, specify the Multi-Site TEP.

This address is used for the inter-site L2 BUM and L3 multicast traffic. This IP address is deployed on all border gateway switches that are part of the same fabric.

Note 

If the site you are configuring is part of the DCNM Multi-Site Domain (MDS), this field will be pre-populated with the information imported from DCNM. In this case, changing the value and re-deploying the infra configuration, will impact traffic between the sites that are part of the MDS.

You can choose to Auto Allocate this field, which will allocate the next available address from the Multi-Site Routing Loopback IP Range you defined in previous section.

Step 6

Within the <fabric-name> tile, select the border gateway.

Step 7

In the right <border-gateway> setting sidebar, specify the Control Plane TEP and Data Plane TEP.

Step 8

Click Add Port to configure the port that connects to the IPN.

Note 

This release does not support importing the port configuration from the DCNM. If the site you are configuring is already part of the DCNM Multi-Site Domain (MDS), you must use the same values that are already configured in DCNM.

Provide the following information specific to your deployment for the port that connects this border gateway to a core switch or another border gateway:

  • From the Ethernet Port ID dropdown, select the port that connects to the IPN.

  • In the IP Address field, enter the IP address and netmask.

  • In the Remote Address field, provide the IP address of the remote device to which the port is connected.

  • In the Remote ASN field, provide the remote site's ID.

  • In the MTU field, enter the port's MTU.

    MTU of the spine port should match MTU on IPN side.

    You can specify either inherit or a value between 576 and 9000.

  • For BGP Authentication, you can pick either None or Simple (MD5).

    If you select Simple, you must also provide the Authentication Key.

Deploying Infra Configuration

This section describes how to deploy the Infra configuration to each DCNM site.

Before you begin

You must have the general and site-specific infra configurations completed as described in previous sections of this chapter.

Procedure

Step 1

Ensure that there are no configuration conflicts or resolve them if necessary.

The Deploy button will be disabled and a warning will be displayed if there are any configuration conflicts from the already configured settings in each site. For example, if a VRF or network with the same name exists in multiple sites but uses different VNI in each site.

In case of configuration conflicts:

  1. Click Click to View link in the conflict notification pop-up.

  2. Note down the specific configurations that are causing the conflicts.

    For example, in the following report, there are ID mismatches between VRFs and networks in fab1 and fab2 sites.

  3. Click the X button to close the report, then exit Infra configuration screen.

  4. Unmanage the site in MSO, as described in Removing Sites.

    You do not need to remove the site from the Nexus Dashboard, simply unmanage it in MSO GUI.

  5. Resolve the existing configuration conflicts.

  6. Manage the site again, as described in Adding Cisco DCNM Sites.

    Since the site is already added in Nexus Dashboard, simply enable it for management in MSO.

  7. Verify that all conflicts are resolved and the Deploy button is available.

Step 2

Deploy configuration.

  1. In the top right of the Fabric Connectivity Infra screen, choose the appropriate Deploy option to deploy the configuration.

    If you are configuring only DCNM sites, simply click Deploy to deploy the Infra configuration.

  2. Wait for configuration to be deployed.

    When you deploy infra configuration, MSO will signal the DCNM to configure the underlay and the EVPN overlay between the border gateways.

    When configuration is successfully deployed, you will see a green checkmark next to the site in the Fabric Connectivity Infra screen: