Mechanisms

For topologies using transport gateways as routing hubs, or Multi-Region Fabric networks, Cisco Catalyst SD-WAN uses the following mechanisms to ensure that devices choose the same path between two endpoints, for traffic in both directions:

Related Topics

Translating OMP Metrics to BGP Attributes

Translating OMP Metrics to an OSPF Metric

Configure a Router to Translate OMP Metrics to BGP or OSPF Using a CLI Template

Monitor RIB Metric Translation

Advertising P1 Routes

Edge router ER11 advertises P1 routes. The process of re-originating these routes to border routers, and eventually to ER21 and ER22 occurs from left to right in the illustration. In the process, border routers assign affinity groups and derived affinity groups when re-originating routes.

Routers in the network choose preferred routes as follows:

• Outside the core region: According to affinity group preference

• In the core region: According to the lowest derived affinity group (dag) value

Advertising P2 Routes

Edge routers ER21 and ER22 advertise P2 routes. The process of re-originating these routes to border routers, and eventually to ER11occurs from right to left in the illustration. In the process, border routers assign affinity groups and derived affinity groups when re-originating routes.

Routers in the network choose preferred routes as follows:

• Outside the core region: According to affinity group preference

• In the core region: According to the lowest derived affinity group (dag) value

Result

The following figure shows that the result of the configuration is symmetric routing for flows between, in this example, PC10 and PC20:

Default Behavior

In this example, the edge routers and transport gateway routers operate as follows:

• ER11: Subscribes only to VRF1, and advertises prefix P1 in VRF1.

• ER 21: Subscribes only to VRF2, and advertises prefix P2 in VRF2.

• All of the transport gateway routers handle both VRF1 and VRF2 traffic, and therefore re-originate both P1 (in VRF1) and P2 (in VRF2) routes.

By default, the network provides VRF isolation, meaning that when a device advertises routes in various VRFs, Cisco SD-WAN Controllers filter the routes before providing them to other devices. Specifically, a Cisco SD-WAN Controller advertises a VRF x route only to devices subscribing to VRF x. So in this example, by default, ER11, which subscribes only to VRF1, would not receive P2 routes, which are advertised in VRF2. Similarly, ER21, which subscribes only to VRF2, would not receive P1 routes, which are advertised in VRF1.

Consequently, VRF isolation would prevent traffic flow between ER11 and ER21, which subscribe exclusively to different VRFs.

Route Leaking

Route leaking enables devices to advertise routes across VRFs by exporting ("leaking") a route from one VRF to another.

• Source VRF of a route: Original VRF of the route

• Current VRF of a route: VRF to which the route was exported

When advertising exported routes, routers keep track of the source VRF and the current VRF, so the background of each route is preserved. This point factors in to the logic explained below.

This example has the following route leaking configured:

• An inbound control policy for ER11 configures it to receive VRF1 routes and export them to VRF2. Result: ER11 advertises the P1 prefix in both VRF1 and VRF2 to its associated transport gateways, TGW11 and TGW12.

• An inbound control policy for ER21 configures it to receive VRF2 routes and export them to VRF1. Result: ER21 advertises the P2 prefix in both VRF2 and VRF1 to its associated transport gateways, TGW21 and TGW22.

As mentioned earlier, after the route leaking, devices track, for each route, the source VRF (where the route came from) and the current VRF (VRF to which it was leaked).

Calculating the Derived Affinity Group

A transport gateway device, as in this example, or a border router in a similar example, assigns a derived affinity group (dag) to a route that it is re-originating as follows:

1. If the originating router is configured with affinity group preference auto (see ER11 in the example), then the re-originating device (for example, TGW11) determines the dag according to its own (TGW11's) affinity group configuration, as follows:

   1. For the leaked route, consider its source VRF and current VRF. Choose the numerically lower of the two values. Call this x.

   2. Do one of the following

      • If the re-originating device only has a system-level affinity group, not VRF-specific affinity groups, then:

        Use the system-level affinity group number for the dag. Assign a dag of that number when re-originating the route.

      • If the re-originating device has a VRF-specific affinity group configured for VRF x described in step a, then:

        Use this VRF-specific affinity group number for the dag. Assign a dag of this number when re-originating the route.

2. If the originating router is not configured with affinity group preference auto (see ER21 in the example), then the re-originating device (for example, TGW21) must consider the affinity preference order configured on the originating device when determining the dag for re-originated routes, as follows:

   1. For the leaked route, consider its source VRF and current VRF. Choose the numerically lower of the two values. Call this x.

   2. Do one of the following

      • If the re-originating device only has a system-level affinity group, not VRF-specific affinity groups, then:

        Check the affinity group preference order of the originating device (see ER21). Determine the item number of where the system-level affinity group number occurs in the preference order (item 1, 2, 3, and so on, in the preference order list). Assign a dag of this item number when re-originating the route.

        In the example of TGW21 and ER21, determine where affinity group 2 occurs in the preference order of ER21, which is (1, 2). It is item 2 in the list. So assign a dag of 2 when re-originating the route.

      • If the re-originating device has a VRF-specific affinity group configured for VRF x described in step a, then:

        Using this VRF-specific affinity group, check the affinity group preference order of the originating device. Determine the item number of where the VRF-specific affinity group number occurs in the preference order (item 1, 2, 3, and so on, in the preference order list). Assign a dag of this item number when re-originating the route.

        Hypothetically, in the example, if TGW21, in addition to having a system-level affinity group of 2, also had a VRF-specific affinity group of 1 for VRF1, then when TGW21 received from ER21 a P2 route leaked to VRF1, it would consider the preference order of the originating device (ER21). In this hypothetical example with a VRF-specific affinity group of 1, for a route received from ER21, it would check where affinity group 1 occurs in the preference order of ER21, which is (1, 2). It is item 1 in the list. So TGW2 would assign a dag of 1 when re-originating the route.

Example

In the scenario described in the illustration, a route leaked from VRF2 to VRF1 has source VRF value of 2 and a current VRF value of 1. When a transport gateway re-originates this route, it assigns it a dag according to the number 1, which is the lower of the two VRF numbers. For example, if TGW12 is re-originating a route with a source VRF value of 1 and current VRF value of 2, it chooses 1, which is the lower of the two VRF numbers. It therefore calculates the dag according to VRF1. TGW12 has a system-level affinity group of 1 and a VRF-specific affinity group of 2 for VRF1. Since it is calculating the dag according to VRF1, it assigns the re-originated route a dag value of 2, taken from the VRF-specific affinity group.

To consider a hypothetical, if TGW12 had a system-level affinity group of 5 and a VRF1-specific affinity group of 7, then for a route with source VRF 1 and current VRF 2, TGW12 would assign a dag of 7, taken from the VRF-specific affinity group of 7 for VRF1.

Before You Begin

If you configure a router with an affinity preference order manually and also configure auto preference order, the auto preference order has priority for selecting the next hop.

However, the manually configured preference list is still useful for path filtering using the filter route outbound affinity-group preference command. For information about filtering out paths for routers that are not on the device’s affinity list, see Information About Router Affinity Groups and see the filter route outbound affinity-group preference command reference in the Cisco IOS XE SD-WAN Qualified Command Reference.

Configure a Router to Use Automatic Affinity Group Preference

1. From the Cisco SD-WAN Manager menu, choose Configuration > Templates.

2. Click Feature Templates.

3. Do one of the following:

   • To create a System template for a device, click Add Template, choose a device type, and click Cisco System.

   • To edit an existing System template, locate a System template in the table of existing feature templates, click … adjacent to the template, and choose Edit.

4. In the Affinity Group Preference Auto field, choose On.

5. Click Save if creating a new template, or Update if editing an existing template.

Example

The following example configures affinity group 1 for VRF1:

system
  affinity-per-vrf 1 vrf-range 1

The following example configures affinity group 4 for the VRF range 3 to 6:

system
  affinity-per-vrf 4 vrf-range 3-6

Note	For information about verifying the VRF-specific affinity group configuration, see Verify the VRF-Specific Affinity Group Configuration on a Router.

Before You Begin

For more information about using CLI templates, see CLI Add-On Feature Templates and CLI Templates. By default, CLI templates execute commands in global configuration mode.

If you configure a router with both affinity-group preference-auto and affinity-group preference list, the affinity-group preference-auto command has priority for selecting a next hop.

However, the affinity-group preference list command is still useful for path filtering using the filter route outbound affinity-group preference command. For information about filtering out paths for routers that are not on the device’s affinity list, see Information About Router Affinity Groups and see the filter route outbound affinity-group preference command reference in the Cisco IOS XE SD-WAN Qualified Command Reference.

Configure a Router to Use Automatic Affinity Group Preference

1. Enter system configuration mode.

   system

2. Configure automatic affinity group preference.

   affinity-group preference-auto

Example

system
  affinity-group preference-auto

Example 1

This example applies to a scenario in which the underlay network uses BGP.

router bgp 1
  address-family ipv4 vrf 2
    redistribute omp translate-rib-metric
    propagate-aspath

Example 2

This example applies to a scenario in which the underlay network uses OSPF.

router ospf 1 vrf 1
  redistribute omp translate-rib-metric

Example 3

This example applies to a scenario in which the underlay network uses OSPFv3 IPv4.

router ospfv3 1
  address-family ipv4 vrf 1
    redistribute omp translate-rib-metric

Example 4

This example applies to a scenario in which the underlay network uses OSPFv3 IPv6.

router ospfv3 1
  address-family ipv6 vrf 1
    redistribute omp translate-rib-metric

Example

Router#show sdwan omp routes 10.1.1.0/24

Example

Device#traceroute vrf 1 10.1.1.1 numeric
Type escape sequence to abort.
Tracing the route to 10.1.1.1
VRF info: (vrf in name/id, vrf out name/id)
  1 209.165.200.225 3 msec 1 msec 1 msec
  2 209.165.200.226 2 msec 1 msec 1 msec
  3 10.1.1.1 4 msec * 4 msec

Example

Device#show platform software sdwan rp active internal "omp daemon" | include Affinity
…
Affinity per VRF:

Affinity Group Number: 1 for VRF Range: 1-1
Affinity Group Number: 5 for VRF Range: 2-8

Example 1

The following example shows a control policy that matches VRF1 routes and exports them to VRF2, and matches VRF2 routes and exports them to VRF1.

sdwanController#show running-config policy control-policy
policy
 control-policy LEAK_1_TO_2
  sequence 1 
   match route 
    vpn-list VRF1
   !
   action accept
    export-to
     vpn 2
    !
   !
  !
  default-action accept
 !
 control-policy LEAK_2_TO_1
  sequence 1
   match route
    vpn-list VRF2
   !
   action accept
    export-to
     vpn 1
    !
   !
  !
  default-action accept
 !
!

Example 2

The following example shows the sites to which the two policies configured in the previous example are applied.

sdwanController#show running-config apply-policy
apply-policy
 site-list SL1100
  control-policy LEAK_1_TO_2 in
 !
 site-list SL1300
  control-policy LEAK_2_TO_1 in
 !
!

Example

In the following example, the derived affinity group is 2.

Device#show sdwan omp routes 192.168.1.0/24 detail
…
  preference       not set
  affinity group  None
  derived-affinity-group  2
  affinity-preference-order  None
  region-id          0
  br-preference      not set

OMP Route Metric for IPv4 Routes

To show the OMP route metric for each IPv4 route prefix that a transport gateway is redistributing, use the show ip route command on the transport gateway. The OMP route metric, which is 66, is shown in bold in the output, and the administrative distance is 251.

Router#show ip route vrf 1 omp

Routing Table: 1

      10.0.0.0/32 is subnetted, 1 subnets
m        10.10.10.10 [251/66] via 172.16.0.1, 00:09:15
…

OMP Route Metric for IPv6 Routes

To show the OMP route metric for each IPv6 route prefix that a transport gateway is redistributing, use the show ipv6 route command on the transport gateway. The OMP route metric, which is 66, is shown in bold in the output, and the administrative distance is 251.

Router#show ipv6 route vrf 1 omp
m   2001:DB8::/128 [251/66]
     via 172.16.0.1%default
…

OSPF Metric for IPv4 Routes

To show the OSPF metric that the transport gateway uses when distributing IPv4 routes to OSPF, use the show ip ospf command on the transport gateway. The OSPF metric, which is determined by the OMP route metric, is 66 in this example, and is shown in bold in the output.

Router#show ip ospf 1 rib redistribution
            OSPF Router with ID (192.168.0.1) (Process ID 1)

  Base Topology (MTID 0)

OSPF Redistribution
10.10.10.10/32, type 2, metric 66, tag 0, from OMP_AGENT Router
   via 172.16.0.1, unknown interface
…

OSPF Metric for IPv6 Routes

To show the OSPF metric that the transport gateway uses when distributing IPv6 routes to OSPF, use the show ospfv3 command on the transport gateway. The OSPF metric, which is determined by the OMP route metric, is 66 in this example, and is shown in bold in the output.

Router#show ospfv3 vrf 1  ipv6 rib redistribution
          OSPFv3 10 address-family ipv6 vrf 1 (router-id 192.168.0.1)
 
2001:DB8::/128, type 2, metric 66, tag 0, from omp
  via 172.16.0.1
…