Introduction
This document describes several Inter-VSAN Routing (IVR) configuration scenarios with different Virtual Storage Area Network (VSAN) and Multilayer Data Switch (MDS) topologies.
IVR Configuration
IVRv1 configuration steps: (Deprecated)
IVR 1 (non-NAT) is deprecated from Nexus Operating System (NX-OS) Release 5.2(1) and later
IVR non-NAT mode is not supported in Cisco NX-OS Release 5.2(x).
If you have IVR non-NAT mode configured, see the “Upgrading Guidelines Specific to NX-OS Release 5.2(1)”
section for instructions on how to migrate to IVR NAT mode
- Make sure Fibre Channel (FC) domain IDs are unique
- Enable IVR
- Turn on Cisco Fabric Services (CFS) for IVR
- Create IVR VSAN topology
- Activate IVR VSAN topology
- Create IVR zones
- Crate IVR zoneset
- Activate IVR zoneset
- Commit IVR configuration
IVRv2 configuration steps: (preferred)
- Enable IVR
- Enable IVR NAT
- Turn on CFS for IVR
- Create IVR VSAN topology
- Activate IVR VSAN topology
- Create IVR zones
- Create IVR zoneset
- Activate IVR zoneset
- Commit IVR configuration
Cisco recommends to use IVR2 with User-Configure-Topology
Example to configure Scenario 2:
Host 1 wwpn: 21:00:00:e0:8b:1f:fe:d9 vsan 10
Storage 1 wwpn: 21:00:00:04:cf:8c:53:13 vsan 20
MDS 1 switch wwpn: 20:00:00:0d:ec:01:ca:40 vsan 10, 500
MDS 2 switch wwpn: 20:00:00:0d:ec:07:ae:c0 vsan 20, 500
IVR VSAN topology database:
AFID1: MDS1 - VSAN 10,500
AFID1: MDS2 – VSAN 20,500
MDS1(config)# ivr enable
MDS1(config)# ivr nat
MDS1(config)# ivr distribute
MDS2(config)# ivr enable
MDS2(config)# ivr nat
MDS2(config)# ivr distribute
MDS1(config)# ivr vsan-topology database
MDS1(config-ivr-topology-db)# autonomous-fabric-id switch-wwn 20:00:00:0d:ec:01:ca:40 vsan-range 10,500
MDS1(config-ivr-topology-db)# autonomous-fabric-id switch-wwn 20:00:00:0d:ec:07:ae:c0 vsan-range 20,500
MDS1(config)# ivr vsan-topology activate
MDS1(config)# ivr commit
MDS1(config)# ivr zone name ivr_zone1
MDS1(config-ivr-zone)# member pwwn 21:00:00:e0:8b:1f:fe:d9 vsan 10
MDS1(config-ivr-zone)# member pwwn 21:00:00:04:cf:8c:53:13 vsan 20
MDS1(config)# ivr zoneset name IVR_ZONESET1
MDS1(config-ivr-zoneset)# member ivr_zone1
MDS1(config)# ivr zoneset activate name IVR_ZONESET1
MDS1(config)# ivr commit
MDS1# show ivr vsan-topology active
AFID SWITCH WWN Active cfs. VSAN Switch-Name
------------------------------------------------------------------------------------------------------------------------
1 20:00:00:0d:ec:01:ca:40* yes yes 10,500
1 20:00:00:0d:ec:07:ae:c0 yes yes 20,500
MDs1# show ivr zoneset active
Zoneset name IVR_ZONESET1
Zone name ivr_zone1
*pwwn 21:00:00:e0:8b:1f:fe:d9 vsan 10 autonomous-fabric-id 1
*pwwn 21:00:00:04:cf:8c:53:13 vsan 20 autonomous-fabric-id 1
MDS1# show zoneset active vsan 10
Zoneset name nozoneset vsan 10
Zone name IVRZ_ivr_zone1 vsan 10
*fcid 0x0b0000 [pwwn 21:00:00:e0:8b:1f:fe:d9]
*fcid 0x1600ab [pwwn 21:00:00:04:cf:8c:53:13]
IVR Scenario 1
VSAN 10 and VSAN 20 are on MDS1. Host in VSAN 10 would like to use Storage in VSAN 20. MDS 1 is a border switch.
IVR VSAN topology database:
AFID1 MDS1 VSAN 10,20
IVR Scenario 2
Host in VSAN 10 would like to use Storage in VSAN 20.
Transit VSAN 500 is used between MDS switches.
MDS 1 and MDS 2 are border switches
IVR VSAN topology database:
AFID1: MDS1 - VSAN 10,500
AFID1: MDS2 – VSAN 20,500
IVR Scenario 3
Host 1 and Host 2 in VSAN 10 would like to use Storage 1 in VSAN 20
Both VSANs are configured in both switches
MDS1 is not running IVR
MDS1 is an edge switch
MDS2 is a border switch
IVR VSAN topology database:
AFID1: MDS2 - VSAN 10,20
IVR Scenario 4
Host 1 and Host 2 in VSAN 10 would like to use Storage 1 in VSAN 20
Host 1 will use VSAN 10 Inter-Switch Link (ISL) to cross to MDS 2 then use transit VSAN 500 to go from VSAN 10 to VSAN 20
IVR VSAN topology database:
AFID1: MDS1 - VSAN 10,500
AFID1: MDS2 - VSAN 10, 20, 500
IVR Scenario 5
Host 1 in VSAN 10 would like to use Storage 1 in VSAN 20.
Two transit VSAN 500 and 600 in the path
IVR VSAN topology database:
AFID1: MDS1 - VSAN 10,500
AFID1: MDS2 - VSAN 500, 600
AFID1: MDS3 - VSAN 20, 600
IVR Scenario 6
Host 1 in VSAN 10 would like to user Storage 1 in VSAN 20.
Transit VSAN 500 with two parallel paths are used
IVR VSAN topology database:
AFID1: MDS1 - VSAN 10,500
AFID1: MDS2 - VSAN 500, 600
IVR Scenario 7
Host 1 in VSAN 10 would like to user Storage 1 in VSAN 20.
Two parallel transits VSAN 500 and VSAN 600. Fabric Shortest Path First (FSPF) costs are the same
Only one transit VSAN can be used, no load balancing.
IVR1: The first available transit VSAN is chosen regardless of its FSPF cost. If the chosen transit VSAN ISL is down, IVR will automatically failover to the other transit VSAN. No load balancing between transit VSANs.
IVR2: The first transit VSAN 500 is chosen permanently. If the VSAN 500 ISL is down, IVR path will be down. IVR will never attempt to use or failover to the other transit VSAN 600. No load balancing between transit VSANs.
IVR VSAN topology database:
AFID1: MDS1 - VSAN 10, 500, 600
AFID1: MDS2 - VSAN 20, 500, 600
IVR Version 1
- All FC Domains across all VSANs must be unique.
- The source and destination FCISs of FC frames exchange between Host 1 and Storage 1 are unchanged.
- Only the VSAN is rewritten in the EISL frame header.
- Command: ivr internal vsan-rewrite-list vsan XX
IVR VSAN topology database:
AFID1: MDS1 - VSAN 10, 500
AFID1: MDS2 - VSAN 20, 500
IVR Version 2
- IVR NAT must be explicitly enabled with “ivr nat” command on each IVR enabled switch
- IVR NAT allows for overlapping Domain IDs
- IVR NAT allows for overlapping VSAN IDs
- FC NAT always rewrites S_ID and D_ID in the FC frames
- Domain from a remote VSAN is represented in a local VSAN with a virtual FCID
- Command: ivr internal vsan-rewrite-list vsan XX
IVR VSAN topology database:
AFID1: MDS1 - VSAN 10, 500
AFID1: MDS2 - VSAN 20, 500
Overlapping VSANs
- More that on AFID can be used after SAN-OS 2.1(1a)
- Multiple AFID allow overlapping VSAN IDs
- AFID can be between 1 – 64
- Each unique VSAN is defined by AFID/VSAN pair
- Example below, VSAN 10 is not trunked between MDS 1 and MDS 2
IVR VSAN topology database:
AFID1: MDS1 - VSAN 10
AFID2: MDS1 - VSAN 500
AFID2: MDS2 - VSAN 500
AFID3: MDS2 - VSAN 10