Contents

Introduction

Prerequisites

Requirements

Components Used

Conventions

Background Information

Why Appliance Port VLANs Should be Allowed on Uplinks

Definition of a Unified Storage Port

Appliance Port Port-Channel

When to Use Trunk or Access Mode

Situations to Avoid

Appliance Port Failover

Network Uplink Failure

Appliance Port Troubleshooting

Related Information

Introduction

This document is intended to assist Unified Computing Systems (UCS) administrators who configure direct attached storage on the Cisco UCS platform.

Contributed by Dmitri Filenko and Andreas Nikas, Cisco TAC Engineers.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

This document is not restricted to specific software and hardware versions.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to Cisco Technical Tips Conventions for information on document conventions.

Background Information

UCS appliance ports are used to directly connect a storage appliance to UCS fabric interconnects.

An appliance port behave similar to virtual ethernet (vEthernet or vEth) ports:

- It contains a list of allowed virtual LANs (VLANs).

- MAC addresses on these interfaces are learned by the fabric interconnect.

- It requires an uplink for pinning.

When a VLAN is created for an appliance port in the Appliances section of LAN tab, make sure to create the same VLAN in the LAN Cloud tab.

Why Appliance Port VLANs Should be Allowed on Uplinks

There are number of reasons why an upstream switch should allow storage appliance port traffic. These include:

- If storage must be accessed outside of the UCS domain.

- If storage and servers are located in different subnets.

- If storage is configured in Active/Passive mode and both fabric interconnects require communication to the same controller.

- In certain failover scenarios.

Definition of a Unified Storage Port

Prior to UCS Release 2.1(1a), appliance ports only worked for IP-based storage traffic, such as the Network File System (NFS) and the Internet Small Computer System Interfaces (iSCSIs). In UCS Releases 2.1(1a) and later, the ability for both IP-based storage and Fibre Channel over Ethernet (FCoE) to be used on the same interface was added. This type of interface is called a Unfied Storage port. In order to use this feature, the storage controller must have a Converged Network Adapter (CNA) that is capable of FCoE and traditional Ethernet on the same port.

Tip: For information about how to configure a Unified Storage port, refer to the Configuring an Appliance Port as a Unified Storage Port section of the Cisco UCS Manager GUI Configuration Guide, Release 2.1.

In order to verify that the port is configured as a unified storage port, login to the Cisco NX-OS shell and verify the port's current configuration with this command:

ucs01-A(nxos)# show running-config interface eth 1/5

interface Ethernet1/5

description AF: UnifiedStorage

...

Appliance Port Port-Channel

UCS supports Static and Link Aggregation Control Protocol (LACP) port channels for appliance port configuration. However, there is no virtual Port-Channel (vPC) support.

When to Use Trunk or Access Mode

The decision about whether you should configure your appliance ports in Trunk or Access mode depends on the capabilities of the storage appliance. If the storage appliance has the capability to add VLAN tags, then Cisco recommends that you configure the appliance ports in Trunk mode and configure the VLAN tagging on the storage side for maximum flexibility. In this case, multiple VLANs can be used on the same link, which allows for isolation of different protocols on the wire. If the storage controller is not capable of VLAN tagging, then the use of an access port is required.

Situations to Avoid

- Configuration of the VLAN tagging on both the storage side and on the UCS sidesimultaneously.

Referred to as double VLAN tagging, this configuration breaks communication over the VLAN. If a VLAN tag is added in the appliance port configuration on the UCS side (done by configuring the appliance port in either Access mode or when using the Native VLAN while in Trunk mode), then do not configure tagging for the same VLAN on the storage controller side.

- Use of the same VLAN for multiple storage protocol traffic

As a best practice, each protocol should be placed into a separate VLAN. For example, when you access both the NFS Shares and the iSCSI LUNs via the same appliance port, configure the appliance port in Trunk mode with two different VLANs allowed across the link (one for the NFS and another for the iSCSI).
Note: The native VLAN on the appliance port is not required in this configuration.

Appliance Port Failover

Failover cannot be configured in the UCS side for appliance ports. By design, UCS fabric interconnects operate as two independent fabrics. Failover must be configured on the storage side and must be implemented with the correct network design while specific failover behavior for storage controllers are kept in mind, dependent on the storage controller model.

Network Uplink Failure

With default behavior, appliance ports are shut down if the uplink that is pinned to it goes down.

In order to change this behavior, configure a Network Control Policy and set the action to be taken upon uplink failure to Warning. For more information, refer to the Network Control Policy section of the Cisco UCS Manager GUI Configuration Guide, Release 2.2.

Appliance Port Troubleshooting

In the majority of implementations, communication between the blade servers and storage controllers that are plugged into the appliance ports are over the same broadcast domain (Layer 2 of the Open Systems Interconnect (OSI) model). In order to verify that this Layer 2 communication works correctly, you must verify whether the UCS Fabric Interconnect has learned the MAC address of the storage controller on the appliance port and on the correct VLAN.

In order to check the MAC address table, log into the UCS CLI console, connect to the NX-OS shell,and verify the list of allowed VLANs on the appliance port. You can then view the MAC address entries for the VLAN that are to be used for communication with the appliance port.

Here are the commands and output for verification:

F340-31-14-UCS-2-A# connect nxos a

F340-31-14-UCS-2-A(nxos)# show run int eth 1/11

!! Command: show running-config interface Ethernet1/11

!! Time: Fri Mar 29 07:02:29 2013

version 5.0(3)N2(2.11b)

interface Ethernet1/11

description A: Appliance

no pinning server sticky

pinning server pinning-failure link-down

no cdp enable

switchport mode trunk

switchport trunk allowed vlan 170

F340-31-14-UCS-2-A(nxos)# show mac address-table vlan 170

Legend:

* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC

age - seconds since last seen,+ - primary entry using vPC Peer-Link

VLAN     MAC Address      Type      age     Secure NTFY    Ports

---------+-----------------+--------+---------+------+----+------------------

* 170      0025.b500.004f    static    0          F    F  Veth780

* 170      0025.b500.005f    static    0          F    F  Veth779

* 170      010a.84ff.e4fe    dynamic   0         F    F  Eth1/11

In the this output, there are two blade servers, Veth780 and Veth779, and the MAC address of the storage controller is learned on Eth1/11. These devices should be able to communicate with each other if there are no other configuration issues on the end devices.

If no MAC addresses are learned on the appliance port while the correct VLAN is specified, then return to the appliance port configuration and re-confirm the trunk configuration. Also, ensure that the communication link on the storage appliance is in Active mode in the case of an Active/Passive link configuration. You can also check the MAC address table on fabric interconnect B, dependent on the link that is active on the storage controller side.

When the MAC addresses of the server and the storage controller are learned on the fabric interconnect within the same VLAN, the fabric interconnect switches the traffic locally without the use of the upstream switches. At this juncture, you can use an Internet Control Message Protocol (ICMP) request (ping) in order to test the communication between the end points.

Related Information

- UCS Manager Release 2.1(1) Configuration Guide

- Cisco Unified Computing System (UCS) Storage Connectivity Options and Best Practices with NetApp Storage

- Technical Support & Documentation - Cisco Systems