Catalyst 4500 Series Switch Software Configuration Guide, Release IOS XE 3.3(0)XO

Related Documents

Stateful Switchover

Development of the SSO feature is an incremental step within an overall program to improve the availability of networks constructed with Cisco IOS XE switches.

In specific Cisco networking devices that support dual supervisor engines, SSO takes advantage of supervisor engine redundancy to increase network availability. SSO achieves this by establishing one of the supervisor engines as the active processor while the other supervisor engine is designated as the standby processor. Following an initial synchronization between the two supervisor engines, SSO dynamically synchronizes supervisor engine state information between them in real-time.

A switchover from the active to the standby supervisor engine occurs when the active supervisor engine fails or is removed from the networking device.

Cisco NSF is used with SSO. Cisco NSF allows the forwarding of data packets to continue along known routes while the routing protocol information is being restored following a switchover. With Cisco NSF, peer networking devices do not experience routing flaps, which reduce loss of service outages for customers.

Figure 5-1 illustrates how SSO is typically deployed in service provider networks. In this example, Cisco NSF with SSO is enabled at the access layer (edge) of the service provider network. A fault at this point could result in loss of service for enterprise customers requiring access to the service provider network.

For Cisco NSF protocols that require neighboring devices to participate in Cisco NSF, Cisco NSF-aware software images must be installed on those neighboring distribution layer devices. Depending on your objectives, you may decide to deploy Cisco NSF and SSO features at the core layer of your network. Doing this can help reduce the time required to restore network capacity and service for certain failures, which leads to additional availability.

Figure 5-1 Cisco NSF with SSO Network Deployment: Service Provider Networks

Additional levels of availability may be gained by deploying Cisco NSF with SSO at other points in the network where a single point of failure exists. Figure 5-2 illustrates an optional deployment strategy that applies Cisco NSF with SSO at the enterprise network access layer. In this example, each access point in the enterprise network represents another single point of failure in the network design. In the event of a switchover or a planned software upgrade, enterprise customer sessions would continue uninterrupted through the network in this example.

Figure 5-2 Cisco NSF with SSO Network Deployment: Enterprise Networks

For further information on SSO, see the Stateful Switchover document.

NSF

Cisco NSF works with the SSO feature in Cisco IOS XE software. SSO is a prerequisite of Cisco NSF. NSF works with SSO to minimize the amount of time a network is unavailable to its users following a switchover. The main objective of Cisco NSF is to continue forwarding IP packets following a supervisor engine switchover.

Usually, when a networking device restarts, all routing peers of that device detect that the device went down and then came back up. This transition results in what is called a routing flap, which could spread across multiple routing domains. Routing flaps caused by routing restarts create routing instabilities, which are detrimental to the overall network performance. Cisco NSF helps to suppress routing flaps in SSO-enabled devices, thus reducing network instability.

Cisco NSF allows for the forwarding of data packets to continue along known routes while the routing protocol information is being restored following a switchover. With Cisco NSF, peer networking devices do not experience routing flaps. Data traffic is forwarded while the standby supervisor engine assumes control from the failed active supervisor engine during a switchover. The ability of physical links to remain up through a switchover and to be kept current with the Forwarding Information Base (FIB) on the active supervisor engine is key to Cisco NSF operation.

ISSU Process

The ISSU process allows you to perform a Cisco IOS XE software upgrade or downgrade while the system continues to forward packets. (For an illustration of the commands used during the ISSU process, refer to Figure 5-8.) Cisco IOS XE ISSU takes advantage of the Cisco IOS XE high availability infrastructure—Cisco NSF with SSO and hardware redundancy—and eliminates downtime associated with software upgrades by allowing changes while the system remains in service (see ).

SSO and NSF mode support configuration and runtime state synchronization from the active to the standby supervisor engine. For this process, the IOS XE software image on both the active and the standby supervisor engines must be the same. When images on active and standby supervisor engines are different, ISSU allows the two supervisor engines to be kept in synchronization even when these two versions of Cisco IOS XE support different sets of features and commands.

Figure 5-3 High Availability Features and Hardware Redundancy in the ISSU Process

An ISSU-capable switch consists of two supervisor engines (active and standby) and 200 or more linecards. Before initiating the ISSU process, copy the Cisco IOS XE software into the file systems of both supervisor engines (see Figure 5-4).

Note In the following figure, Cisco IOS XE 3.x.y SG represents the current version of Cisco IOS XE 3.z.y SG represents the image you are migrating to.

Figure 5-4 Copy New Version of Cisco IOS XE Software on Both Supervisor Engines

After you have copied the Cisco IOS XE software to both file systems, load the new version of Cisco IOS XE software onto the standby supervisor engine (see Figure 5-5).

Note Without the ISSU feature, SSO/NSF cannot function between the active and standby supervisor engines when they are running different versions of the Cisco IOS XE image.

Figure 5-5 Load New Version of Cisco IOS XE Software on the Standby Supervisor Engine

After a switchover (NSF/SSO, not RPR), the standby supervisor engine takes over as the new active supervisor engine (see ).

Figure 5-6 Switch Over to Standby Supervisor Engine

The former active supervisor engine is loaded with an old Cisco IOS XE image so that if the new active supervisor engine experiences problems, you can abort and conduct a switchover to the former active, which is already running the old software image. Next, the former active supervisor engine is loaded with the new version of Cisco IOS XE software and becomes the new standby supervisor engine (see ).

Figure 5-7 Load New Standby Supervisor Engine with New Cisco IOS XE Software

shows the steps during the ISSU process.

Figure 5-8 Steps During the ISSU Process

Note Accept version stops the rollback timer.

Performing an ISSU Upgrade: 2 Methods

There are two ways to perform an ISSU upgrade: manually, with four commands; or automatically, with one command.

The normal ISSU upgrade process involves issuing four separate ISSU exec commands
(issu loadversion, issu runversion, issu acceptversion, issue commitversion) along with additional show command invocations to evaluate the success of each command before proceeding. Although the ISSU process is complicated, you should not expect disruption of service. The use of multiple ISSU commands dictates an additional level of care to ensure no service disruption. However, in some scenarios, this upgrade procedure might be cumbersome and of minimal value. A typical example is during a network upgrade that involves performing an ISSU upgrade on a large number of Catalyst 4500 switches. In these cases, we recommend that you first perform the normal (four command) ISSU upgrade procedure on one switch (possibly in a lab environment) to verify successful upgrade. Then, use a single issu changeversion command to perform an automatic ISSU on the rest of the Catalyst 4500 switches in the network.

Changeversion Process

The issu changeversion command launches a single-step complete ISSU upgrade cycle. It performs the logic for all four of the standard commands (issu loadversion, issu runversion, issu acceptversion, and issu commitversion) without user intervention, streamlining the upgrade through a single CLI step.

Additionally, issu changeversion allows the upgrade process to be scheduled for a future time. This enables you to stage a number of systems to perform upgrades sequentially when a potential disruption would be least harmful.

After the standby supervisor engine initializes and the system reaches a terminal state (RPR/SSO), the upgrade process is complete and the BOOT variable is permanently written with the new IOS XE software image. Hence, a reset on any RP will keep the system booting the new software image. Console and syslog messages will be generated to notify anyone monitoring the upgrade that the state transition has occurred.

Similar to the normal ISSU upgrade procedure, the in-progress upgrade procedure initiated by the
issu changeversion command can be aborted with the issu abortversion command. If the system detects any problems or detects an unhealthy system during an upgrade, the upgrade might be automatically aborted.

When the issu runversion command is entered during the four step manual upgrade process, if any incompatible ISSU clients exist, the upgrade process reports them and their side effects, and allows the user to abort the upgrade. While performing a single-step upgrade process, when the process reaches the runversion state, it will either automatically continue with the upgrade provided the base clients are compatible, or automatically abort because of client incompatibility. If the user wants to continue the upgrade procedure in RPR mode, the user must use the normal ISSU command set and specify the force option when entering the issu loadversion command.

Changeversion: Quick Option (LV to INIT)

The issu changeversion command provides an optional quick command option that can reduce the time required to perform the automatic ISSU upgrade. When the quick command option is applied, the ISSU upgrade state transition differs from that described previously. With this option, the software logic up the loadversion stage remains the same as previously described, and the logic that performs runversion and commitversion is combined. This logic skips the step in the upgrade procedure that loads the old software version on the new standby (old active) supervisor, reducing the time required for the automatic ISSU upgrade by about a third.

Scheduled Changeversion: “in” and “at” Options

issu changeversion provides in and at command options that enable you to schedule a future automatic ISSU upgrade.

The at command option schedules an automatic ISSU upgrade to begin at a specific time. This option specifies an exact time (hh:mm, 24 hour format) in the next 24 hours at which the upgrade will occur.

The in command option schedules an automatic ISSU upgrade to begin after a certain amount of time has elapsed. This option specifies the number of hours and minutes (hh:mm format) that must elapse before an upgrade will occur, with a maximum value of 99:59.

Changeversion Deployment Scenario

The typical issu changeversion command usage scenario is for experienced users with a large installed base. These users typically validate a new image using a topology and configuration similar to their production network. The validation process should be done using both the existing multi-command process and the new issu changeversion command process. Once users certify an IOS XE software image and want to roll it out broadly, they can use the single command process to perform an efficient upgrade of their network.

Aborting an In-Progress Changeversion Procedure

The issu changeversion command functionality is designed to perform an ISSU software upgrade without user intervention. However, status messages are displayed to the console as the upgrade transitions through the various states. If any anomalies are noticed during the automatic upgrade, perhaps with peers or other parts of the network, you can use the issu abortversion command to manually abort the upgrade at any point in the process prior to the commitversion operation.

Guidelines for Performing ISSU

Be aware of the following guidelines while performing the ISSU process:

• Even with ISSU, it is recommended that upgrades be performed during a maintenance window.
• The new features should not be enabled (if they require change of configuration) during the ISSU process.

Note Enabling them will cause the system to enter RPR mode because commands are only supported on the new version.

• In a downgrade scenario, if any feature is not available in the downgrade revision of the Cisco IOS XE software handle, that feature should be disabled prior to initiating the ISSU process.

Compatibility Matrix

ISSU requires additional information to determine compatibility between software versions. Therefore, a compatibility matrix is defined that contains information about other IOS XE software image with respect to the one in question.

This compatibility matrix represents the compatibility of two software versions, one running on the active and the other on the standby supervisor engine, and to allow the system to determine the highest operating mode it can achieve. Incompatible versions will not be able to progress to SSO operational mode.

The compatibility matrix represents the compatibility relationship a Cisco IOS XE software image has with all of the other Cisco IOS XE software versions within the designated support window (for example, all of those software versions the IOS XE software image “knows” about) and is populated and released with every IOS XE software image. The matrix stores compatibility information between its own release and prior releases. It is always the newest release that contains the latest information about compatibility with existing releases in the field. The compatibility matrix is available within the Cisco IOS XE software image and on Cisco.com so that users can determine in advance whether an upgrade can be done using the ISSU process.

You can perform the ISSU process when the old and new Cisco IOS XE software are compatible. The compatibility matrix information stores the compatibility among releases as follows:

• Compatible—The base-level system infrastructure and all optional HA-aware subsystems are compatible. An in-service upgrade or downgrade between these versions will succeed with minimal service impact. The matrix entry designates the images to be compatible (C).
• Base-level compatible—One or more of the optional HA-aware subsystems is not compatible. An in-service upgrade or downgrade between these versions will succeed; however, some subsystems will not be able to maintain state always during the transition from the old to the new version of Cisco IOS XE. The matrix entry designates the images to be base-level compatible (B).
• Incompatible—A core set of system infrastructure exists in Cisco IOS XE that must be able to interoperate in a stateful manner for SSO to function correctly. If any of these required features or subsystems is not interoperable, then the two versions of the Cisco IOS XE software image are declared to be incompatible. An in-service upgrade or downgrade between these versions is not possible. The matrix entry designates the images to be incompatible (I). The system operates in RPR mode during the upgrade process when the versions of Cisco IOS XE at the active and standby supervisor engines are incompatible.
• Cisco IOS XE determines the compatibility between the active and the standby IOS XE software dynamically during STANDBY boot up. The matrix is represented by “x”.

To display the compatibility matrix data between two software versions on a given system, enter the show issu comp-matrix stored command.

Note This command is useful only for verification purposes because it is available only after the ISSU process has started. You might want to check the compatibility matrix prior to starting ISSU. Use the Feature Navigator to obtain the needed information:

http://tools.cisco.com/ITDIT/CFN/jsp/index.jsp

SNMP Support for ISSU

SNMP for SSO provides a mechanism for synchronizing the SNMP configurations and the MIBs that support SSO from the active supervisor engine to the standby supervisor engine.

Compatibility Verification Using Cisco Feature Navigator

The ISSU application on Cisco Feature Navigator allows you to:

• Select a specific software bundle.
• Identify which software images are compatible with the selected software image.
• Compare two IOS XE software images and understand the compatibility level of the software images (that is, compatible, base-level compatible, and incompatible), or dynamically determined.
• Compare two software images and see the client compatibility for each ISSU client.
• Provide links to release notes for the software image.

Verifying the ISSU Software Installation

During the ISSU process, there are five valid states: disabled, init, load version, run version, and system reset. Use the show issu state command to obtain the current ISSU state:

• Disabled state—The state for the standby supervisor engine while this engine is resetting.
• Init state—The initial state for two supervisor engines, one active and one standby, before the ISSU process is started. It is also the final state after the ISSU process completes.
• Load version (LV) state—The standby supervisor engine is loaded with the new version of Cisco IOS XE software.
• Run version (RV) state—The issu runversion command forces the switchover of the supervisor engines. The newly active supervisor engine runs the new Cisco IOS XE software image.
• While running ISSU, if both supervisor engines are reset due to power outage, for example, the ISSU context is lost and returns to the init state. Both supervisor engines return to the old software.

You can verify the ISSU software upgrade by entering show commands to provide information on the state of the during the ISSU process:

This example shows how to display the state and the current status of the supervisor engine during the ISSU process:

Switch> enable

Switch# show issu state

Switch# show redundancy

Verifying Redundancy Mode Before Beginning the ISSU Process

Before you begin the ISSU process, verify the redundancy mode for the system and be sure to configure NSF and SSO.

The following example displays verification that the system is in SSO mode, that slot 3 is the active supervisor engine, and that slot 4 is the standby supervisor engine. Both supervisor engines are running the same Cisco IOS XE software release.

Verifying the ISSU State Before Beginning the ISSU Process

Ensure that the active and standby supervisor engines are up and in ISSU Init state and that both supervisor engines are running the same current image.

The following example displays the ISSU state before the process begins:

The new version of the Cisco IOS XE software must be present on both of the supervisor engines. The directory information displayed for each of the supervisor engines shows that the new version is present.

Loading New Cisco IOS XE Software on the Standby Supervisor Engine

This task describes how to use ISSU to load a new version of Cisco IOS XE software to the standby supervisor engine.

Prerequisites

• Ensure that the new version of Cisco IOS XE software image is already present in the file system of both the active and standby supervisor engines. Also ensure that appropriate boot parameters (BOOT string and config-register) are set for the active and standby supervisor engines.

Note auto-boot must be enabled for ISSU to succeed.

• Optionally, perform additional tests and commands to determine the current state of peers and interfaces for later comparison.
• Ensure the system (both active and standby supervisor engines) is in SSO redundancy mode. If the system is in RPR mode, you can still upgrade the system using the ISSU CLI commands, but the system will experience extended packet loss during the upgrade.'

Refer to the Stateful Switchover document for more details on how to configure SSO mode on supervisor engines.

• For ISSU to function, the IOS XE file names on the active and standby supervisor engines must match.

Perform the following steps at the active supervisor engine:

This example shows how to start the ISSU process, boot the standby supervisor engine in the Standby Hot state, and load the standby supervisor engine slot (6) with the new IOS XE software image:

The following example shows how the forced option places the system in RPR mode:

The following example shows the redundancy mode as RPR:

Switching to the Standby Supervisor Engine

This task describes how to switchover to the standby supervisor engine, which is running the new Cisco IOS XE software image. Perform the following steps at the active supervisor engine.

This example shows how to cause a switchover to the former standby supervisor engine (slot 6), reset the former active supervisor engine and reload it with the old IOS XE software image so it becomes the standby supervisor engine:

A switchover happens at this point. At the new active supervisor engine, do the following after old active supervisor engine comes up as standby.

Note The new active supervisor engine is now running the new version of software, and the standby supervisor engine is running the old version of software and is in the standby hot state.

Once Runversion has completed, the new active supervisor engine will be running the new version of software and the previously active supervisor engine will now become the standby supervisor engine. The standby will be reset and reloaded, but it will remain on the previous version of software and come back online in standbyhot status. The following example shows how to verify these conditions:

Stopping the ISSU Rollback Timer (Optional)

This optional task describes how to stop the rollback timer.

If you do not run the following procedure before the rollback timer “timeout,” the system automatically aborts the ISSU process and reverts to the original Cisco IOS XE software version. By default the rollback timer is 45 minutes.

Use the following information to decide what action you should take:

• If you want to retain your switch in the runversion state for an extended period, you need to stop the rollback timer by entering the acceptversion command. Then validate the new software and enter the commitversion command (as described in the following section).
• If you want to proceed with the commitversion operation within the rollback timer window of 45 minutes (or the configured value), you do not need to use the acceptversion command to stop the roll-back timer.

Note The issu acceptversion command may be optionally executed after the issu runversion command.

This example displays the Timer before you stop it. In the following example, the “Automatic Rollback Time” information indicates the amount of time remaining before an automatic rollback will occur.

Loading New Cisco IOS XE Software on the New Standby Supervisor Engine

This task explains how to load new version of Cisco IOS XE software to the new standby supervisor engine.

Perform the following steps at the active supervisor engine:

This example shows how to reset and reload the current standby supervisor engine (slot 1) with the new Cisco IOS XE software version. After you enter the commitversion command, the standby supervisor engine boots in the Standby Hot state.

Use the following to verify that the standby supervisor engine is reloaded with the new software.

The ISSU process has completed. At this stage, any further Cisco IOS XE software version upgrade or downgrade will require that a new ISSU process be invoked.

Using changeversion to Automate an ISSU Upgrade

This task describes how to use the issu changeversion command to perform a one step ISSU upgrade.

Prerequisites

• Ensure that the new version of Cisco IOS XE software image is already present in the file system of both the active and standby supervisor engines. Also ensure that appropriate boot parameters (BOOT string and config-register) are set for the active and standby supervisor engines
• Optionally, perform additional tests and commands to determine the current state of peers and interfaces for later comparison.
• Ensure the system (both active and standby supervisor engines) is in SSO redundancy mode. If the system is in RPR mode, you can still upgrade the system using the ISSU CLI commands, but the system will experience extended packet loss during the upgrade.'

Refer to the Stateful Switchover document for more details on how to configure SSO mode on supervisor engines (refer to Chapter 8, “Configuring Supervisor Engine Redundancy Using RPR and SSO on Supervisor Engine 8-E”).

• For ISSU to function, the IOS XE software image file names on the active and standby supervisor engines must match.

Perform the following steps at the active supervisor engine:

This example shows how to initiate an ISSU upgrade process using the issu changeversion command on slot number 5, the slot for the current active supervisor engine. The show issu state detail and show redundancy command output is included to show the supervisor state before and after the upgrade procedure.

Note The success messages included in the output below is displayed after some delay because the ISSU upgrade procedure progresses through the ISSU states.

Note Standby reloads with target image.

Note Switchover occurs.

Look at the console of new active supervisor engine.

Note The new standby supervisor engine reloads with target image; changeversion is successful upon SSO terminal state is reached.

This example shows how to use issu changeversion with the at command option to schedule an ISSU upgrade procedure to automatically start at the specified time. This example specifies that the ISSU upgrade should be started at 16:30 (24 hour format). The show issu state detail and show redundancy command output is included to show the supervisor state before and after the issu changeversion command was entered.

Aborting a Software Upgrade During ISSU

You can abort the ISSU process at any stage manually (prior to entering the issu commitversion command) by entering the issu abortversion command. The issu abortversion command may also be issued after entering the issu changeversion command while the automatic ISSU upgrade is still in progress. The ISSU process also aborts on its own if the software detects a failure.

Note If you enter the issu abortversion command before the standby supervisor engine becomes hot, the traffic might be disrupted.

If you abort the process after you issue the issu loadversion command, the standby supervisor engine is reset and reloaded with the original software.

If the process is aborted after you enter either the issu runversion or issu acceptversion command, then a second switchover is performed to the new standby supervisor engine that is still running the original software version. The supervisor engine that had been running the new software is reset and reloaded with the original software version.

Note Ensure that the standby supervisor is fully booted before issuing the abortversion command on an active supervisor engine.

The following task describes how to abort the ISSU process before you complete the ISSU process with the issu commitversion command.

Perform the following task on the active supervisor engine.

This example shows how to abort the ISSU process on slot number 6, the slot for the current active supervisor engine. In this example, the ISSU upgrade process is in the Runversion state when the issu abortversion command is entered:

Configuring the Rollback Timer to Safeguard Against Upgrade Issues

Cisco IOS XE software maintains an ISSU rollback timer, to safeguard against an upgrade that may leave the new active supervisor engine in a state in which communication with the standby supervisor engine is severed.

You may want to configure the rollback timer to fewer than 45 minutes (the default) so that the user need not wait in case the new software is not committed or the connection to the switch was lost while it was in runversion mode. A user may want to configure the rollback timer to more than 45 minutes in order to have enough time to verify the operation of the new Cisco IOS XE software before committing the new software image.

Note The valid timer value range is from 0 to 7200 seconds (two hours). A value of 0 seconds disables the rollback timer.

Once you are satisfied that the new image at the active supervisor engine has been successful and you want to remain in the current state, you may indicate acceptance by issuing the issu acceptversion command, which stops the rollback timer.

Issuing the issu commitversion command at this stage is equal to entering both the issu acceptversion and the issu commitversion commands. Use the issu commitversion command if you do not intend to run in the current state for a period of time and are satisfied with the new software version.

Note The rollback timer can be configured only in the ISSU Init state.

This task explains how to configure the rollback timer:

This example shows how to set the rollback timer to 3600 seconds:

Switch> enable

Switch# configure terminal

Enter configuration commands, one per line. End with CNTL/Z.

Switch(config)# issu set rollback-timer 3600

% Rollback timer value set to [ 3600 ] seconds

Switch(config)# exit

Switch# show issu rollback-timer

Rollback Process State = Not in progress

Configured Rollback Time = 60:00

The Rollback Timer cannot be set in loadversion or runversion state, as the following example illustrates:

Displaying ISSU Compatibility Matrix Information

The ISSU compatibility matrix contains information about other IOS XE software releases and the version in question. This compatibility matrix represents the compatibility of the two software versions, one running on the active and the other on the standby supervisor engine, and the matrix allows the system to determine the highest operating mode it can achieve. This information helps the user identify whether to use ISSU.

This task shows how to display information about the ISSU compatibility matrix.

This example shows how to display negotiated information regarding the compatibility matrix:

This example shows how to display stored information regarding the compatibility matrix:

Dynamic Image Version Compatibility (DIVC) feature is supported in IOS XE releases. With DIVC, we store Dynamic(0) rather than Incomp(1), Base(2), or Comp(3), and determine compatibility during run-time when two different DIVC-capable IOS XE software images are running in the active and standby supervisor engines during ISSU.

For Catalyst 4500 switches, a value of Dynamic(0) in the stored compatibility-matrix normally results in Base(2) or Comp(3) upon run-time negotiation between the two IOS XE software images; as of today, you never observe Incomp(1) as long as the other IOS XE name is present in the stored compatibility-matrix.

This example shows how to display negotiated information regarding non-IOSd clients:

ISSU (IOS In-Service Software Upgrade)

http://www.cisco.com/en/US/partner/products/ps7149/products_ios_protocol_group_home.html

Enhanced High System Availability

http://www.cisco.com/en/US/docs/ios-xml/ios/ha/configuration/xe-3s/ha-config-stateful-switchover.html