Introduction
This document describes how to recover a Multilayer Datacenter Switch (MDS) from Boot Loader Prompt and Boot Mode.
Prerequisites
Requirements
Cisco recommends that you have a knowledge of these topics:
- MDS9000 CLI configuration commands
- How to set up a console connection to MDS switch.
- How to set up or connect MDS switch to a network via its management port.
- Familiarity with TFTP.
Components Used
Information in this document is based on the software requirement release 6.2(9) and later. This document is using an MDS9148S switch, switchname tempest that supports 12 to 48 line-rate 16 Gbps Fibre Channel ports.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Background Information
In order to function correctly, MDS switches require a valid non-corrupted Kickstart and System image to boot up completely. Invalid, corrupted, or missing Kickstart images prevent a switch to boot up and stops at the Boot Loader Prompt. A switch with a valid Kickstart image, but an invalid or missing System Image, ends up in Boot Mode.
Switch Boot Up Sequence
BIOS > Loader > Kickstart Image > System Image
- The Basic Input Output System (BIOS) loads the Loader.
- The Loader loads the Kickstart Image into RAM and starts the Kickstart Image.
- The Kickstart Image loads and starts the System Image.
- The System Image reads the Startup Configuration file.
Switch Prompt
Phase
|
Prompt
|
Description
|
BIOS
|
Non Bootable
|
BIOS begins the power on self test and memory test.
|
Loader
|
loader>
|
Boot loader uncompresses loaded software to boot a kickstart image.
Ctrl + C (Control Key and C Key) to enter boot loader prompt.
|
Kickstart
|
switch(boot)#
|
When the boot loader phase is over, press Ctrl + ] (Control key and Right Bracket key) to enter the boot mode.
|
System
|
log in:
|
The system image loads the startup configuration file and returns a switch to log in prompt.
|
Recovery Procedure
From switch console session.
loader>
Step 1. Download the switch’s Kickstart and System images, and put them into the TFTP server directory.
In this example, TFTP server IP address is 10.104.204.100.
Step 2. Configure the switch management IP address, subnet mask, and its default gateway.
Check config options in loader prompt using help option
loader> help
help - print online help
boot - Boot image
cmdline - <arguments>
dir - List Directory
help - print online help
network - Set network communication settings
reload - Reboot the board.
serial - Set serial console settings
show - Show loader configuration
loader> net --ip=10.66.78.80 --nm=255.255.255.128 --gw=10.66.78.4
In this example:
- The switch interface mgmt0 IP address is 10.66.78.80 255.255.255.128
- The switch default gateway IP address is 10.66.78.4
OR
The IP assignment synatx in loader prompt varies slightly for switch HW models.
Example listed here in case of T&V-series Fabric switches.
loader> set ip 10.66.78.80 255.255.255.128
loader> set gw 10.66.78.4
Note: Step 2 config required if you are booting the switch for the first time (for the RMA switch) or the switch has been write erase boot.
Step 3. Boot the Kickstart image which is located in the TFTP server directory, in order to bring the switch to the boot mode.
loader> boot tftp://10.104.204.100/m9100-s5ek9-kickstart-mz.8.3.1.bin
eTSEC2: Speed 1000, full duplex
Using eTSEC2 device
TFTP from server 10.104.204.100; our IP address is 10.66.78.80; sending through gateway 10.66.78.4
Filename 'm9100-s5ek9-kickstart-mz.8.3.1.bin'.
Load address: 0x8000000
Loading: *#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#######
done
Bytes transferred = 26027008 (18d2400 hex)
NBI at 08000000 size 26027008
Booting image at addr 0x00800000 ...
Memory <- <0x0 0x0 0x1 0x0> (4096MB)
ethernet0: local-mac-address <- 10:05:ca:b6:c5:a0
ethernet1: local-mac-address <- 00:e0:0c:00:01:fd
ethernet2: local-mac-address <- 00:e0:0c:00:02:fd
CPU clock-frequency <- 0x3f941f80 (1067MHz)
CPU timebase-frequency <- 0x3f941f8 (67MHz)
CPU bus-frequency <- 0x1fca0fc0 (533MHz)
zImage starting: loaded at 0x00800000 (sp: 0x7fedc620)
Allocating 0x620d88 bytes for kernel ...
gunzipping (0x00000000 <- 0x00817000:0x00de3838)...done 0x5bc060 bytes
Using loader supplied ramdisk at 0x2800000-0x3de8e00
initrd head: 0x1f8b0808
Linux/PowerPC load: rw root=/dev/ram0 rdbase=0x7000000 card_index=9046 maxcpus=2 ip=off ramdisk_size=262144 noquiet obfl_type_ide=1 kgdboc=ttyS0,9600,B console=ttyS0,9600n8nn loader_ver="02.01.17" card_index=9046 quiet bootdev=mgmt0 server_ip=64.104.204.100 ksimg=m9100-s5ek9-kickstart-mz.8.3.1.bin isanimg= Finalizing device tree... flat tree at 0xdf0140
Ísetup_arch: bootmem
mpc85xx_ds_setup_arch()
arch: exit
[ 0.060378] Host controller irq 26
[ 0.103390] Assign root port irq 26
[ 0.144046] Host controller irq 24
[ 0.193691] Assign root port irq 24
[ 0.810630] physmap-flash physmap-flash.0: Could not reserve memory region
[ 1.084751] Enabling all PCI devices
INIT: Checking all filesystems..r.r.r.retval=[1]
r done
WARNING: Image sync is going to be disabled after a loader netboot.
Loading system software
No system image is specified
/etc/init.d/functions: li
INIT: Sending processes the KILL signal
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2018, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
switch(boot)#
Step 4. Recover the switch from the boot mode.
1. Check IP config in switch boot prompt.
switch(boot)# sh ip
ip routing is disabled
Management Interface: ip address 10.66.78.80/25
ip default-gateway 10.66.78.4
If IP Config not found, then configure the interface management IP address and default gateway IP address again as shown.
switch(boot)# config t
Enter configuration commands, one per line. End with CNTL/Z.
switch(boot)(config)# interface mgmt 0
switch(boot)(config-if)# ip address 10.66.78.80 255.255.255.128
switch(boot)(config-if)# no shut
switch(boot)(config-if)# exit
switch(boot)(config)# ip default-gateway 10.66.78.4
switch(boot)(config)# exit
-
Test IP connectivity between TFTP server and the switch (ping response is not available in loader prompt).
Ping the switch management IP address from the TFTP server.
C:\Users\mpandika> ping 10.66.78.80
Pinging 10.66.78.80 with 32 bytes of data:
Reply from 10.66.78.80: bytes=32 time=2ms TTL=60
Reply from 10.66.78.80: bytes=32 time=3ms TTL=60
Reply from 10.66.78.80: bytes=32 time=3ms TTL=60
Reply from 10.66.78.80: bytes=32 time=3ms TTL=60
Ping statistics for 10.66.78.80:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 2ms, Maximum = 3ms, Average = 2ms
C:\Users\mpandika>
- Download the Kickstart and System images from the TFTP server to the switch’s bootflash.
switch(boot)# copy tftp://10.104.204.100/m9100-s5ek9-kickstart-mz.8.3.1.bin bootflash:
Trying to connect to tftp server......
Connection to server Established. Copying Started.....
TFTP get operation was successful
Copy complete, now saving to disk (please wait)...
switch(boot)#
switch(boot)#copy tftp://10.104.204.100/m9100-s5ek9-mz.8.3.1.bin bootflash:
Trying to connect to tftp server......
Connection to server Established. Copying Started.....
TFTP get operation was successful
Copy complete, now saving to disk (please wait)...
switch(boot)#
4. Verify the Kickstart and System images are successfully downloaded to the switch’s bootflash.
switch(boot)# dir bootflash:
4096 Dec 22 2002 01:38:31 .partner/
297 Nov 09 2017 07:18:54 MDS201711091818530370.lic
297 Nov 09 2017 07:28:42 MDS201711091828418770.lic
4096 Jul 23 2018 14:44:25 lost+found/
26027008 Aug 08 2018 02:55:31 m9100-s5ek9-kickstart-mz.8.3.1.bin
109635731 Aug 08 2018 02:53:38 m9100-s5ek9-mz.8.3.1.bin
3430222 Feb 01 2000 02:31:30 moonlake.1.f.xsvf
23467 Jul 30 2018 12:40:56 mts.log
347607 Jul 28 2017 04:46:31 portchannel
4096 Dec 22 2002 02:38:54 scripts/
31812 Jun 30 2018 05:07:27 snmp.pcap
Usage for bootflash: filesystem
1264746496 bytes used
2106781696 bytes free
3371528192 bytes total
switch(boot)#
5. Load the System image.
switch(boot)# load bootflash:m9100-s5ek9-mz.8.3.1.bin
Uncompressing system image: bootflash:/m9100-s5ek9-mz.8.3.1.bin
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
Load plugins that defined in image conf: /isan/plugin_img/img.conf
No Patching support on this platform
Loading plugin 0: core_plugin...
No Patching support on this platform
Enter pboot_chk_compatibility
num srgs 1
0: swid-core-s5ek9m, swid-core-s5ek9m
num srgs 1
0: swid-sup-mini-ks, swid-sup-mini-ks
INIT: Switching to runlevel: 3
INIT: Sending processes the TERM signal
switch(boot)#
INIT:
2018 Aug 8 03:13:03 %SYSLOG-2-SYSTEM_MSG : Syslogs wont be logged into logflash until logflash is online
2018 Aug 8 03:13:06 %KERN-0-SYSTEM_MSG: [ 1.084751] Enabling all PCI devices - kernel
2018 Aug 8 03:14:13 switch %CARDCLIENT-2-REG: OK
System is coming up ... Please wait ...
System is coming up ... Please wait ...
2018 Aug 8 03:14:22 switch %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Bootstrap Replay Started.
System is coming up ... Please wait ...
User Access Verification
10.66.78.80 login: 2018 Aug 8 03:14:41 switch %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Bootstrap Replay Done.
2018 Aug 8 03:14:44 switch %PMON-SLOT1-2-PMON_CRIT_INFO: Port Monitor Critical Information: Config download success .
2018 Aug 8 03:14:45 switch %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Ascii Replay Started.
2018 Aug 8 03:14:45 switch %PMON-SLOT1-2-PMON_CRIT_INFO: Port Monitor Critical Information: Policy (slowdrain) activation is successfull .
2018 Aug 8 03:14:46 switch %PLATFORM-2-PS_OK: Power supply 1 ok(Serial number QCS18242L8D)
2018 Aug 8 03:14:46 switch %PLATFORM-2-PS_FANOK: Fan in Power supply 1 ok
2018 Aug 8 03:14:46 switch %PLATFORM-2-PS_FAIL: Power supply 2 failed or shut down(Serial number QCS18242L0H)
2018 Aug 8 03:14:46 switch %PLATFORM-2-FAN_OK: Fan module ok
2018 Aug 8 03:14:46 switch %PLATFORM-2-FAN_OK: Fan module ok
2018 Aug 8 03:14:46 switch %PLATFORM-2-FAN_OK: Fan module ok
2018 Aug 8 03:14:46 switch %PLATFORM-2-FAN_OK: Fan module ok
2018 Aug 8 14:15:35 tempest %ASCII-CFG-2-CONFIG_REPLAY_STATUS: Ascii Replay Done.
User Access Verification
tempest login:
Note: Switch was previously configured with switchname tempest.
6. Log in to the switch.
tempest login: admin
Password:
Last login: Wed Aug 8 14:20:01 on pts/0
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2018, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
tempest#
7. Verify the boot variables.
tempest# show boot
Current Boot Variables:
kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.1.1.bin
system variable = bootflash:/m9100-s5ek9-mz.8.1.1.bin
Boot POAP Disabled
No module boot variable set
Boot Variables on next reload:
kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.1.1.bin
system variable = bootflash:/m9100-s5ek9-mz.8.1.1.bin
Boot POAP Disabled
No module boot variable set
tempest#
8. Set the boot variables to the correct image.
tempest# show version
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Documents: http://www.cisco.com/en/US/products/ps9372/tsd_products_support_series_home.html
Copyright (c) 2002-2018, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained herein are owned by
other third parties and are used and distributed under license.
Some parts of this software are covered under the GNU Public
License. A copy of the license is available at
http://www.gnu.org/licenses/gpl.html.
Software
BIOS: version 2.1.17
loader: version N/A
kickstart: version 8.3(1)
system: version 8.3(1)
BIOS compile time: 01/08/14
system image file is: bootflash:///m9100-s5ek9-mz.8.3.1.bin
system compile time: 7/30/2018 12:00:00 [07/12/2018 19:32:46]
Hardware
cisco MDS 9148S 16G 48 FC (1 Slot) Chassis ("2/4/8/16 Gbps FC/Supervisor")
Motorola, e500v2 with 4088556 kB of memory.
Processor Board ID JAE18250A8C
Device name: tempest
bootflash: 4018896 kB
Kernel uptime is 0 day(s), 0 hour(s), 47 minute(s), 44 second(s)
Last reset at 362296 usecs after Wed Aug 8 13:02:31 2018
Reason: Reset Requested by CLI command reload
System version: 8.1(1)
Service:
plugin
Core Plugin
tempest# config t
Enter configuration commands, one per line. End with CNTL/Z.
tempest(config)# boot kickstart bootflash:m9100-s5ek9-kickstart-mz.8.3.1.bin
Performing image verification and compatibility check, please wait....
tempest(config)# boot system bootflash:m9100-s5ek9-mz.8.3.1.bin
Performing image verification and compatibility check, please wait....
tempest(config)#
9. Verify the set Boot Variables.
tempest# show boot
Current Boot Variables:
kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.3.1.bin
system variable = bootflash:/m9100-s5ek9-mz.8.3.1.bin
Boot POAP Disabled
No module boot variable set
Boot Variables on next reload:
kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.1.1.bin
system variable = bootflash:/m9100-s5ek9-mz.8.1.1.bin
Boot POAP Disabled
No module boot variable set
10. Save the switch’s running-config.
tempest# copy running-config startup-config
[########################################] 100%
Copy complete.
tempest#
11. Verify the set Boot Variables post copy running-config startup-config.
tempest# show boot
Current Boot Variables:
kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.3.1.bin
system variable = bootflash:/m9100-s5ek9-mz.8.3.1.bin
Boot POAP Disabled
No module boot variable set
Boot Variables on next reload:
kickstart variable = bootflash:/m9100-s5ek9-kickstart-mz.8.3.1.bin
system variable = bootflash:/m9100-s5ek9-mz.8.3.1.bin
Boot POAP Disabled
No module boot variable set