General Troubleshooting

This chapter provides procedures for troubleshooting the most common problems encountered when operating the NCS 1004 chassis. To troubleshoot specific alarms, see the Alarm Troubleshooting chapter. If you cannot find what you are looking for, contact Cisco Technical Support (1 800 553-2447).

Validate and Troubleshoot Installation of Software Packages

Procedure

Step 1

show version

Displays the software version and details such as system uptime.

Example:

RP/0/RP0/CPU0:ios# show version     
Mon Nov 22 22:50:09.240 IST
Cisco IOS XR Software, Version 7.5.1
Copyright (c) 2013-2021 by Cisco Systems, Inc.

Build Information:
Built By     : xxxxxxxxx
Built On     : Sun Nov 21 23:25:35 PST 2021
Built Host   : iox-lnx-054
Workspace    : /auto/srcarchive15/prod/7.5.1/ncs1004/ws
Version      : 7.5.1
Location     : /opt/cisco/XR/packages/
Label        : 7.5.1

cisco NCS-1004 () processor
System uptime is 2 minutes

Step 2

show install repository

Displays a list of all the installed software packages on NCS 1004.

Example:

RP/0/RP0/CPU0:ios# show install repository      
Wed Oct 13 20:14:15.984 IST
6 package(s) in XR repository:
    ncs1004-mini-x-7.5.1
    ncs1004-mpls-2.0.0.0-r751
    ncs1004-k9sec-2.1.0.0-r751.x86_64
    ncs1004-xr-7.5.1
    ncs1004-mpls-te-rsvp-2.1.0.0-r751

Step 3

show install active

Displays a list of all the installed and active software packages on NCS 1004.

The following sample output displays active software packages.

Example:

RP/0/RP0/CPU0:ios# show install active     Mon Mar 11 07:31:12.302 UTC
Wed Oct 13 20:14:15.984 IST
Node 0/RP0/CPU0 [RP]
  Boot Partition: xr_lv19
  Active Packages: 5
    ncs1004-mini-x-7.5.1
    ncs1004-mpls-2.0.0.0-r751
    ncs1004-k9sec-2.1.0.0-r751.x86_64
    ncs1004-xr-7.5.1
    ncs1004-mpls-te-rsvp-2.1.0.0-r751

Step 4

show install committed

Displays a list of all committed software packages on NCS 1004.

The committed software packages are the software packages that are booted on an NCS 1004 reload. Committed packages are the packages that are persistent across reloads. If you install and activate a package, it remains active until the next reload. If you commit a package set, all packages in that set remain active across reloads until the package set is replaced with another committed package set.

The following sample output displays the committed software packages.

Example:

RP/0/RP0/CPU0:ios# show install committed     
Wed Oct 13 20:14:15.984 IST
Node 0/RP0/CPU0 [RP]
  Boot Partition: xr_lv19
  Committed Packages: 5
    ncs1004-mini-x-7.5.1
    ncs1004-mpls-2.0.0.0-r751
    ncs1004-k9sec-2.1.0.0-r751.x86_64
    ncs1004-xr-7.5.1
    ncs1004-mpls-te-rsvp-2.1.0.0-r751

Step 5

show install log

Displays information on the history of the installation operations. This command provides information about both successful and failed installation operations on NCS 1004. You can also verify a Service Maintenance Update (SMU) installation using this command.

Example:

RP/0/RP0/CPU0:ios# show install log 49 detail
Wed Dec  9 01:19:18.680 UTC
Dec 09 01:19:07 Install operation 49 started by root:
 install add source tftp://10.105.236.167 ncs1004-k9sec.rpm
Dec 09 01:19:08 Action 1: install add action started
Dec 09 01:19:08 ERROR! Either file is not proper or error in getting rpm metadata from rpm file

Dec 09 01:19:08 ERROR!! failed to complete install add precheck

Dec 09 01:19:09 Install operation 49 aborted
Dec 09 01:19:10 Ending operation 49

In the above example, either a wrong rpm package is used or the rpm package is corrupted.

For failure on install add source, check that the package is correctly named and is available at the location.

What to do next

If the expected active software packages are not displayed, install the packages (if required) and activate the packages using the install activate package_name command.

Troubleshoot the Management Interface

Before you begin

Management interface should be configured.

Procedure

Step 1

show interfaces mgmtEth instance

Displays the management interface configuration.

Example:

RP/0/RP0/CPU0:ios# show interfaces MgmtEth 0/RP0/CPU0/0
Fri Nov 13 19:42:29.716 UTC
MgmtEth0/RP0/CPU0/0 is administratively down, line protocol is administratively down
  Interface state transitions: 0
  Hardware is Management Ethernet, address is badb.adba.d098 (bia badb.adba.d098)
  Internet address is 10.58.227.183/24
  MTU 1514 bytes, BW 100000 Kbit (Max: 100000 Kbit)
     reliability 255/255, txload 0/255, rxload 0/255
  Encapsulation ARPA,
  Full-duplex, 100Mb/s, CX, link type is autonegotiation
  loopback not set,
  ARP type ARPA, ARP timeout 04:00:00
  Last input never, output never
  Last clearing of "show interface" counters never
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     0 packets input, 0 bytes, 0 total input drops
     0 drops for unrecognized upper-level protocol
     Received 0 broadcast packets, 0 multicast packets
              0 runts, 0 giants, 0 throttles, 0 parity
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     0 packets output, 0 bytes, 0 total output drops
     Output 0 broadcast packets, 0 multicast packets
     0 output errors, 0 underruns, 0 applique, 0 resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions

  1. In the above result, the management interface is administratively down. Use the no shut command to enable the management interface.

    The following example shows sample output from the show running-config interface mgmtEth command when the management interface is in the no shut state. 

    RP/0/RP0/CPU0:ios#show running-config interface mgmtEth 0/RP0/CPU0/0
Fri Nov 13 19:42:54.368 UTC
interface MgmtEth0/RP0/CPU0/0
 ipv4 address dhcp
!

You can also use the show interfaces summary and show interfaces brief commands in the Cisco IOS XR EXEC mode to verify the management interface status.

  • The following example shows sample output from the show interfaces summary command. 

    RP/0/RP0/CPU0:ios# show interfaces summary
Wed Mar  4 06:14:52.995 UTC
Interface Type          Total    UP       Down     Admin Down
--------------          -----    --       ----     ----------
ALL TYPES               4        2        0        2
--------------
IFT_ETHERNET            3        1        0        2
IFT_NULL                1        1        0        0

  • The following example shows sample output from the show interfaces brief command. 

    RP/0/RP0/CPU0:ios# show interfaces brief
Wed Mar  4 06:15:51.689 UTC

               Intf       Intf        LineP    Encap  MTU        BW
               Name       State       State    Type (byte)    (Kbps)
--------------------------------------------------------------------------------
                Nu0          up          up    Null  1500          0
     Mg0/RP0/CPU0/0          up          up    ARPA  1514    1000000
     Mg0/RP0/CPU0/1  admin-down  admin-down    ARPA  1514    1000000
     Mg0/RP0/CPU0/2  admin-down  admin-down    ARPA  1514    1000000

Step 2

When the line protocol is down, you must verify the Layer 3 connectivity. You can perform the following steps.

  1. Check the Ethernet cable connection and physical connectivity of NCS 1004 to get the line protocol up.

  2. Ensure ARP connectivity.

  3. Use the ping command to check reachability and network connectivity on the IP network.

  4. Verify the static IP and default gateway configuration.

Troubleshoot Environmental Parameters

Some of the common environmental problems are listed below.

  • Fan failure

  • Fan not detected

  • Fan speed problem

  • Power module fails

  • Power module not detected

  • Temperature of the device exceeds a threshold value

  • Voltage of the device exceeds a threshold value

Procedure

Step 1

admin

Enters system admin EXEC mode.

Example:

 RP/0/RP0/CPU0:ios# admin

Step 2

show environment [ all | fan | power | voltages | current | temperatures ] [ location | location]

Displays the environmental parameters of NCS 1004.

Example:

The following example shows sample output from the show environment command with the fan keyword. 

sysadmin-vm:0_RP0# show environment fanWed Mar  4  05:36:33.678 UTC+00:00
=============================================
                        Fan speed (rpm)
Location     FRU Type           FAN_0   FAN_1
---------------------------------------------
0/FT0        NCS1K4-FAN          7020    6930
0/FT1        NCS1K4-FAN          6780    6690
0/FT2        NCS1K4-FAN          6810    6720
0/PM0        NCS1K4-AC-PSU      25376   24352
0/PM1        NCS1K4-AC-PSU      11200   11232

The following example shows sample output from the show environment command with the temperatures keyword. 

sysadmin-vm:0_RP0# show environment temperatures location 0/RP0
Wed Mar  4  05:44:51.221 UTC+00:00
================================================================================
Location  TEMPERATURE                 Value   Crit Major Minor Minor Major  Crit
          Sensor                    (deg C)   (Lo) (Lo)  (Lo)  (Hi)  (Hi)   (Hi)
--------------------------------------------------------------------------------
0/RP0
          TEMP_LOCAL                     32    -10    -5     0    55    65    70
          TEMP_REMOTE1                   32    -10    -5     0    55    65    70
          TEMP_CPU_DIE                   31    -10    -5     0    75    80    90

The following example shows sample output from the show environment command with the power keyword. 

sysadmin-vm:0_RP0# show environment power
Wed Mar  4  05:45:35.640 UTC+00:00
================================================================================
CHASSIS LEVEL POWER INFO: 0
================================================================================
   Total output power capacity (N + 1)             :    2000W +       0W
   Total output power required                     :     910W
   Total power input                               :     456W
   Total power output                              :     407W

Power Group 0:
================================================================================
   Power       Supply     ------Input----   ------Output---      Status
   Module      Type        Volts     Amps    Volts     Amps
================================================================================
   0/PM0       2kW-AC        0.0      0.0      0.0      0.0    FAILED or NO PWR

Total of Power Group 0:          0W/  0.0A         0W/  0.0A

Power Group 1:
================================================================================
   Power       Supply     ------Input----   ------Output---      Status
   Module      Type        Volts     Amps    Volts     Amps
================================================================================
   0/PM1       2kW-AC      227.8      2.0     12.0     33.9    OK

Total of Power Group 1:        456W/  2.0A       407W/ 33.9A

================================================================================
   Location     Card Type            Power       Power       Status
                                     Allocated   Used
                                     Watts       Watts
================================================================================
   0/0          NCS1K4-LC-FILLER         0           -       RESERVED
   0/1          NCS1K4-1.2T-K9         260         101       ON
   0/2          NCS1K4-1.2TL-K9        260         168       ON
   0/3          NCS1K4-LC-FILLER         0           -       RESERVED
   0/RP0        NCS1K4-CNTLR-K9         55           -       ON
   0/FT0        NCS1K4-FAN             100           -       ON
   0/FT1        NCS1K4-FAN             100           -       ON
   0/FT2        NCS1K4-FAN             100           -       ON
   0/SC0        NCS1004                 35           -       ON

The following example shows sample output from the show environment command with the voltages keyword. 

sysadmin-vm:0_RP0# show environment voltages location 0/RP0
Wed Mar  4  05:47:24.668 UTC+00:00
================================================================================
Location  VOLTAGE                     Value   Crit Minor Minor  Crit
          Sensor                      (mV)    (Lo) (Lo)  (Hi)   (Hi)
--------------------------------------------------------------------------------
0/RP0
          ADM1266_VH1_12V             12028  10800 11040 12960 13200
          ADM1266_VH3_3V3              3306   3036  3135  3465  3564
          ADM1266_VH4_2V5              2492   2300  2375  2625  2700
          ADM1266_VP1_1V8              1801   1656  1710  1890  1944
          ADM1266_VP2_1V2              1201   1104  1140  1260  1296
          ADM1266_3V3_STAND_BY         3293   3036  3135  3465  3564
          ADM1266_VP4_3V3_CPU          3301   3036  3135  3465  3564
          ADM1266_VP5_2V5_CPU          2494   2300  2375  2625  2700
          ADM1266_VP6_1V8_CPU          1797   1656  1710  1890  1944
          ADM1266_VP7_1V24_VCCREF      1236   1140  1178  1302  1339
          ADM1266_VP8_1V05_CPU         1045    966   997  1102  1134
          ADM1266_VP9_1V2_DDR_VDDQ     1196   1104  1140  1260  1296
          ADM1266_VP10_1V0_VCCRAM      1074    500   650  1300  1400
          ADM1266_VP11_VNN              882    400   550  1300  1400
          ADM1266_VP12_VCCP            1068    300   450  1300  1400
          ADM1266_VP13_0V6_VTT          599    552   570   630   648
          ADM1293_DB_5V0               5007   4600  4750  5250  5400
          ADM1293_DB_3V3               3305   3036  3135  3465  3564
          ADM1293_DB_5V0_USB_0         5007   4000  4500  5500  6000
          ADM1293_DB_5V0_USB_1         5017   4000  4500  5500  6000
          ADM1293_MB_5V0_PMOD0         5062   4600  4750  5250  5400
          ADM1293_MB_5V0_PMOD1         5032   4600  4750  5250  5400
          ADM1293_MB_2V5_PLL           2483   2300  2375  2625  2700

Step 3

show inventory

Displays inventory information for all the physical entities of NCS 1004. 

RP/0/RP0/CPU0:ios# show inventory
Wed Mar  4 05:10:17.107 UTC
NAME: "0/0", DESCR: "Network Convergence System 1004 Filler"
PID: NCS1K4-LC-FILLER, VID: V01, SN: N/A

NAME: "0/1", DESCR: "NCS1K4 12x QSFP28 2 Trunk C-Band DWDM card"
PID: NCS1K4-1.2T-K9, VID: V00, SN: CAT2250B0AE

NAME: "0/1-Optics0/1/0/2", DESCR: "Cisco 100G QSFP28 AOC Pluggable Optics Module"
PID: QSFP-100G-AOC3M   , VID: V03, SN: INL22262339-A

NAME: "0/1-Optics0/1/0/4", DESCR: "Cisco 100GE QSFP28 SR4 Pluggable Optics Module"
PID: QSFP-100G-SR4-S, VID: V03, SN: AVF2219S16U

NAME: "0/1-Optics0/1/0/5", DESCR: "Cisco 100G QSFP28 LR4-S Pluggable Optics Module"
PID: QSFP-100G-LR4-S, VID: V02, SN: JFQ2145701U

NAME: "0/1-Optics0/1/0/6", DESCR: "Cisco 100GE QSFP28 SR4 Pluggable Optics Module"
PID: QSFP-100G-SR4-S, VID: ES1, SN: AVF1925G012

NAME: "0/1-Optics0/1/0/7", DESCR: "Cisco 100G QSFP28 LR4-S Pluggable Optics Module"
PID: QSFP-100G-LR4-S, VID: V02, SN: JFQ2145706N

NAME: "0/1-Optics0/1/0/8", DESCR: "Cisco QSFP-100G-LR4 Pluggable Optics Module"
PID: ONS-QSFP28-LR4, VID: V01, SN: JFQ19026014

NAME: "0/1-Optics0/1/0/9", DESCR: "Cisco 100G QSFP28 LR4-S Pluggable Optics Module"
PID: QSFP-100G-LR4-S, VID: V02, SN: OPM220518HS

NAME: "0/1-Optics0/1/0/10", DESCR: "Cisco 100G QSFP28 SM-SR Pluggable Optics Module"
PID: QSFP-100G-SM-SR, VID: V02, SN: INL21490043

NAME: "0/1-Optics0/1/0/11", DESCR: "Cisco 100G QSFP28 CWDM4 Pluggable Optics Module"
PID: QSFP-100G-CWDM4-S , VID: V01, SN: JFQ211930JL

NAME: "0/1-Optics0/1/0/12", DESCR: "Cisco 100G QSFP28 CWDM4 Pluggable Optics Module"
PID: QSFP-100G-CWDM4-S, VID: V02, SN: JFQ2210801H

NAME: "0/2", DESCR: "NCS1K4 12x QSFP28 2 Trunk L-Band DWDM card"
PID: NCS1K4-1.2TL-K9   , VID: V00, SN: CAT2337B0S4

NAME: "0/2-Optics0/2/0/2", DESCR: "Cisco 100G QSFP28 AOC Pluggable Optics Module"
PID: QSFP-100G-AOC3M, VID: V03, SN: INL22262332-A

NAME: "0/2-Optics0/2/0/4", DESCR: "Cisco 100G QSFP28 SM-SR Pluggable Optics Module"
PID: QSFP-100G-SM-SR, VID: V02, SN: FNS22070HWF

NAME: "0/2-Optics0/2/0/5", DESCR: "Cisco 100G QSFP28 SM-SR Pluggable Optics Module"
PID: QSFP-100G-SM-SR, VID: V02, SN: SPT2225302D

NAME: "0/2-Optics0/2/0/6", DESCR: "Cisco 100G QSFP28 LR4-S Pluggable Optics Module"
PID: QSFP-100G-LR4-S, VID: V02, SN: FNS22310Z1X

NAME: "0/2-Optics0/2/0/8", DESCR: "Cisco QSFP-100G-LR4 Pluggable Optics Module"
PID: ONS-QSFP28-LR4, VID: V01, SN: FNS20520R8Z

NAME: "0/2-Optics0/2/0/9", DESCR: "Cisco 100G QSFP28 AOC Pluggable Optics Module"
PID: QSFP-100G-AOC3M, VID: V03, SN: INL23312282-A

NAME: "0/2-Optics0/2/0/10", DESCR: "Cisco 100G QSFP28 AOC Pluggable Optics Module"
PID: QSFP-100G-AOC3M, VID: V03, SN: INL23312282-B

NAME: "0/2-Optics0/2/0/11", DESCR: "Cisco 100G QSFP28 LR4-S Pluggable Optics Module"
PID: QSFP-100G-LR4-S, VID: V02, SN: FNS23080LKF

NAME: "0/3", DESCR: "Network Convergence System 1004 Filler"
PID: NCS1K4-LC-FILLER, VID: V01, SN: N/A

NAME: "0/RP0", DESCR: "Network Convergence System 1004 Controller"
PID: NCS1K4-CNTLR-K9, VID: V00, SN: CAT2231B069

NAME: "0/SC0", DESCR: "Network Convergence System 1004 Chassis"
PID: NCS1004, VID: V00, SN: CAT2231B192

NAME: "Rack 0", DESCR: "Network Convergence System 1004 Chassis"
PID: NCS1004, VID: V00, SN: CAT2231B192

NAME: "0/FT0", DESCR: "Network Convergence System 1004 Fan"
PID: NCS1K4-FAN, VID: V00, SN: CAT2231B2GL

NAME: "0/FT1", DESCR: "Network Convergence System 1004 Fan"
PID: NCS1K4-FAN, VID: V00, SN: CAT2231B2H4

NAME: "0/FT2", DESCR: "Network Convergence System 1004 Fan"
PID: NCS1K4-FAN, VID: V00, SN: CAT2231B2GW

NAME: "0/PM0", DESCR: "Network Convergence System 1004 AC Power Supply Unit"
PID: NCS1K4-AC-PSU, VID: V00, SN: POG2221CL1V

NAME: "0/PM1", DESCR: "Network Convergence System 1004 AC Power Supply Unit"
PID: NCS1K4-AC-PSU, VID: V00, SN: POG2221CL04

What to do next

Environment parameter anomalies are logged in the syslog. Hence, if an environment parameter displayed in the show environment command output is not as expected, check the syslog using the show logging command. The syslog provides details on any logged problems.

Verify Firmware Version

The firmware on various hardware components of NCS 1004 must be compatible with the installed Cisco IOS XR image. Incompatibility may cause the NCS 1004 to malfunction.

To verify the firmware version, perform the following procedure.

Procedure

Step 1

show hw-module fpd 


RP/0/RP0/CPU0:ios# show hw-module fpd 
Fri Nov 26 14:53:27.188 UTC

Auto-upgrade:Disabled
                                                                         FPD Versions
                                                                        ==============
Location   Card type             HWver FPD device       ATR Status    Running   Programd  
-----------------------------------------------------------------------------------------
0/0        NCS1K4-OTN-XPL        3.0   LC_CPU_MOD_FW        CURRENT    75.10     75.10    
0/0        NCS1K4-OTN-XPL        7.0   LC_DP_MOD_FW         CURRENT     3.10      3.10    
0/0        NCS1K4-OTN-XPL        2.0   LC_QSFPDD_PORT_11    CURRENT    61.2013   61.2013  
0/0        NCS1K4-OTN-XPL        2.0   LC_QSFPDD_PORT_9     CURRENT    61.2013   61.2013  
0/1        NCS1K4-OTN-XP         2.0   LC_CPU_MOD_FW        CURRENT    75.10     75.10    
0/1        NCS1K4-OTN-XP         7.0   LC_DP_MOD_FW         CURRENT     3.10      3.10    
0/1        NCS1K4-OTN-XP         2.0   LC_QSFPDD_PORT_11    CURRENT    61.2013   61.2013  
0/1        NCS1K4-OTN-XP         2.0   LC_QSFPDD_PORT_9     CURRENT    61.2013   61.2013  
0/RP0      NCS1K4-CNTLR-K9       5.0   CSB_IMG           S  CURRENT     0.200     0.200   
0/RP0      NCS1K4-CNTLR-K9       5.0   TAM_FW               CURRENT    36.08     36.08    
0/RP0      NCS1K4-CNTLR-K9       1.14  BIOS              S  CURRENT     5.30      5.30    
0/RP0      NCS1K4-CNTLR-K9       5.0   CPU_FPGA             CURRENT     1.14      1.14    
0/PM1      NCS1K4-AC-PSU         0.1   PO-PriMCU            CURRENT     2.70      2.70    
0/SC0      NCS1004               2.0   BP_FPGA              CURRENT     1.25      1.25    
0/SC0      NCS1004               2.0   XGE_FLASH            CURRENT    18.04     18.04

Displays firmware information of various hardware components of NCS 1004 in the Cisco IOS XR EXEC mode.

In the previous output, some of the significant fields are:

  • FPD Device—Name of the hardware component such as FPD, CFP, and so on.

  • ATR—Attribute of the hardware component. Some of the attributes are:

    • B—Backup Image

    • S—Secure Image

    • P—Protected Image

  • Status—Upgrade status of the firmware. The different states are:

    • CURRENT—The firmware version is the latest version.

    • NOT READY—The firmware of the FPD is not ready for upgrade.

    • NEED UPGD—A newer firmware version is available in the installed image. We recommended that upgrade be performed.

    • UPGD PREP—The firmware of the FPD is preparing for upgrade.

    • RLOAD REQ—The upgrade is completed, and the card requires a reload.

    • UPGD DONE—The firmware upgrade is successful.

    • UPGD FAIL—The firmware upgrade has failed.

    • UPGD SKIP—The upgrade is skipped because the installed firmware version is higher than the version available in the image.

    • Running—Current version of the firmware running on the FPD.

Step 2

show fpd package

Use the show fpd package command to display the FPD image version available with this software release for each hardware component. 

RP/0/RP0/CPU0:ios# show fpd package 
Fri May  8 05:11:47.819 UTC

=============================== ================================================
                                        Field Programmable Device Package
                                ================================================
                                               Req     SW      Min Req   Min Req
Card Type            FPD Description           Reload  Ver     SW Ver  Board Ver
=================== ========================== ====== ======= ======== =========
--------------------------------------------------------------------------------
NCS1004-K9           BP_FPGA(A)                 NO      1.25    1.25    0.0
                     XGE_FLASH(A)               YES    18.04   18.04    0.0
--------------------------------------------------------------------------------
NCS1K4-1.2T-K9       LC_CPU_MOD_FW(A)           YES    75.10   75.10    0.0
                     LC_OPT_MOD_FW(A)           YES     1.25    1.25    0.0
--------------------------------------------------------------------------------
NCS1K4-1.2T-L-K9     LC_CPU_MOD_FW(A)           YES    75.10   75.10    0.0
                     LC_OPT_MOD_FW(A)           YES     1.25    1.25    0.0
--------------------------------------------------------------------------------
NCS1K4-1.2TL-K9      LC_CPU_MOD_FW(A)           YES    75.10   75.10    0.0
                     LC_OPT_MOD_FW(A)           YES     1.25    1.25    0.0
--------------------------------------------------------------------------------
NCS1K4-2-QDD-C-K9    LC_CPU_MOD_FW(A)           YES    75.10   75.10    0.0
                     LC_OPT_MOD_FW(A)           YES     1.26    1.26    0.0
--------------------------------------------------------------------------------
NCS1K4-2KW-AC        PO-PriMCU(A)               NO      2.70    2.70    0.0
                     PO-PriMCU(A)               NO      2.70    2.70    0.1
--------------------------------------------------------------------------------
NCS1K4-AC-PSU        PO-PriMCU(A)               NO      2.70    2.70    0.0
                     PO-PriMCU(A)               NO      2.70    2.70    0.1
--------------------------------------------------------------------------------
NCS1K4-CNTLR         BIOS(A)                    YES     5.30    5.30    1.5
                     CSB_IMG                    YES     0.200   0.200   0.0
--------------------------------------------------------------------------------
NCS1K4-CNTLR-B-K9    BIOS(A)                    YES     5.30    5.30    1.0
                     CSB_IMG                    YES     0.200   0.200   0.0
--------------------------------------------------------------------------------
NCS1K4-DC-PSU        PO-PriMCU(A)               NO      1.12    1.12    0.0
                     PO-PriMCU(A)               NO      1.12    1.12    0.1
--------------------------------------------------------------------------------
NCS1K4-OTN-XP       LC_CFP2_PORT_0(A)           NO      0.00    0.00    0.0
			LC_CFP2_PORT_0(A)           NO      1.00    1.00    1.0
                    LC_CFP2_PORT_0(A)           NO      1.52    1.52    2.0
                    LC_CFP2_PORT_1(A)           NO      0.00    0.00    0.0
                    LC_CFP2_PORT_1(A)           NO      1.00    1.00    1.0
                    LC_CFP2_PORT_1(A)           NO      1.52    1.52    2.0
                    LC_CPU_MOD_FW(A)            YES     75.10   75.10   0.0
                    LC_DP_MOD_FW(A)             YES     3.10    3.10    1.0
                    LC_DP_MOD_FW(A)             YES     11.10   11.10   2.0
                    LC_DP_MOD_FW(A)             YES     11.10   11.10   3.0
                    LC_DP_MOD_FW(A)             YES     1.10    1.10    4.0
                    LC_DP_MOD_FW(A)             YES     3.10    3.10    7.0
                    LC_DP_MOD_FW(A)             YES     1.10    1.10    8.0
                    LC_QSFPDD_PORT_11(A)         NO     0.00    0.00    0.0
                    LC_QSFPDD_PORT_11(A)         NO    61.2013 61.2013  1.0
                    LC_QSFPDD_PORT_11(A)         NO    61.2013 61.2013  2.0
                    LC_QSFPDD_PORT_9(A)          NO    0.00     0.00    0.0
                    LC_QSFPDD_PORT_9(A)          NO    61.2013 61.2013  1.0
                    LC_QSFPDD_PORT_9(A)          NO    61.2013 61.2013  2.0
--------------------------------------------------------------------------------
NCS1K4-OTN-XPL      LC_CFP2_PORT_0(A)           NO      0.00    0.00    0.0
			LC_CFP2_PORT_0(A)           NO      1.00    1.00    1.0
                    LC_CFP2_PORT_0(A)           NO      1.52    1.52    2.0
                    LC_CFP2_PORT_1(A)           NO      0.00    0.00    0.0
                    LC_CFP2_PORT_1(A)           NO      1.00    1.00    1.0
                    LC_CFP2_PORT_1(A)           NO      1.52    1.52    2.0
                    LC_CPU_MOD_FW(A)            YES     75.10   75.10   0.0
                    LC_DP_MOD_FW(A)             YES     3.10    3.10    1.0
                    LC_DP_MOD_FW(A)             YES     11.10   11.10   2.0
                    LC_DP_MOD_FW(A)             YES     11.10   11.10   3.0
                    LC_DP_MOD_FW(A)             YES     1.10    1.10    4.0
                    LC_DP_MOD_FW(A)             YES     3.10    3.10    7.0
                    LC_DP_MOD_FW(A)             YES     1.10    1.10    8.0
                    LC_QSFPDD_PORT_11(A)         NO     0.00    0.00    0.0
                    LC_QSFPDD_PORT_11(A)         NO    61.2013 61.2013  1.0
                    LC_QSFPDD_PORT_11(A)         NO    61.2013 61.2013  2.0
                    LC_QSFPDD_PORT_9(A)          NO    0.00     0.00    0.0
                    LC_QSFPDD_PORT_9(A)          NO    61.2013 61.2013  1.0
                    LC_QSFPDD_PORT_9(A)          NO    61.2013 61.2013  2.0
--------------------------------------------------------------------------------
NCS1K4-TESTUNIT      LC_CPU_MOD_FW(A)           YES     0.01    0.01    0.0

What to do next

Upgrade all the FPDs using the upgrade hw-module location all fpd all command in the Cisco IOS XR EXEC mode. After upgrade is completed, the Status column shows RLOAD REQ if the software requires reload.

If Reload is required

If the FPGA location is 0/RP0, use the admin hw-module location 0/RP0 reload command. This command reboots only the CPU. As a result, traffic is not impacted. If the FPGA location is 0/0, use the admin hw-module location all reload command. This command reboots the chassis. As a result, traffic is impacted. After the reload is completed, the new FPGA runs the current version.


Caution
The upgrade of OTNXP LC_DP_MOD_FW and LC_OPT_MOD_FW FPDs affect traffic. Hence, you must perform this upgrade during a maintenance window.

If Firmware Upgrade Fails

If firmware upgrade fails, use the show logging command to view the details and upgrade the firmware again using the above commands.


Note
You can upgrade the firmware version of power modules, only when both the power modules are present and powered on.

Loopback

You can configure the loopback on the CoherentDSP, FC, OTU, and Ethernet controllers to identify connection problems. The loopback can be configured only in the maintenance mode. Use the controller controller-type and the secondary-admin-state maintenance commands to place the controllers in the maintenance mode.


Note
Internal and line loopbacks are supported on the FC, OTU, and Ethernet controllers whereas only internal loopbacks are supported on the CoherentDSP controllers.

Configuring Loopback on the 1.2T Card

To configure the loopback, use the following commands:

configure

controller controllertype Rack/Slot/Instance/Port

sec-admin-state maintenance

loopback [ line | internal ]

commit

Example 1

The following example shows how a line loopback is configured on the Ethernet controller. 


RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller HundredGigECtrlr 1/0/1/10 secondary-admin-state maintenance
RP/0/RP0/CPU0:ios(config)#commit
Fri Feb 22 19:49:46.504 UTC
RP/0/RP0/CPU0:ios(config)#exit

The following example shows how to verify a line loopback configured on the Ethernet controller. 


RP/0/RP0/CPU0:ios#show controller HundredGigECtrlr 0/1/0/10
Fri Feb 22 19:50:08.328 UTC
Operational data for interface HundredGigECtrlr0/1/0/10:

State:
    Administrative state: enabled
    Operational state: Up
    LED state: Green On
    Maintenance: Enabled
    AINS Soak: Pending
      Total Duration: 0 hour(s) 30 minute(s)
      Remaining Duration: 0 hour(s) 30 minute(s) 0 second(s)
    Laser Squelch: Enabled

Phy:
    Media type: Not known
    Statistics:
        FEC:
            Corrected Codeword Count: 0
            Uncorrected Codeword Count: 0

Autonegotiation disabled.

Operational values:
    Speed: 100Gbps
    Duplex: Full Duplex
    Flowcontrol: None
    Loopback: None (or external)
    BER monitoring:
        Not supported
    Forward error correction: Standard (Reed-Solomon)
    Holdoff Time: 0ms
RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller HundredGigECtrlr 0/1/0/10 loopback line
RP/0/RP0/CPU0:ios(config)#commit
RP/0/RP0/CPU0:ios(config)#exit
RP/0/RP0/CPU0:ios#show controller HundredGigECtrlr 0/1/0/10
Fri Feb 22 20:01:00.521 UTC
Operational data for interface HundredGigECtrlr0/1/0/10:

State:
    Administrative state: enabled
    Operational state: Up
    LED state: Green On
    Maintenance: Enabled
    AINS Soak: Pending
      Total Duration: 0 hour(s) 30 minute(s)
      Remaining Duration: 0 hour(s) 30 minute(s) 0 second(s)
    Laser Squelch: Enabled

Phy:
    Media type: Not known
    Statistics:
        FEC:
            Corrected Codeword Count: 0
            Uncorrected Codeword Count: 6

Autonegotiation disabled.

Operational values:
    Speed: 100Gbps
    Duplex: Full Duplex
    Flowcontrol: None
    Loopback: Line
    BER monitoring:
        Not supported
    Forward error correction: Standard (Reed-Solomon)
    Holdoff Time: 0ms

Example 2

The following example shows how to verify an internal loopback configured on the coherent DSP controller.

RP/0/RP0/CPU0:ios#show controllers coherentDSP  0/0/0/0
Fri Mar 13 22:00:20.951 UTC

Port                                            : CoherentDSP 0/0/0/0
Controller State                                : Up
Inherited Secondary State                       : Normal
Configured Secondary State                      : Maintenance
Derived State                                   : Maintenance
Loopback mode                                   : Internal
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable
Bandwidth                                       : 200.0Gb/s

Alarm Information:
LOS = 0 LOF = 1 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 3 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0
Detected Alarms                                 : None

Bit Error Rate Information
PREFEC  BER                                     : 0.00E+00
POSTFEC BER                                     : 0.00E+00
Q-Factor                                        : 16.70 dB

Q-Margin                                        : 0.99dB

TTI :
        Remote hostname                         : ios
        Remote interface                        : CoherentDSP 0/0/0/0
        Remote IP addr                          : 0.0.0.0

FEC mode                                        : Soft-Decision 27

AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

Configuring Loopback on OTN-XP Card

From R7.2.1 onwards, OTN-XP card supports loopback on the OTU2, OTU2e, OTU4, 10GE, and CoherentDSP controllers.

From R7.3.2 onwards, OTN-XP card supports loopback on the 100GE and 400GE controllers.

From R7.5.2 onwards, OTN-XP card supports loopback on the 16G FC and 32G FC controllers.

The CoherentDSP controller supports both line and internal.

To configure the loopback on the controllers, use the following commands:

configure

controller controller type Rack/Slot/Instance/Port/Lane number

sec-admin-state maintenance

loopback [ line | internal ]

commit

The range of Lane number is 1–4.

Example 1

The following example shows how an internal loopback is configured on the 10GE controller. 


RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller tenGigECtrlr 0/0/0/5/2
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#sec-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#loopback internal 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#commit

The following example shows how to verify an internal loopback configured on the 10GE controller. 


RP/0/RP0/CPU0:ios#show controllers tenGigECtrlr 0/0/0/5/2
Thu Apr 23 10:47:48.020 UTC
Operational data for interface TenGigECtrlr0/0/0/5/2:

State:
    Administrative state: enabled
    Operational state: Up
    LED state: Green On
    Maintenance: Enabled
    AINS Soak: None
      Total Duration: 0 hour(s) 0 minute(s)
      Remaining Duration: 0 hour(s) 0 minute(s) 0 second(s)
    Laser Squelch: Disabled

Phy:
    Media type: Not known

Autonegotiation disabled.

Operational values:
    Speed: 10Gbps
    Duplex: Full Duplex
    Flowcontrol: None
    Loopback: Internal
    Inter-packet gap: standard (12)
    BER monitoring:
        Not supported
    Holdoff Time: 0ms

Example 2

The following example shows how a line loopback is configured on the OTU2e controller.


RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller otu2e 0/0/0/11/3 
RP/0/RP0/CPU0:ios(config-otu2e)#secondary-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-otu2e)#loopback line
RP/0/RP0/CPU0:ios(config-otu2e)#commit
Thu Apr 23 10:55:19.319 UTC
RP/0/RP0/CPU0:ios(config-otu2e)#end

The following example shows how to verify a line loopback configured on the OTU2e controller.


RP/0/RP0/CPU0:ios#show controllers otu2e 0/0/0/11/3 
Thu Apr 23 10:55:28.014 UTC

Port                                            : OTU2E 0/0/0/11/3
Controller State                                : Up
Inherited Secondary State                       : Normal
Configured Secondary State                      : Maintenance
Derived State                                   : Maintenance
Loopback mode                                   : Line
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable
Bandwidth                                       : 10.0Gb/s

Alarm Information:
LOS = 0 LOF = 1 LOM = 0
OOF = 1 OOM = 1 AIS = 0
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 0 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0     
Detected Alarms                                 : None

Bit Error Rate Information
PREFEC  BER                                     : 0.00E+00 
POSTFEC BER                                     : 0.00E+00 

TTI :
        Remote hostname                         : ios
        Remote interface                        : OTU2E 0/0/0/11/3
        Remote IP addr                          : 0.0.0.0

FEC mode                                        : STANDARD

AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

Example 3

The following example shows how an internal loopback is configured on the OTU2 controller.

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller otu2 0/0/0/5/1
RP/0/RP0/CPU0:ios(config-otu2)#secondary-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-otu2)#loopback internal
RP/0/RP0/CPU0:ios(config-otu2)#commit
Thu Apr 23 11:01:00.562 UTC
RP/0/RP0/CPU0:ios(config-otu2)#end
The following example shows how to verify an internal loopback configured on the OTU2 controller.

RP/0/RP0/CPU0:ios#show controllers otU2 0/0/0/5/1
Thu Apr 23 11:01:04.126 UTC

Port                                            : OTU2 0/0/0/5/1
Controller State                                : Up
Inherited Secondary State                       : Normal
Configured Secondary State                      : Maintenance
Derived State                                   : Maintenance
Loopback mode                                   : Internal
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable
Bandwidth                                       : 10.0Gb/s

Alarm Information:
LOS = 0 LOF = 0 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 0 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0     
Detected Alarms                                 : None

Bit Error Rate Information
PREFEC  BER                                     : 0.00E+00 
POSTFEC BER                                     : 0.00E+00 

TTI :
        Remote hostname                         : SM-TRC SAPI-SECSM-TRC DA
        Remote IP addr                          : 209.165.200.229

FEC mode                                        : STANDARD

AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

Example 4

The following example shows how an internal loopback is configured on the OTU4 controller.

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller otu4 0/0/0/0
RP/0/RP0/CPU0:ios(config-otu4)#secondary-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-otu4)#loopback internal 
RP/0/RP0/CPU0:ios(config-otu4)#commit
Thu Apr 23 11:05:22.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end
The following example shows how to verify an internal loopback configured on the OTU4 controller.

RP/0/RP0/CPU0:ios#show controllers otu4 0/0/0/0
Thu Apr 23 11:05:30.281 UTC

Port                                            : OTU4 0/0/0/0
Controller State                                : Up
Inherited Secondary State                       : Normal
Configured Secondary State                      : Maintenance
Derived State                                   : Maintenance
Loopback mode                                   : Internal
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable
Bandwidth                                       : 100.0Gb/s

Alarm Information:
LOS = 1 LOF = 0 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 0 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0     
Detected Alarms                                 : None 

Bit Error Rate Information
PREFEC  BER                                     : 0.00E+00 
POSTFEC BER                                     : 0.00E+00 

TTI :
        Remote hostname                         : ios
        Remote interface                        : OTU4 0/0/0/0
        Remote IP addr                          : 0.0.0.0

FEC mode                                        : STANDARD

AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

Example 5

The following example shows how an internal loopback is configured on the 16G FC controller:


RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller SixteenGigFibreChanCtrlr 0/2/0/1/1
RP/0/RP0/CPU0:ios(config-SixteenGigFibreChanCtrlr)#sec-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-SixteenGigFibreChanCtrlr)#loopback internal 
RP/0/RP0/CPU0:ios(config-SixteenGigFibreChanCtrlr)#commit
Thu Apr 11 10:05:21.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end

The following example shows how to verify the internal loopback configured on the 16G FC controller:


RP/0/RP0/CPU0:ios#show controller SixteenGigFibreChanCtrlr 0/1/0/0/2

Sat Apr  9 22:50:38.930 UTC

Operational data for Fibre Channel controller SixteenGigFibreChanCtrlr0/1/0/0/2

State:
	Admin State           : Up
	Operational state     : Up
	LED state             : Green On
	Secondary admin state : Maintenance
	AINS Soak             : None
	    Total Duration    : 0 hour(s) 0 minute(s)
	    Remaining Duration: 0 hour(s) 0 minute(s) 0 second(s)
	Laser Squelch         : Disabled

Performance Monitoring is enabled

Operational values:
	Speed                    : 16 Gbps
	Loopback                 : Internal
	BER monitoring:
	    Not supported
	Hold-off Time            : 0 ms
	Forward Error Correction : Not Configured
RP/0/RP0/CPU0:ios#

Example 6

The following example shows how an internal loopback is configured on the 32G FC controller:


RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller ThirtyTwoGigFibreChanCtrlr 0/1/0/6/4
RP/0/RP0/CPU0:ios(config-ThirtyTwoGigFibreChanCtrlr)#loopback internal

RP/0/RP0/CPU0:ios(config-ThirtyTwoGigFibreChanCtrlr)#commit

Sat Apr  9 22:50:11.666 UTC
RP/0/RP0/CPU0:ios(config-ThirtyTwoGigFibreChanCtrlr)#end

The following example shows how to verify the internal loopback configured on the 32G FC controller:

RP/0/RP0/CPU0:ios#show controller ThirtyTwoGigFibreChanCtrlr 0/1/0/6/4

Sat Apr  9 22:50:39.082 UTC

Operational data for Fibre Channel controller ThirtyTwoGigFibreChanCtrlr0/1/0/6/4

State:
	Admin State           : Up
	Operational state     : Up
	LED state             : Green On
	Secondary admin state : Maintenance
	AINS Soak             : None
	    Total Duration    : 0 hour(s) 0 minute(s)
	    Remaining Duration: 0 hour(s) 0 minute(s) 0 second(s)
	Laser Squelch         : Disabled

Performance Monitoring is enabled

Operational values:
	Speed                    : 32 Gbps
	Loopback                 : Internal
	BER monitoring:
	    Not supported
	Hold-off Time            : 0 ms
	Forward Error Correction : Standard(Reed Solomon)
RP/0/RP0/CPU0:ios#

Example: Loopback Configuration in 4X100G MXP on 100GE Controller

The following example shows how the client internal loopback is configured on the 100GE controller:

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller HundredGigECtrlr 0/2/0/1 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#sec-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#loopback internal 
RP/0/RP0/CPU0:ios(config-otu4)#commit
Thu Sep 23 11:05:22.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end
The following example shows how the client line loopback is configured on the 100GE controller:

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller HundredGigECtrlr 0/2/0/1 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#sec-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#loopback line    
RP/0/RP0/CPU0:ios(config-otu4)#commit
Thu Sep 23 11:05:22.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end
The following example shows how the trunk internal is configured on the coherentDSP controller:

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/2/0/11
RP/0/RP0/CPU0:ios(config-CoDSP)#secondary-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-CoDSP)#loopback internal 
RP/0/RP0/CPU0:ios(config-otu4)#commit
Thu Sep 23 11:05:22.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end
The following example shows how the trunk line is configured on the coherentDSP controller:

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/2/0/11 
RP/0/RP0/CPU0:ios(config-CoDSP)#secondary-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-CoDSP)#loopback line    
RP/0/RP0/CPU0:ios(config-otu4)#commit
Thu Sep 23 11:05:22.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end

Example: Loopback Configuration in 400G-TXP on 400GE Controller

The following example shows how the client internal loopback is configured on the 400GE controller:

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller FourHundredGigECtrlr 0/2/0/10 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#sec-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#loopback internal 
RP/0/RP0/CPU0:ios(config-otu4)#commit
Thu Sep 23 11:05:22.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end
The following example shows how the client line loopback is configured on the 100GE controller:

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller FourHundredGigECtrlr 0/2/0/10 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#sec-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#loopback line    
RP/0/RP0/CPU0:ios(config-otu4)#commit
Thu Sep 23 11:05:22.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end
The following example shows how the trunk internal is configured on the coherentDSP controller:

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/2/0/10
RP/0/RP0/CPU0:ios(config-CoDSP)#secondary-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-CoDSP)#loopback internal 
RP/0/RP0/CPU0:ios(config-otu4)#commit
Thu Sep 23 11:05:22.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end
The following example shows how the trunk line is configured on the coherentDSP controller:

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/2/0/10 
RP/0/RP0/CPU0:ios(config-CoDSP)#secondary-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-CoDSP)#loopback line    
RP/0/RP0/CPU0:ios(config-otu4)#commit
Thu Sep 23 11:05:22.429 UTC
RP/0/RP0/CPU0:ios(config-otu4)#end

Configure Loopback in Inverse Muxponder Configured on the OTN-XP Card

You can configure loopback on the coherentDSP controllers in the inverse muxponder configuration.


Note
You must configure loopback on both trunk ports 12 and 13, otherwise traffic goes down.

The following example shows how loopback is configured on both the trunk ports:

RP/0/RP0/CPU0:ios#configure
Thu Sep 30 14:16:04.678 UTC
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/2/0/12 
RP/0/RP0/CPU0:ios(config-CoDSP)#secondary-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-CoDSP)#loopback internal 
RP/0/RP0/CPU0:ios(config-CoDSP)#commit
Thu Sep 30 14:16:19.594 UTC
RP/0/RP0/CPU0:ios(config-CoDSP)#controller coherentDSP 0/2/0/13   
RP/0/RP0/CPU0:ios(config-CoDSP)#secondary-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-CoDSP)#loopback internal 
RP/0/RP0/CPU0:ios(config-CoDSP)#commit
Thu Sep 30 14:16:32.390 UTC
RP/0/RP0/CPU0:ios(config-CoDSP)#

The following examples shows how to verify loopback configured on the OTN-XP card in the inverse muxponder configuration:

RP/0/RP0/CPU0:ios#show controllers coherentDSP 0/2/0/12 
Thu Sep 30 14:17:04.411 UTC

Port                                            : CoherentDSP 0/2/0/12
Controller State                                : Up
Inherited Secondary State                       : Normal
Configured Secondary State                      : Maintenance
Derived State                                   : Maintenance
Loopback mode                                   : Internal
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable
Bandwidth                                       : 200.0Gb/s

Alarm Information:
LOS = 2 LOF = 0 LOM = 0
OOF = 1 OOM = 0 AIS = 1
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 0 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0     FLEXO_GIDM = 0
FLEXO-MM = 0    FLEXO-LOM = 0   FLEXO-RDI = 1
FLEXO-LOF = 0   
Detected Alarms                                 : None

Bit Error Rate Information
PREFEC  BER                                     : 2.46E-08 
POSTFEC BER                                     : 0.00E+00 
Q-Factor                                        : 14.60 dB 

Q-Margin                                        : 8.30dB 

TTI :
        Remote hostname                         : ios
        Remote interface                        : CoherentDSP 0/2/0/12
        Remote IP addr                          : 0.0.0.0

FEC mode                                        : O_FEC

Flexo-Mode                                      : Enable
Flexo Details:
        Tx GID                                  : 1
        TX IID                                  : 1, 2, 
        Rx GID                                  : 1
        RX IID                                  : 1, 2, 

Flexo Peers Information:
        Controller                              : CoherentDSP0_2_0_13 
        OTUCn rate                              : OTUC2


AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds


RP/0/RP0/CPU0:ios#sh controllers coherentDSP 0/2/0/13 
Thu Sep 30 14:17:08.140 UTC

Port                                            : CoherentDSP 0/2/0/13
Controller State                                : Up
Inherited Secondary State                       : Normal
Configured Secondary State                      : Maintenance
Derived State                                   : Maintenance
Loopback mode                                   : Internal
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable
Bandwidth                                       : 200.0Gb/s

Alarm Information:
LOS = 1 LOF = 0 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 0 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0     FLEXO_GIDM = 0
FLEXO-MM = 0    FLEXO-LOM = 0   FLEXO-RDI = 1
FLEXO-LOF = 0   
Detected Alarms                                 : None

Bit Error Rate Information
PREFEC  BER                                     : 0.00E+00 
POSTFEC BER                                     : 0.00E+00 
Q-Factor                                        : 15.70 dB 

Q-Margin                                        : 9.50dB 

TTI :
        Remote IP addr                          : 0.0.0.0

FEC mode                                        : O_FEC

Flexo-Mode                                      : Enable
Flexo Details:
        Tx GID                                  : 1
        TX IID                                  : 3, 4, 
        Rx GID                                  : 1
        RX IID                                  : 3, 4, 

Flexo Peers Information:
        Controller                              : CoherentDSP0_2_0_12 
        OTUCn rate                              : OTUC2


AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

Configuring Loopback on 2-QDD-C Card

From R7.3.1 onwards, 2-QDD-C card supports loopback on the 100 and 400GE controllers.


Note
On applying client-side loopbacks, traffic is looped and does not continue in the 2-QDD-C card. QSFP squelching happens on applying internal loopback.

To configure the loopback on the controllers, use the following commands.

configure

controller controllertype Rack/Slot/Instance/Port/Lanenumber

sec-admin-state maintenance

loopback [ line | internal ]

commit

Example

From R7.5.2 onwards, Loopback is supported for the OTUCn-REGEN mode on the coherent DSP controller.

The following example shows how to configure an internal loopback on a coherent DSP controller.

Sun Dec 26 14:34:02.733 UTC
RP/0/RP0/CPU0:ios(config)#controller CoherentDSP 0/3/0/12
RP/0/RP0/CPU0:ios(config-CoDSP)#secondary-admin-state maintenance
RP/0/RP0/CPU0:ios(config-CoDSP)#commit
Sun Dec 26 14:34:03.437 UTC
RP/0/RP0/CPU0:ios(config-CoDSP)#end

The following example shows how to verify internal loopback configured on a coherent DSP controller.


RP/0/RP0/CPU0:ios#show controller CoherentDSP 0/3/0/12

Sun Dec 26 14:34:28.391 UTC

Port : CoherentDSP 0/3/0/12
Controller State : Up
Inherited Secondary State : Normal
Configured Secondary State : Maintenance
Derived State : Maintenance
Loopback mode : Line
BER Thresholds : SF = 1.0E-5 SD = 1.0E-7
Performance Monitoring : Enable
Bandwidth : 200.0Gb/s

Alarm Information:
LOS = 1 LOF = 0 LOM = 0
OOF = 1 OOM = 0 AIS = 1
IAE = 0 BIAE = 0 SF_BER = 0
SD_BER = 0 BDI = 5 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0 FLEXO_GIDM = 0
FLEXO-MM = 0 FLEXO-LOM = 0 FLEXO-RDI = 0
FLEXO-LOF = 0
Detected Alarms : None

Bit Error Rate Information
PREFEC BER : 3.12E-07
POSTFEC BER : 0.00E+00
Q-Factor : 14.00 dB

Q-Margin : 6.40dB

TTI :
Remote hostname : ios
Remote interface : CoherentDSP 0/2/0/13
Remote IP addr : 0.0.0.0

FEC mode : O_FEC

Flexo-Mode : Enable
Flexo Details:
Tx GID : 1
TX IID : 1, 2,
Rx GID : 1
RX IID : 1, 2,

AINS Soak : None
AINS Timer : 0h, 0m
AINS remaining time : 0 seconds

Example 1

Configuring Loopback on the QXP Card

The following example shows how to configure internal loopback on a coherent DSP controller.

RP/0/RP0/CPU0:ios#configure
Fri Jul 8 10:42:51.329 UTC
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/0/0/0
RP/0/RP0/CPU0:ios(config-CoDSP)#secondary-admin-state maintenance
RP/0/RP0/CPU0:ios(config-CoDSP)#loopback internal
RP/0/RP0/CPU0:ios(config-CoDSP)#commit
Fri Jul 8 10:43:48.644 UTC
RP/0/RP0/CPU0:ios(config-CoDSP)#end

The following example shows how to verify the internal loopback configured on a coherent DSP controller.

RP/0/RP0/CPU0:ios#show controllers coherentDSP 0/0/0/0
Fri Jul 8 10:45:53.820 UTC
Port : CoherentDSP 0/0/0/0
Controller State : Down
Inherited Secondary State : Normal
Configured Secondary State : Maintenance
Derived State : Maintenance
Loopback mode : Internal
BER Thresholds : SF = 1.0E-5 SD = 1.0E-7
Performance Monitoring : Enable
Bandwidth : 400.0Gb/s
Alarm Information:
LOS = 2 LOF = 0 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0 SF_BER = 0
SD_BER = 0 BDI = 0 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0 FLEXO_GIDM = 0
FLEXO-MM = 0 FLEXO-LOM = 0 FLEXO-RDI = 0
FLEXO-LOF = 0
Detected Alarms : LOS
Bit Error Rate Information
PREFEC BER : 5.00E-01
POSTFEC BER : 0.00E+00
Q-Factor : 0.00 dB
Q-Margin : 0.00dB
OTU TTI Received
FEC mode : C_FEC
Flexo-Mode : Enable
Flexo Details:
Tx GID : 0
Rx GID : 0
AINS Soak : None
AINS Timer : 0h, 0m
AINS remaining time : 0 seconds

Example 2

The following example shows how to configure line loopback on a coherent DSP controller.

RP/0/RP0/CPU0:ios#configure
Fri Jul 8 10:48:48.577 UTC
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/0/0/0
RP/0/RP0/CPU0:ios(config-CoDSP)#secondary-admin-state maintenance
RP/0/RP0/CPU0:ios(config-CoDSP)#loopback line
RP/0/RP0/CPU0:ios(config-CoDSP)#commit
Fri Jul 8 10:49:26.809 UTC
RP/0/RP0/CPU0:ios(config-CoDSP)#end

The following example shows how to verify the line loopback configured on a coherent DSP controller.

RP/0/RP0/CPU0:ios#show controllers coherentDSP 0/0/0/0
Fri Jul 8 10:49:44.073 UTC
Port : CoherentDSP 0/0/0/0
Controller State : Down
Inherited Secondary State : Normal
Configured Secondary State : Maintenance
Derived State : Maintenance
Loopback mode : Line
BER Thresholds : SF = 1.0E-5 SD = 1.0E-7
Performance Monitoring : Enable
Bandwidth : 400.0Gb/s
Alarm Information:
LOS = 2 LOF = 0 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0 SF_BER = 0
SD_BER = 0 BDI = 0 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0 FLEXO_GIDM = 0
FLEXO-MM = 0 FLEXO-LOM = 0 FLEXO-RDI = 0
FLEXO-LOF = 0
Detected Alarms : LOS
Bit Error Rate Information
PREFEC BER : 5.00E-01
POSTFEC BER : 0.00E+00
Q-Factor : 0.00 dB
Q-Margin : 0.00dB
OTU TTI Received
FEC mode : C_FEC
Flexo-Mode : Enable
Flexo Details:
Tx GID : 0
Rx GID : 0
AINS Soak : None
AINS Timer : 0h, 0m
AINS remaining time : 0 seconds

Example 3

The following example shows how to configure internal loopback on the 400GE controller.

RP/0/RP0/CPU0:ios#configure
Fri Jul 8 11:19:26.286 UTC
RP/0/RP0/CPU0:ios(config)#controller FourHundredGigECtrlr 0/0/0/3
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#sec-admin-state maintenance
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#loopback internal
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#commit
Fri Jul 8 11:19:47.496 UTC
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#end

The following example shows how to verify the internal loopback configured on the 400GE controller.

RP/0/RP0/CPU0:ios#show controllers FourHundredGigECtrlr 0/0/0/3
Fri Jul 8 11:19:59.597 UTC
Operational data for interface FourHundredGigECtrlr0/0/0/3:
State:
Administrative state: enabled
Operational state: Down (Reason: State undefined)
LED state: Red On
Maintenance: Enabled
AINS Soak: None
Total Duration: 0 hour(s) 0 minute(s)
Remaining Duration: 0 hour(s) 0 minute(s) 0 second(s)
Laser Squelch: Disabled
Insert Idle Ingress: Disabled
Insert Idle Egress: Disabled
Phy:
Media type: Not known
Alarms:
Current:
Loss of Signal
Statistics:
FEC:
Corrected Codeword Count: 702710
Uncorrected Codeword Count: 1147
Autonegotiation disabled.
Operational values:
Speed: 400Gbps
Duplex: Full Duplex
Flowcontrol: None
Loopback: Internal
BER monitoring:
Not supported
Forward error correction: Standard (Reed-Solomon)
Holdoff Time: 0ms

Example 4

The following example shows how to configure line loopback on the 4X100GE MXP.

RP/0/RP0/CPU0:ios(config)#controller hundredGigECtrlr 0/3/0/1/1
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#loopback line 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#sec-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#commit

The following example shows how to verify the line loopback configured on the 4X100GE MXP.

RP/0/RP0/CPU0:ios#sh controllers hundredGigECtrlr 0/3/0/1/1
Fri Jul 22 10:34:39.730 UTC
Operational data for interface HundredGigECtrlr0/3/0/1/1:

State:
    Administrative state: enabled
    Operational state: Up
    LED state: Green On
    Maintenance: Enabled
    AINS Soak: None
      Total Duration: 0 hour(s) 0 minute(s)
      Remaining Duration: 0 hour(s) 0 minute(s) 0 second(s)
    Laser Squelch: Disabled
    Insert Idle Ingress: Disabled
    Insert Idle Egress: Disabled

Phy:
    Media type: Not known
    Statistics:
        FEC:
            Corrected Codeword Count: 6110368                  Valid: True       Start time: 13:10:41 Thu Jul 21 2022
            Uncorrected Codeword Count: 2771                   Valid: True       Start time: 13:10:41 Thu Jul 21 2022
        PCS:
            Total BIP errors: 63700992                         Valid: True       Start time: 13:10:41 Thu Jul 21 2022
            Total frame errors: 0                              Valid: False      Start time: 13:10:41 Thu Jul 21 2022
            Total Bad SH: 0                                    Valid: False      Start time: 13:10:41 Thu Jul 21 2022

Autonegotiation disabled.

Operational values:
    Speed: 100Gbps
    Duplex: Full Duplex
    Flowcontrol: None
    Loopback: Line
    BER monitoring:
        Not supported
    Forward error correction: Standard (Reed-Solomon)
    Holdoff Time: 0ms

Example 5

The following example shows how to configure internal loopback on the 4X100GE MXP.

RP/0/RP0/CPU0:ios#conf
RP/0/RP0/CPU0:ios(config)#controller hundredGigECtrlr 0/3/0/7/1
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#sec-admin-state maintenance 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#loopback internal 
RP/0/RP0/CPU0:ios(config-eth-ctrlr)#commit

The following example shows how to verify the internal loopback configured on the 4X100GE MXP.

RP/0/RP0/CPU0:ios#show controller HundredGigECtrlr 0/3/0/7/1
Fri Jul 22 10:40:34.928 UTC

Operational data for interface HundredGigECtrlr0/3/0/7/1:

State:
    Administrative state: enabled
    Operational state: Down (Reason: State undefined)
    LED state: Red On
    Maintenance: Enabled
    AINS Soak: None
      Total Duration: 0 hour(s) 0 minute(s)
      Remaining Duration: 0 hour(s) 0 minute(s) 0 second(s)
    Laser Squelch: Disabled
    Insert Idle Ingress: Disabled
    Insert Idle Egress: Disabled

Phy:
    Media type: Not known
    Alarms:
        Current:
            Loss of Signal
    Statistics:
        FEC:
            Corrected Codeword Count: 31426046
            Uncorrected Codeword Count: 2187

Autonegotiation disabled.

Operational values:
    Speed: 100Gbps
    Duplex: Full Duplex
    Flowcontrol: None
    Loopback: Internal
    BER monitoring:
        Not supported
    Forward error correction: Standard (Reed-Solomon)
    Holdoff Time: 0ms

LLDP Drop

Link Layer Discovery Protocol (LLDP) Snooping is enabled by default on all ethernet controllers.

To verify the LLDP neighbors, use the following commands:

RP/0/RP0/CPU0:ios#show lldp neighbors detail 
Tue Mar 12 11:49:20.819 IST
Capability codes:
        (R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
        (W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other

------------------------------------------------
Local Interface: HundredGigECtrlr0/1/0/7
Chassis id: 008a.96cd.34e1
Port id: Hu0/0/0/4
Port Description - not advertised
System Name: ncs5500_node

System Description: 
 6.1.4, NCS-5500

Time remaining: 116 seconds
Hold Time: 120 seconds
System Capabilities: R
Enabled Capabilities: R
Management Addresses - not advertised
Peer MAC Address: 00:8a:96:cd:34:10


------------------------------------------------
Local Interface: HundredGigECtrlr0/1/0/13
Chassis id: 008a.96cd.34e1
Port id: Hu0/0/0/5
Port Description - not advertised
System Name: ncs5500_node

System Description: 
 6.1.4, NCS-5500

Time remaining: 90 seconds
Hold Time: 120 seconds
System Capabilities: R
Enabled Capabilities: R
Management Addresses - not advertised
Peer MAC Address: 00:8a:96:cd:34:14


Total entries displayed: 2


RP/0/RP0/CPU0:ios#show lldp neighbors
Tue Mar 12 16:17:56.713 IST
Capability codes:
        (R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
        (W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other

Device ID       Local Intf               Hold-time  Capability     Port ID
ncs5500_node    HundredGigECtrlr0/1/0/7  120        R               Hu0/0/0/4
ncs5500_node    HundredGigECtrlr0/1/0/13 120        R               Hu0/0/0/5

Total entries displayed: 2

When you enable LLDP drop on the client controller ports of the muxponder or muxponder slice, the LLDP frames drop on the ports without forwarding.


Note
LLDP on 400GE is not supported on the OTN-XP card.

Configuring LLDP Drop

You can configure the LLDP drop for a muxponder or muxponder slice. By default, the LLDP drop status is set to False. On enabling the LLDP Drop, the status is set to True.

To configure LLDP drop on a muxponder use the following command:

configure

hw-module location location mxponder drop-lldp

Note
Use the no form of the command to disable LLDP drop.

commit

Limitation

  • When you disable LLDP globally, the LLDP gets disabled on all the interfaces.


Note

By default, LLDP is enabled for NCS 1004. But when you enable and disable LLDP in the global configuration mode, LLDP gets disabled on all the interfaces.

Workaround: You must enable LLDP globally or reload the Router. 

RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios#hw-module location 0/1 mxponder drop-lldp
RP/0/RP0/CPU0:ios#commit

configure

hw-module location location mxponder-slice slice-number drop-lldp

Note
Use the no form of the command to disable LLDP drop.

To configure LLDP drop on a muxponder slice, use the following command:

commit

The following is a sample in which slice 0 client ports are enabled with LLDP drop.


RP/0/RP0/CPU0:ios#configure
RP/0/RP0/CPU0:ios(config)#hw-module location 0/1 mxponder-slice 0 drop-lldp
RP/0/RP0/CPU0:ios(config)#commit

Verifying the Status of LLDP Drop

To verify the LLDP drop enabled status, use the following command. 

RP/0/RP0/CPU0:ios#show hw-module location all mxponder
Fri Feb 22 13:22:19.281 UTC

Location:             0/0
Client Bitrate:       NONE
Trunk  Bitrate:       NONE
Status:               Not Provisioned


Location:             0/1
Slice ID:             0
Client Bitrate:       100GE
Trunk  Bitrate:       500G
Status:               Provisioned
LLDP Drop Enabled:    FALSE
Client Port                     Mapper/Trunk Port          CoherentDSP0/1/0/0
                                Traffic Split Percentage

HundredGigECtrlr0/1/0/2         ODU40/1/0/0/0                      100
HundredGigECtrlr0/1/0/3         ODU40/1/0/0/1                      100
HundredGigECtrlr0/1/0/4         ODU40/1/0/0/2                      100
HundredGigECtrlr0/1/0/5         ODU40/1/0/0/3                      100
HundredGigECtrlr0/1/0/6         ODU40/1/0/0/4                      100


Location:             0/1
Slice ID:             1
Client Bitrate:       100GE
Trunk  Bitrate:       500G
Status:               Provisioned
LLDP Drop Enabled:    FALSE
Client Port                     Mapper/Trunk Port          CoherentDSP0/1/0/1
                                Traffic Split Percentage

HundredGigECtrlr0/1/0/8         ODU40/1/0/1/0                      100
HundredGigECtrlr0/1/0/9         ODU40/1/0/1/1                      100
HundredGigECtrlr0/1/0/10        ODU40/1/0/1/2                      100
HundredGigECtrlr0/1/0/11        ODU40/1/0/1/3                      100
HundredGigECtrlr0/1/0/12        ODU40/1/0/1/4                      100


Location:             0/2
Slice ID:             0
Client Bitrate:       100GE
Trunk  Bitrate:       500G
Status:               Provisioned
LLDP Drop Enabled:    FALSE
Client Port                     Mapper/Trunk Port          CoherentDSP0/2/0/0
                                Traffic Split Percentage

HundredGigECtrlr0/2/0/2         ODU40/2/0/0/0                      100
HundredGigECtrlr0/2/0/3         ODU40/2/0/0/1                      100
HundredGigECtrlr0/2/0/4         ODU40/2/0/0/2                      100
HundredGigECtrlr0/2/0/5         ODU40/2/0/0/3                      100
HundredGigECtrlr0/2/0/6         ODU40/2/0/0/4                      100


Location:             0/2
Slice ID:             1
Client Bitrate:       100GE
Trunk  Bitrate:       500G
Status:               Provisioned
LLDP Drop Enabled:    FALSE
Client Port                     Mapper/Trunk Port          CoherentDSP0/2/0/1
                                Traffic Split Percentage

HundredGigECtrlr0/2/0/8         ODU40/2/0/1/0                      100
HundredGigECtrlr0/2/0/9         ODU40/2/0/1/1                      100
HundredGigECtrlr0/2/0/10        ODU40/2/0/1/2                      100
HundredGigECtrlr0/2/0/11        ODU40/2/0/1/3                      100
HundredGigECtrlr0/2/0/12        ODU40/2/0/1/4                      100


Location:             0/3
Slice ID:             0
Client Bitrate:       100GE
Trunk  Bitrate:       300G
Status:               Provisioned
LLDP Drop Enabled:    TRUE
Client Port                     Mapper/Trunk Port          CoherentDSP0/3/0/0
                                Traffic Split Percentage

HundredGigECtrlr0/3/0/2         ODU40/3/0/0/0                      100
HundredGigECtrlr0/3/0/3         ODU40/3/0/0/1                      100
HundredGigECtrlr0/3/0/4         ODU40/3/0/0/2                      100

Trail Trace Identifier

The Trail trace identifier (TTI) feature helps you to identify the signal from the source to the destination within the network. You can configure the TTI sent or expected string only in ASCII string format. When the expected TTI string does not match the received TTI trace string, the controller goes down and the OTUK-TIM alarm is raised. To configure TTI on the coherent DSP controllers, use the following commands:

configure

controller coherentDSP R/S/I/P tti {sent | expected} ascii tti-string

commit


Note
The tti-string can have a maximum of 64 characters.

The following sample displays how to configure TTI on a coherent DSP controller with the sent and expected strings set to the same ASCII string. The state of the controller is up. 

RP/0/RP0/CPU0:ios#config
Fri Mar 15 08:03:02.094 UTC
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/1/0/1 tti sent ascii 1234
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/1/0/1 tti expected ascii 1234
RP/0/RP0/CPU0:ios(config)#commit
Fri Mar 15 08:03:49.725 UTC
RP/0/RP0/CPU0:ios(config)#exit
RP/0/RP0/CPU0:ios#show controllers coherentDSP 0/1/0/1
Fri Mar 15 08:04:06.290 UTC

Port                                            : CoherentDSP 0/1/0/1
Controller State                                : Up
Inherited Secondary State                       : Normal
Configured Secondary State                      : Normal
Derived State                                   : In Service
Loopback mode                                   : None
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable

Alarm Information:
LOS = 0 LOF = 0 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 1 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0
Detected Alarms                                 : None

Bit Error Rate Information
PREFEC  BER                                     : 7.7E-03
POSTFEC BER                                     : 0.0E+00

OTU TTI Sent
     OPERATOR SPECIFIC  ASCII                   : 1234
                                                :
     OPERATOR SPECIFIC HEX                      : 31323334000000000000000000000000
                                                : 00000000000000000000000000000000
OTU TTI Received
     OPERATOR SPECIFIC  ASCII                   : 1234
                                                :
     OPERATOR SPECIFIC HEX                      : 31323334000000000000000000000000
                                                : 00000000000000000000000000000000
OTU TTI Expected
     OPERATOR SPECIFIC  ASCII                   : 1234
                                                :
     OPERATOR SPECIFIC HEX                      : 31323334000000000000000000000000
                                                : 00000000000000000000000000000000
FEC mode                                        : Soft-Decision 27

AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

The following example shows how to configure TTI on a coherent DSP controller with the sent and expected strings set to different ASCII strings. The state of the controller goes down and the TIM alarm is raised. 

RP/0/RP0/CPU0:ios#config
Fri Mar 15 08:54:29.780 UTC
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/1/0/1 tti sent ascii 1234
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/1/0/1 tti expected ascii 5678
RP/0/RP0/CPU0:ios(config)#commit
Fri Mar 15 08:56:12.293 UTC
RP/0/RP0/CPU0:ios(config)#exit
RP/0/RP0/CPU0:ios#show controllers coherentDSP 0/1/0/1
Fri Mar 15 08:56:33.910 UTC

Port                                            : CoherentDSP 0/1/0/1
Controller State                                : Down
Inherited Secondary State                       : Normal
Configured Secondary State                      : Normal
Derived State                                   : In Service
Loopback mode                                   : None
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable

Alarm Information:
LOS = 1 LOF = 0 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 3 TIM = 1
FECMISMATCH = 0 FEC-UNC = 0
Detected Alarms                                 : BDI TIM

Bit Error Rate Information
PREFEC  BER                                     : 8.2E-03
POSTFEC BER                                     : 0.0E+00

OTU TTI Sent
     OPERATOR SPECIFIC  ASCII                   : 1234
                                                :
     OPERATOR SPECIFIC HEX                      : 31323334000000000000000000000000
                                                : 00000000000000000000000000000000
OTU TTI Received
     OPERATOR SPECIFIC  ASCII                   : 1234
                                                :
     OPERATOR SPECIFIC HEX                      : 31323334000000000000000000000000
                                                : 00000000000000000000000000000000
OTU TTI Expected
     OPERATOR SPECIFIC  ASCII                   : 5678
                                                :
     OPERATOR SPECIFIC HEX                      : 35363738000000000000000000000000
                                                : 00000000000000000000000000000000
FEC mode                                        : Soft-Decision 27

AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

Configure TTI on OTN-XP Card

You can configure the TTI sent or expected string in the full ASCII format, or Source Access Point Identifier (SAPI)/Destination Access Point Identifier (DAPI) format on OTU, ODU, ODU-flex, ODUCn, and coherentDSP controllers for the OTN-XP card.

From R7.3.1 onwards, coherentDSP controller supports only the full ASCII string format.

From R7.3.2 onwards, coherentDSP controller supports SAPI/DAPI string format in addition to the full ASCII string format.

You can configure TTI for the following muxponder modes:

  • 10G-Grey-MXP

  • 4x100G-MXP-400G-TXP

The following table lists the ASCII format that is supported on each muxponder mode for TTI:

Table 1. ASCII Format Supported on Each Muxponder Mode

Muxponder Mode

ASCII with Character String

Controller

10G Grey

Full ASCII

64-character

OTU2, OTU2E, OTU4, ODU4, ODU2E (10G mapper)

SAPI ASCII

15-character

OTU2, OTU2E, OTU4, ODU4, ODU2E (10G mapper)

DAPI ASCII

15-character

OTU2, OTU2E, OTU4, ODU4, ODU2E (10G mapper)

Operator-specific ASCII

32-character

OTU2, OTU2E, OTU4, ODU4, ODU2E (10G mapper)

4x100G-MXP-400G-TXP

Full ASCII

64-character

OTU4, coherentDSP, ODUC4, ODU4 (100G mapper), and ODU-FLEX (400G mapper)

SAPI ASCII

15-character

OTU4, coherentDSP, ODUC4, ODU4 (100G mapper), and ODU-FLEX (400G mapper)

DAPI ASCII

15-character

OTU4, coherentDSP, ODUC4, ODU4 (100G mapper), and ODU-FLEX (400G mapper)

Operator-specific ASCII

32-character

OTU4, coherentDSP, ODUC4, ODU4 (100G mapper), and ODU-FLEX (400G mapper)

To configure TTI, use the following commands:

configure

controller controller-type R/S/I/P tti {sent | expected} {ascii | sapi ascii | dapi ascii | operator-specific ascii } tti-string

commit


Note

We recommend that you configure TTI in the SAPI/DAPI ASCII format.
The following is a sample configuration on ODU4 controller: 

RP/0/RP0/CPU0:ios#configure 
Thu Sep 30 17:19:11.804 UTC
Current Configuration Session  Line       User     Date                     Lock
00001000-00005c15-00000000     vty0       root     Thu Sep 30 11:50:12 2021 
RP/0/RP0/CPU0:ios(config)#controller odU4 0/1/0/12/7 tti sent sapi ascii cisco123
RP/0/RP0/CPU0:ios(config)#controller odU4 0/1/0/12/7 tti expect sapi ascii 123cisco
RP/0/RP0/CPU0:ios(config)#controller odU4 0/1/0/12/7 tti sent dapi ascii dapistring123
RP/0/RP0/CPU0:ios(config)#controller odU4 0/1/0/12/7 tti expected dapi ascii 123stringdapi
RP/0/RP0/CPU0:ios(config)#controller odU4 0/1/0/12/7 tti sent operator-specific ascii operation123
RP/0/RP0/CPU0:ios(config)#controller odU4 0/1/0/12/7 tti expected operator-specific ascii 123operator
RP/0/RP0/CPU0:ios(config)#commit 
Thu Sep 30 17:21:49.521 UTC
RP/0/RP0/CPU0:ios(config)#end
The following is a sample output of the TTI configuration on the ODU4 controller:

RP/0/RP0/CPU0:ios#show controllers odU4 0/1/0/12/7 
Thu Sep 30 17:22:30.658 UTC

Port                                            : ODU4 0/1/0/12/7
Controller State                                : Down
Inherited Secondary state                       : Normal
Configured Secondary state                      : Normal
Derived State                                   : In Service
Loopback mode                                   : None
BER Thresholds                                  : SF = 1.0E-6  SD = 1.0E-7

Performance Monitoring                          : Enable 

Path Monitoring Mode                            : Operational 
PM TIM-CA state                                 : Disable 

Alarm Information:
AIS = 3 IAE = 0 BIAE = 0
SF_BER = 0      SD_BER = 0      BDI = 1
OCI = 0 LCK = 2 PTIM = 0
TIM = 1 CSF = 0 GFP LFD = 0
GFP LOCS = 0    GFP LOCCS = 0   GFP UPM = 0

Detected Alarms                                 : TIM 

ODU TTI Sent 
     SAPI  ASCII                                : c i s c o 1 2 3 
     SAPI  HEX                                  : 00636973636F31323300000000000000
     DAPI  ASCII                                : d a p i s t r i n g 1 2 3 
     DAPI  HEX                                  : 0064617069737472696E673132330000
     OPERATOR SPECIFIC ASCII                    : operation123
     OPERATOR SPECIFIC HEX                      : 6F7065726174696F6E3132330000000000000000000000000000000000000000
ODU TTI Received 

ODU TTI Expected 
     SAPI  ASCII                                : 1 2 3 c i s c o 
     SAPI  HEX                                  : 00313233636973636F00000000000000
     DAPI  ASCII                                : 1 2 3 s t r i n g d a p i 
     DAPI  HEX                                  : 00313233737472696E67646170690000
     OPERATOR SPECIFIC ASCII                    : 123operator
     OPERATOR SPECIFIC HEX                      : 3132336F70657261746F72000000000000000000000000000000000000000000
AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

Private Line Emulation(PLE) supported           : No

You can configure TTI on OTUCn-REGEN mode on the OTN-XP Card.

The following sample displays how to configure TTI on a coherent DSP controller port 12 on the OTUCn-REGEN mode.

Mon Dec 27 12:03:53.642 UTC
RP/0/RP0/CPU0:ios(config)#controller CoherentDSP 0/3/0/12 tti sent ascii 1234cisco
RP/0/RP0/CPU0:ios(config)#commit
Mon Dec 27 12:03:54.333 UTC
RP/0/RP0/CPU0:ios(config)#end
Mon Dec 27 12:03:55.434 UTC
RP/0/RP0/CPU0:ios(config)#controller CoherentDSP 0/3/0/12 tti expected ascii cisco1234
RP/0/RP0/CPU0:ios(config)#commit
Mon Dec 27 12:03:56.137 UTC
RP/0/RP0/CPU0:ios(config)#end

The following sample verifies the TTI configuration on the inverse muxponder configured on the OTUCn-REGEN mode.

RP/0/RP0/CPU0:ios#show controllers coherentDSP 0/0/0/12
Tue May 24 17:49:14.301 UTC

Port                                            : CoherentDSP 0/0/0/12
Controller State                                : Up
Inherited Secondary State                       : Normal
Configured Secondary State                      : Normal
Derived State                                   : In Service
Loopback mode                                   : None
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable
Bandwidth                                       : 400.0Gb/s

Alarm Information:
LOS = 0 LOF = 0 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 1 TIM = 1
FECMISMATCH = 0 FEC-UNC = 0     FLEXO_GIDM = 0
FLEXO-MM = 0    FLEXO-LOM = 0   FLEXO-RDI = 0
FLEXO-LOF = 0
Detected Alarms                                 : None

Bit Error Rate Information
PREFEC  BER                                     : 1.55E-04
POSTFEC BER                                     : 0.00E+00
Q-Factor                                        : 11.10 dB

Q-Margin                                        : 4.70dB

OTU TTI Sent
     FULL TTI ASCII                             : cisco123
                                                :
     FULL TTI HEX                               : 636973636F3132330000000000000000
                                                : 00000000000000000000000000000000
OTU TTI Received
     FULL TTI ASCII                             : 123cisco
                                                :
     FULL TTI HEX                               : 313233636973636F0000000000000000
                                                : 00000000000000000000000000000000
OTU TTI Expected
     FULL TTI ASCII                             : 123cisco
                                                :
     FULL TTI HEX                               : 313233636973636F0000000000000000
                                                : 00000000000000000000000000000000
FEC mode                                        : O_FEC

Flexo-Mode                                      : Enable
Flexo Details:
        Tx GID                                  : 1
        TX IID                                  : 1, 2, 3, 4,
        Rx GID                                  : 1
        RX IID                                  : 1, 2, 3, 4,

AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds


RP/0/RP0/CPU0:ios#
RP/0/RP0/CPU0:ios#show running-config controller coherentDSP 0/0/0/12
Tue May 24 17:49:21.749 UTC
controller CoherentDSP0/0/0/12
tti
  expected ascii 123cisco
  sent ascii cisco123
!
!

Configure TTI on Inverse Muxponder Configuration on the OTN-XP Card

The following sample displays how to configure TTI on a coherent DSP controller port 12 on the OTN-XP in inverse muxponder configuration mode.


Note
TTI configuration is not supported on the DSP controller port 13. 
RP/0/RP0/CPU0:ios#configure
Thu Sep 30 14:18:13.288 UTC
RP/0/RP0/CPU0:ios(config)#controller coherentDSP 0/2/0/12 
RP/0/RP0/CPU0:ios(config-CoDSP)#tti sent sapi ascii cisco
RP/0/RP0/CPU0:ios(config-CoDSP)#commit

The following sample verifies the TTI configuration on the inverse muxponder configured on the OTN-XP Card.

RP/0/RP0/CPU0:ios#show controllers coherentDSP 0/2/0/12 
Thu Sep 30 14:19:05.367 UTC

Port                                            : CoherentDSP 0/2/0/12
Controller State                                : Up
Inherited Secondary State                       : Normal
Configured Secondary State                      : Maintenance
Derived State                                   : Maintenance
Loopback mode                                   : Internal
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable
Bandwidth                                       : 200.0Gb/s

Alarm Information:
LOS = 2 LOF = 0 LOM = 0
OOF = 1 OOM = 0 AIS = 1
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 0 TIM = 0
FECMISMATCH = 0 FEC-UNC = 0     FLEXO_GIDM = 0
FLEXO-MM = 0    FLEXO-LOM = 0   FLEXO-RDI = 1
FLEXO-LOF = 0   
Detected Alarms                                 : None

Bit Error Rate Information
PREFEC  BER                                     : 4.11E-09 
POSTFEC BER                                     : 0.00E+00 
Q-Factor                                        : 14.90 dB 

Q-Margin                                        : 8.60dB 

OTU TTI Sent 
     SAPI  ASCII                                : c i s c o 
     SAPI  HEX                                  : 00636973636F00000000000000000000
     DAPI  ASCII                                : 
     DAPI  HEX                                  : 
     OPERATOR SPECIFIC ASCII                    : 
     OPERATOR SPECIFIC HEX                      : CDCDCDCDED00DBBE2100000000000000050D9D29AD7F00007603BADC7698BADC
OTU TTI Received 
     SAPI  ASCII                                : c i s c o 
     SAPI  HEX                                  : 00636973636F00000000000000000000
FEC mode                                        : O_FEC

Flexo-Mode                                      : Enable
Flexo Details:
        Tx GID                                  : 1
        TX IID                                  : 1, 2, 
        Rx GID                                  : 1
        RX IID                                  : 1, 2, 

Flexo Peers Information:
        Controller                              : CoherentDSP0_2_0_13 
        OTUCn rate                              : OTUC2


AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

Enable TIM CA on Path Monitoring Layer

You can enable Trace Identifier Mismatch (TIM) consequent action (CA) on the Path Monitoring (PM) layer using the pm-tim-ca command on mapper ODUs for Ethernet controller. The TTI transmit string in the SAPI/DAPI format is not configurable on ODUs that are transparent.

For example, the clients that are supported are ODU4, ODU2, and ODU2E, and lower-order ODUs such as ODU2 or ODU2E.

You can configure pm-tim-ca only on mapper ODUs such as ODU2E (10G mapper), ODU4 (100G mapper), and ODU-FLEX (400G mapper).

To configure pm-tim-ca on mapper ODU in the 4x100G-MXP-400G-TXP muxponder mode, use the following commands

configure

controller controller-type R/S/I/P

pm-tim-ca

commit

Troubleshoot the Trunk Port

Procedure

Step 1

show controller coherentDSP R/S/I/P

Displays details of the coherent DSP controller.

Example:

RP/0/RP0/CPU0:ios# show controller coherentDSP 0/0/0/0
Tue Feb 25 11:26:08.235 UTC                                                                                                                                    

Port                                            : CoherentDSP 0/0/0/0
Controller State                                : Up                 
Inherited Secondary State                       : Normal             
Configured Secondary State                      : Normal             
Derived State                                   : In Service         
Loopback mode                                   : None               
BER Thresholds                                  : SF = 1.0E-5  SD = 1.0E-7
Performance Monitoring                          : Enable                  
Bandwidth                                       : 50.0Gb/s                
Alarm Information:
LOS = 1 LOF = 0 LOM = 0
OOF = 0 OOM = 0 AIS = 0
IAE = 0 BIAE = 0        SF_BER = 0
SD_BER = 0      BDI = 0 TIM = 0   
FECMISMATCH = 0 FEC-UNC = 0       
Detected Alarms                                 : None
Bit Error Rate Information
PREFEC  BER                                     : 0.00E+00 
POSTFEC BER                                     : 0.00E+00 
Q-Factor                                        : 0.00 dB  
Q-Margin                                        : -5.00dB
Instantaneous Q_margin                        :  0 dB


TTI :
        Remote IP addr                          : 0.0.0.0
FEC mode                                        : Soft-Decision 15

AINS Soak                                       : None
AINS Timer                                      : 0h, 0m
AINS remaining time                             : 0 seconds

In the above output, you can verify the state of the controller and also verify the alarms related to the trunk port.

Step 2

show controller optics R/S/I/P

Displays details of the optics controller.

Example:

RP/0/RP0/CPU0:ios# show controller optics  0/1/0/3
 Thu Feb 21 19:45:41.088 UTC

 Controller State: Up

 Transport Admin State: Automatic In Service

 Laser State: On

 LED State: Green

 Optics Status

         Optics Type:  Grey optics

         Alarm Status:
         -------------
         Detected Alarms: None


         LOS/LOL/Fault Status:

         Alarm Statistics:

         -------------
         HIGH-RX-PWR = 0            LOW-RX-PWR = 0
         HIGH-TX-PWR = 0            LOW-TX-PWR = 0
         HIGH-LBC = 0               HIGH-DGD = 0
         OOR-CD = 0                 OSNR = 0
         WVL-OOL = 0                MEA  = 0
         IMPROPER-REM = 0
         TX-POWER-PROV-MISMATCH = 0

         Performance Monitoring: Enable

         THRESHOLD VALUES
         ----------------

         Parameter                 High Alarm  Low Alarm  High Warning  Low Warning
         ------------------------  ----------  ---------  ------------  -----------
         Rx Power Threshold(dBm)          4.9      -12.0           0.0          0.0
         Tx Power Threshold(dBm)          3.5      -10.1           0.0          0.0
         LBC Threshold(mA)                N/A        N/A          0.00         0.00

         LBC High Threshold = 98 %
         Polarization parameters not supported by optics

        Total TX Power = 6.39 dBm

        Total RX Power = 5.85 dBm

         Lane  Laser Bias    TX Power    RX Power  Output Frequency
         ----  ----------  ----------  ----------  ----------------
           1      75.0 %    0.59 dBm    0.63 dBm  230.43 THz
           2      68.6 %    0.06 dBm   -0.68 dBm  230.43 THz
           3      69.0 %    0.26 dBm   -0.63 dBm  230.43 THz
           4      69.1 %    0.56 dBm   -0.10 dBm  230.43 THz

 Transceiver Vendor Details

         Form Factor            : QSFP28
         Name                   : CISCO-FINISAR
         Part Number            : FTLC1152RGPL-C2
         Rev Number             : CISCO-FINISAR
         Serial Number          : FNS22150LEC
         PID                    : QSFP-100G-CWDM4-S
         VID                    : V02
         CISCO-FINISAR
         Date Code(yy/mm/dd)    : 18/04/11
         Fiber Connector Type: LC
         Sonet Application Code: Not Set
         Ethernet Compliance Code: 100GBASE-CWDM4

 Transceiver Temperature : 32 Celsius



 AINS Soak                : Running
 AINS Timer               : 0h, 15m
 AINS remaining time      : 771 seconds

In the above output, you can verify the state of the controller, LED state, TX power, RX power, OSNR, and the alarms.

Step 3

If there is an LOS alarm on the trunk port, verify the fiber continuity to the port of NCS 1004 and fix the fiber connection.

What to do next

Verify the performance monitoring parameters of the Optics, Ethernet, and coherent DSP controllers. For more information, see .

Troubleshoot a Failed Commit Configuration

Use the show configuration failed command to get information on why the configuration failed. 


RP/0/RP0/CPU0:ios(config-hwmod-mxp)#show configuration failed
Fri Nov 26 15:27:22.629 UTC
!! SEMANTIC ERRORS: This configuration was rejected by 
!! the system due to semantic errors. The individual 
!! errors with each failed configuration command can be 
!! found below.


hw-module location 0/0
 mxponder-slice 0
  client-port-rate 1 client-type 400GE
!!% Total group bandwidth exceeds the limit: Total group bandwidth exceeds the limit
 !
!
end

Verify the Performance Monitoring Parameters of Controllers

Performance monitoring (PM) parameters are used by service providers to gather, store, set thresholds for, and report performance data for early detection of problems. The user can retrieve both current and historical PM counters for the various controllers in 15 minutes and 1 day intervals.

Procedure

show controllers controllertype R/S/I/P { pm { current | history } { 30 sec | 15-min | 24-hour } { optics | ether | fec | otn } linenumber }

The following sample output displays the current performance monitoring parameters of the Optics controller in 15-minute intervals. Client optics have four lanes. 
RP/0/RP0/CPU0:ios#show controller optics 0/1/0/3 pm current 15-min optics 3

Sat Feb  9 19:33:42.480 UTC

Optics in the current interval [19:30:00 - 19:33:42 Sat Feb 9 2019]

Optics current bucket type : Valid
             MIN       AVG       MAX      Operational      Configured      TCA   Operational      Configured     TCA
                                          Threshold(min)   Threshold(min) (min) Threshold(max)   Threshold(max) (max)
LBC[% ]      : 0.0       0.0       0.0      0.0               NA              NO   100.0            NA              NO

OPT[dBm]     : -40.00    -40.00    -40.00   -30.00            NA              NO   63.32            NA              NO

OPR[dBm]     : -40.00    -40.00    -40.00   -30.00            NA              NO   63.32            NA              NO

FREQ_OFF[Mhz]: 0         0         0        0                 NA              NO   0                NA              NO
The following sample output displays the current performance monitoring parameters of the Optics controller in 15-minute intervals. Trunk optics have one lane. 
RP/0/RP0/CPU0:ios#show controller optics 0/2/0/1  pm  current 15-min optics 1

Sat Feb  9 11:19:15.234 UTC

Optics in the current interval [11:15:00 - 11:19:15 Sat Feb 9 2019]

Optics current bucket type : Valid
             MIN       AVG       MAX      Operational      Configured      TCA   Operational      Configured     TCA
                                          Threshold(min)   Threshold(min) (min) Threshold(max)   Threshold(max) (max)
LBC[% ]      : 0.0       0.0       0.0      0.0               NA              NO   100.0            NA              NO

OPT[dBm]     : -1.51     -1.49     -1.48    -30.00            NA              NO   63.32            NA              NO

OPR[dBm]     : -9.11     -9.07     -9.03    -30.00            NA              NO   63.32            NA              NO

CD[ps/nm]    : 13        15        18       -180000           NA              NO   180000           NA              NO

DGD[ps ]     : 2.00      2.33      3.00     0.01              NA              NO   21474836.46      NA              NO

SOPMD[ps^2]  : 5.00      33.02     79.00    0.01              NA              NO   21474836.46      NA              NO

OSNR[dB]     : 31.50     31.97     32.50    0.01              NA              NO   21474836.46      NA              NO

PDL[dB]      : 0.20      0.34      0.50     0.01              NA              NO   21474836.46      NA              NO

PCR[rad/s]   : 0.00      19.92     93.00    0.01              NA              NO   21474836.46      NA              NO

RX_SIG[dBm]  : -9.05     -9.02     -8.99    -30.00            NA              NO   63.32            NA              NO

FREQ_OFF[Mhz]: -302      -178      -74      -1500             NA              NO   1500             NA              NO

The following sample output displays the current performance monitoring parameters of the Ethernet controller 15-minute intervals. 


RP/0/RP0/CPU0:ios#show controller HundredGigECtrlr 0/1/0/2 pm current 15-min ether
Fri Aug 30 00:37:53.527 UTC
 
ETHER in the current interval [00:30:00 - 00:37:53 Fri Aug 30 2019]
 
ETHER current bucket type : Valid
RX-UTIL[%]                : 100.00               Threshold : 0.00         TCA(enable) : NO
TX-UTIL[%]                : 10.00                Threshold : 0.00         TCA(enable) : NO
RX-PKT                    : 3852414442           Threshold : 0            TCA(enable) : NO
STAT-PKT                  : 0                    Threshold : 0            TCA(enable) : NO
OCTET-STAT                : 5847965122956        Threshold : 0            TCA(enable) : NO
OVERSIZE-PKT              : 0                    Threshold : 0            TCA(enable) : NO
FCS-ERR                   : 0                    Threshold : 0            TCA(enable) : NO
LONG-FRAME                : 0                    Threshold : 0            TCA(enable) : NO
JABBER-STATS              : 0                    Threshold : 0            TCA(enable) : NO
64-OCTET                  : 0                    Threshold : 0            TCA(enable) : NO
65-127-OCTET              : 0                    Threshold : 0            TCA(enable) : NO
128-255-OCTET             : 0                    Threshold : 0            TCA(enable) : NO
256-511-OCTET             : 0                    Threshold : 0            TCA(enable) : NO
512-1023-OCTET            : 0                    Threshold : 0            TCA(enable) : NO
1024-1518-OCTET           : 0                    Threshold : 0            TCA(enable) : NO
IN-UCAST                  : 0                    Threshold : 0
IN-MCAST                  : 0                    Threshold : 0            TCA(enable) : NO
IN-BCAST                  : 0                    Threshold : 0            TCA(enable) : NO
OUT-UCAST                 : 0                    Threshold : 0            TCA(enable) : NO
OUT-BCAST                 : 0                    Threshold : 0            TCA(enable) : NO
OUT-MCAST                 : 0                    Threshold : 0            TCA(enable) : NO
TX-PKT                    : 7053588067           Threshold : 0            TCA(enable) : NO
OUT-OCTET                 : 451429636288         Threshold : 0            TCA(enable) : NO
IFIN-ERRORS               : 0                    Threshold : 0            TCA(enable) : NO
IFIN-OCTETS               : 0                    Threshold : 0            TCA(enable) : NO
STAT-MULTICAST-PKT        : 0                    Threshold : 0            TCA(enable) : NO
STAT-BROADCAST-PKT        : 0                    Threshold : 0            TCA(enable) : NO
STAT-UNDERSIZED-PKT       : 0                    Threshold : 0            TCA(enable) : NO
IN_GOOD_BYTES             : 5847965122956        Threshold : 0            TCA(enable) : NO
IN_GOOD_PKTS              : 3852414442           Threshold : 0            TCA(enable) : NO
IN_DROP_OTHER             : 0                    Threshold : 0            TCA(enable) : NO
OUT_GOOD_BYTES            : 451429636288         Threshold : 0            TCA(enable) : NO
OUT_GOOD_PKTS             : 7053588067           Threshold : 0            TCA(enable) : NO
IN_PKT_64_OCTET           : 0                    Threshold : 0            TCA(enable) : NO
IN_PKTS_65_127_OCTETS     : 0                    Threshold : 0            TCA(enable) : NO
IN_PKTS_128_255_OCTETS    : 0                    Threshold : 0            TCA(enable) : NO
IN_PKTS_256_511_OCTETS    : 0                    Threshold : 0            TCA(enable) : NO
IN_PKTS_512_1023_OCTETS   : 0                    Threshold : 0            TCA(enable) : NO
IN_PKTS_1024_1518_OCTETS  : 3852414442           Threshold : 0            TCA(enable) : NO
OUT_PKT_64_OCTET          : 7053588067           Threshold : 0            TCA(enable) : NO
OUT_PKTS_65_127_OCTETS    : 0                    Threshold : 0            TCA(enable) : NO
OUT_PKTS_128_255_OCTETS   : 0                    Threshold : 0            TCA(enable) : NO
OUT_PKTS_256_511_OCTETS   : 0                    Threshold : 0            TCA(enable) : NO
OUT_PKTS_512_1023_OCTETS  : 0                    Threshold : 0            TCA(enable) : NO
OUT_PKTS_1024_1518_OCTETS : 0                    Threshold : 0            TCA(enable) : NO
TX_UNDERSIZED_PKT         : 0                    Threshold : 0            TCA(enable) : NO
TX_OVERSIZED_PKT          : 0                    Threshold : 0            TCA(enable) : NO
TX_JABBER                 : 0                    Threshold : 0            TCA(enable) : NO
TX_BAD_FCS                : 0                    Threshold : 0            TCA(enable) : NO

The following sample output displays the current performance monitoring parameters of the Coherent DSP controller 15-minute intervals. 

RP/0/RP0/CPU0:ios#show controller coherentDSP 0/2/0/1 pm current 15-min  fec

Sat Feb  9 11:23:42.196 UTC

g709 FEC in the current interval [11:15:00 - 11:23:42 Sat Feb 9 2019]

FEC current bucket type : Valid
    EC-BITS   : 291612035786            Threshold : 903330                 TCA(enable)  : YES
    UC-WORDS  : 0                       Threshold : 5                      TCA(enable)  : YES

                MIN       AVG        MAX    Threshold   TCA     Threshold   TCA
                                              (min)    (enable)   (max)    (enable)
PreFEC BER  : 7.1E-03   7.2E-03   8.1E-03     0E-15        NO     0E-15       NO
PostFEC BER :   0E-15     0E-15     0E-15     0E-15        NO     0E-15       NO

The following sample output displays the current performance monitoring parameters of the optics controller 10-second intervals as flexi-bin. 


RP/0/RP0/CPU0:ios#show controllers optics 0/0/0/0 pm current flex-bin optics 1
Thu May 21 07:43:38.964 UTC

Optics in the current interval [07:43:30 - 07:43:38 Thu May 21 2020]

Flexible bin interval size: 10 seconds

Optics current bucket type : Valid
             MIN       AVG       MAX      Operational      Configured      TCA   Operational      Configured     TCA
                                          Threshold(min)   Threshold(min) (min) Threshold(max)   Threshold(max) (max)
LBC[% ]      : 0.0       0.0       0.0      0.0               NA              NO   0.0              NA              NO   
OPT[dBm]     : -0.13     -0.10     -0.06    0.00              NA              NO   0.00             NA              NO   
OPR[dBm]     : -3.01     -2.96     -2.92    0.00              NA              NO   0.00             NA              NO   
CD[ps/nm]    : -3        -2        -1       0                 NA              NO   0                NA              NO   
DGD[ps ]     : 1.00      1.67      2.00     0.00              NA              NO   0.00             NA              NO   
SOPMD[ps^2]  : 17.00     37.00     81.00    0.00              NA              NO   0.00             NA              NO   
OSNR[dB]     : 37.60     37.60     37.60    0.00              NA              NO   0.00             NA              NO   
PDL[dB]      : 0.60      0.66      0.70     0.00              NA              NO   0.00             NA              NO   
PCR[rad/s]   : 0.00      29.11     80.00    0.00              NA              NO   0.00             NA              NO   
RX_SIG[dBm]  : -3.49     -3.41     -3.36    0.00              NA              NO   0.00             NA              NO   
FREQ_OFF[Mhz]: 191       241       301      0                 NA              NO   0                NA              NO   
SNR[dB]      : 14.50     14.62     14.70    0.00              NA              NO   0.00             NA              NO   
SNR-AX[dB]   : 17.10     17.19     17.30    0.00              NA              NO   0.00             NA              NO   
SNR-AY[dB]   : 11.90     12.06     12.10    0.00              NA              NO   0.00             NA              NO   
SNR-BX[dB]   : 0.00      0.00      0.00     0.00              NA              NO   0.00             NA              NO   
SNR-BY[dB]   : 0.00      0.00      0.00     0.00              NA              NO   0.00             NA              NO   
SOP-S1       : 0.50      0.55      0.59     0.00              NA              NO   0.00             NA              NO   
SOP-S2       : -0.59     -0.52     -0.48    0.00              NA              NO   0.00             NA              NO   
SOP-S3       : -0.67     -0.64     -0.60    0.00              NA              NO   0.00             NA              NO   

Last clearing of "show controllers OPTICS" counters never

Using SNMP for Troubleshooting

The supported MIBs in NCS 1004 are documented in the SNMP chapter of Configuration Guide for Cisco NCS 1004.

Procedure

Use the following commands in EXEC mode to verify and monitor the SNMP for network monitoring and management.

  • show snmp — Displays the status of SNMP communications.

  • show snmp mib access — Displays the counters per OID that indicate the number of times an operation was done on an OID.

  • show snmp mib access time — Displays the timestamp of the last operation on an OID.

  • show snmp trace requests — Displays a log of the high level PDU processing trace points.

  • debug snmp packet — Displays information about every SNMP packet sent or received by NCS 1004.

  • debug snmp requests — Displays information about every SNMP request made by the SNMP manager.

Using Netconf for Troubleshooting

Netconf provides mechanisms to install, manipulate, and delete the configuration of network devices. The Netconf protocol provides a set of operations to manage device configurations and retrieve device state information.

Use the following commands in EXEC mode to retrieve device state information.

Before you begin

  • Verify the installation of k9sec package.

  • Generate the crypto key for SSH using the crypto key generate dsa command.


    Note
    If you access NCS 1004 after regenerating the crypto key, you must remove the ~/.ssh/known_hosts file as there will be a key mismatch between the host and NCS 1004.

  • Configure SSH. 

    
RP/0/RP0/CPU0:ios# configure
RP/0/RP0/CPU0(config)# ssh server v2
RP/0/RP0/CPU0(config)# ssh server netconf port 830 
RP/0/RP0/CPU0(config)# ssh server netconf  vrf default

    Note
    Port 830 is the default Netconf port.
  • Configure Netconf. 
    
RP/0/RP0/CPU0:ios# configure
RP/0/RP0/CPU0(config)# netconf-yang agent ssh

Procedure

Step 1

show netconf-yang clients

Displays the client details for netconf-yang.

Example:

RP/0/RP0/CPU0:ios# show netconf-yang clients
Fri Nov 26 15:28:50.942 UTC
No active netconf sessions found.

Step 2

show netconf-yang statistics

Displays the statistical details for netconf-yang.

Example:

RP/0/RP0/CPU0:ios# show netconf-yang statistics
Fri Nov 26 15:24:06.612 UTC
Summary statistics
                         # requests|             total time|   min time per request|   max time per request|   avg time per request|
other                             0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
close-session                     0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
kill-session                      0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
get-schema                        0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
get                               0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
get-config                        0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
edit-config                       0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
commit                            0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
cancel-commit                     0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
lock                              0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
unlock                            0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
discard-changes                   0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
validate                          0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
xml parse                         0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
netconf processor                 0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
YFW                               0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
pending requests                  0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
invoke rpc                        0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
copy-config                       0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
create-subscription               0|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|       0h  0m  0s   0ms|
List is empty.

Step 3

show netconf-yang trace

Debugs and verifies Netconf.

Example:

RP/0/RP0/CPU0:ios# show netconf-yang trace
Fri Nov 26 15:29:31.430 UTC
68703 wrapping entries (203392 possible, 161152 allocated, 0 filtered, 68703 total)
Nov 25 17:05:39.003 netconf/netconf.trace 0/RP0/CPU0 t8790 #61984: TRC: nc_pxs_ipc_notify_callback_fn:319 IPC_NOTIFY_OPEN
Nov 25 17:05:39.003 netconf/netconf.trace 0/RP0/CPU0 t8790 #61985: TRC: nc_sm_pxs_notify_callback_fn:7243 New NETCONF SSH proxy client connection: 0x7f4868942c90
Nov 25 17:05:39.005 netconf/netconf.trace 0/RP0/CPU0 t8790 #61986: DBG: nc_sm_yf_start_session_generate:877 cidl_yfw_request malloced 0x5607192903c0
Nov 25 17:05:39.005 netconf-yfw/brief.trace 0/RP0/CPU0 t8790 #61987: TRC: yfw_trace:3389 ctx=0x5607173dc830,[Server->yfw] new session
Nov 25 17:05:39.005 netconf-yfw/nacm.trace 0/RP0/CPU0 t8790 #61988: TRC: yfw_nacm_req_author_enforcement:1342 ctx=0x5607173dcc70,[REQ] sess_id: 0, <(null)> is not supported. Skip checking.
Nov 25 17:05:39.006 netconf-yfw/processor.trace 0/RP0/CPU0 t8790 #61989: DBG: yfw_request_process:2702 ctx=0x5607173da7a0,cidl_yfw_request 0x5607192903c0
Nov 25 17:05:39.006 netconf-yfw/brief.trace 0/RP0/CPU0 t8790 #61990: TRC: yfw_request_process:2704 ctx=0x5607173dc830,req cidl_START_SESSION==
Nov 25 17:05:39.006 netconf-yfw/map.trace 0/RP0/CPU0 t8790 #61991: TRC: yfw_map_transform_request:1719 ctx=0x5607173dd0b0,Start
Nov 25 17:05:39.006 netconf-yfw/map.trace 0/RP0/CPU0 t8790 #61992: TRC: yfw_map_transform_request:1763 ctx=0x5607173dd0b0,Mapping not required for this request.
Nov 25 17:05:39.006 netconf-yfw/processor.trace 0/RP0/CPU0 t8790 #61993: DBG: yfw_session_create:174 ctx=0x5607173da7a0,assigned session-id=743885753
Nov 25 17:05:39.006 netconf-yfw/processor.trace 0/RP0/CPU0 t8790 #61994: TRC: yfw_req_session_start:655 ctx=0x5607173da7a0,ses=0x560719330a50,op=11,yfw session not mdt 743885753
Nov 25 17:05:39.006 netconf-yfw/me.trace 0/RP0/CPU0 t8790 #61995: DBG: me_session_create:33072 ctx=0x5607173e9390,session create: usr=0x560719344b00,ses=0x560719330a60
Nov 25 17:05:39.006 netconf-yfw/me.trace 0/RP0/CPU0 t8790 #61996: TRC: me_session_create:33093 ctx=0x5607173e9390,super_user=0
Nov 25 17:05:39.006 netconf-yfw/me.trace 0/RP0/CPU0 t8790 #61997: TRC: me_session_xpath_hash_init:36053 ctx=0x5607173e9390,session_id 743885753, init successfully
Nov 25 17:05:39.011 netconf-yfw/nacm.trace 0/RP0/CPU0 t8790 #61998: TRC: yfw_nacm_session_init:1664 ctx=0x5607173dcc70,sess_id: 743885753, aaa_nacm_is_enabled() rc: 0
Nov 25 17:05:39.011 netconf-yfw/brief.trace 0/RP0/CPU0 t8790 #61999: TRC: me_session_create:33164 ctx=0x5607173dc830,sess (0x560719290be0), session_id: 743885753
Nov 25 17:05:39.011 netconf-yfw/me.trace 0/RP0/CPU0 t8790 #62000: DBG: me_session_create:33190 ctx=0x5607173e9390,Update session info: START
Nov 25 17:05:39.015 netconf-yfw/bk.trace 0/RP0/CPU0 t8790 #62001: TRC: me_bk_sysdb_auth_user_init:561 ctx=0x5607173f7d20,Using authorization methodlist 'default'.
Nov 25 17:05:39.022 netconf-yfw/bk.trace 0/RP0/CPU0 t8790 #62002: TRC: sysdb_backend_session_create:1715 ctx=0x5607173f7d20,auth user init succeess, caller ctx=0x560719292570
Nov 25 17:05:39.022 netconf-yfw/bk.trace 0/RP0/CPU0 t8790 #62003: DBG: sysdb_backend_session_create:1719 ctx=0x5607173f7d20,SysDB backend session successfully created (0x56071929bb00).
Nov 25 17:05:39.022 netconf-yfw/me.trace 0/RP0/CPU0 t8790 #62004: TRC: me_noqt_session_create:35378 ctx=0x5607173e9390,creating non-QT backend session(s)
Nov 25 17:05:39.022 netconf-yfw/calvados.trace 0/RP0/CPU0 t8790 #62005: TRC: calvados_backend_session_create:1635 ctx=0x5607174650a0,tid=0x7f48e9e83840,calvados enqueued session 0x56071929bf40
Nov 25 17:05:39.022 netconf-yfw/calvados.trace 0/RP0/CPU0 t8790 #62006: TRC: calvados_backend_session_create:1692 ctx=0x5607174650a0,tid=0x7f48e9e83840,calling dm_execute
Nov 25 17:05:39.022 netconf-yfw/me.trace 0/RP0/CPU0 t8790 #62007: TRC: me_session_create:33284 ctx=0x5607173e9390,Session ref count incremented (1)
Nov 25 17:05:39.022 netconf-yfw/processor.trace 0/RP0/CPU0 t8790 #62008: DBG: yfw_me_request_result_check:590 ctx=0x5607173da7a0,ses=0x560719330a50,op=11,ME request success.
Nov 25 17:05:39.022 netconf-yfw/bk.trace 0/RP0/CPU0 t8849 #62009: TRC: me_bk_sysdb_auth_request_process:366 ctx=0x5607173f7d20,getting authorization taskmap of user (user ctx=0x56071929bb00).
Nov 25 17:05:39.022 netconf-yfw/bk.trace 0/RP0/CPU0 t8849 #62010: TRC: me_bk_sysdb_get_user_taskmap_netconf_ssh:107 ctx=0x5607173f7d20,agent_name: netconf
Nov 25 17:05:39.022 netconf-yfw/bk.trace 0/RP0/CPU0 t8849 #62011: TRC: me_bk_sysdb_get_user_taskmap_netconf_ssh:142 ctx=0x5607173f7d20,Retrieving authorization taskmap from SSHd child handler, PID: 26449
Nov 25 17:05:39.022 netconf/netconf.trace 0/RP0/CPU0 t8790 #62012: DBG: nc_sm_ses_timeout_elapsed_check_timer_set:6942 Setting abs and idle timeouts
Nov 25 17:05:39.022 netconf-yfw/confd.trace 0/RP0/CPU0 t8864 #62013: TRC: calvados_backend_xdm_wait:1001 ctx=0x56071747d3a0,tid=0x7f48694be700,[Calvados XDM threadpool] dm_wait returned with 0
Nov 25 17:05:39.022 netconf-yfw/confd.trace 0/RP0/CPU0 t8864 #62014: DBG: calvados_backend_xdm_dispatch:1012 ctx=0x56071747d3a0,tid=0x7f48694be700,CA_API_GET_INSTANCE ref_count:-8
---------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------

Verify Alarms

You can view the alarm information using the show alarms command.

Procedure

show alarms [ brief [ card | rack | system ] [ location location ] [ active | history ] | detail [ card | rack | system ] [ location location ] [ active | clients | history | stats ] ]

Displays alarms in brief or detail.

Example:

RP/0/RP0/CPU0:ios# show alarms brief card location 0/RP0/CPU0 active                                                          
Wed Mar  4 06:10:55.959 UTC

------------------------------------------------------------------------------------
Active Alarms
------------------------------------------------------------------------------------
Location        Severity     Group            Set Time                   Description

------------------------------------------------------------------------------------
0/1             Major        FPD_Infra        03/02/2020 07:09:04 UTC    One Or More FPDs Need Upgrade Or Not In Current State

0/2             Major        FPD_Infra        03/03/2020 14:27:33 UTC    One Or More FPDs Need Upgrade Or Not In Current State

0/2             Major        Ethernet         03/03/2020 20:33:33 UTC    HundredGigECtrlr0/2/0/9 - Carrier Loss On The LAN

0/2             Critical     Controller       03/03/2020 20:34:05 UTC    Optics0/2/0/3 - Improper Removal

0/2             NotAlarmed   OTN              03/03/2020 20:34:08 UTC    ODU40/2/0/0/2 - OPUK Client Signal Failure

0/2             NotAlarmed   OTN              03/03/2020 20:34:05 UTC    ODU40/2/0/1/2 - OPUK Client Signal Failure

RP/0/RP0/CPU0:ios# show alarms brief card location 0/RP0/CPU0 active                                                          
Fri May  8 04:46:29.582 UTC

------------------------------------------------------------------------------------
Active Alarms
------------------------------------------------------------------------------------
Location        Severity     Group            Set Time                   Description

------------------------------------------------------------------------------------
0/2             NotReported  OTN              05/07/2020 14:25:05 UTC    ODU20/2/0/0/2/3 - Path Monitoring Alarm Indication Signal

0/2             NotReported  OTN              05/07/2020 14:25:05 UTC    ODU2E0/2/0/0/2/4 - Path Monitoring Alarm Indication Signal

0/1             NotReported  OTN              05/07/2020 14:24:41 UTC    ODU20/1/0/0/2/3 - Path Monitoring Alarm Indication Signal

0/1             NotReported  OTN              05/07/2020 14:25:03 UTC    ODU20/1/0/1/11/3 - Path Monitoring Alarm Indication Signal

0/1             NotReported  OTN              05/07/2020 14:25:03 UTC    ODU2E0/1/0/1/11/4 - Path Monitoring Alarm Indication Signal

0/3             NotReported  OTN              05/07/2020 14:24:41 UTC    ODU20/3/0/0/2/3 - Path Monitoring Alarm Indication Signal

0/3             NotReported  OTN              05/07/2020 14:24:41 UTC    ODU2E0/3/0/0/2/4 - Path Monitoring Alarm Indication Signal

0/1             Major        Ethernet         05/07/2020 14:24:41 UTC    TenGigECtrlr0/1/0/4/1 - Remote Fault

Note

 
In the maintenance mode, all the alarms are suppressed and the show alarms command will not show the alarms details. Use the show controllers controllertype R/S/I/P command to view the client and trunk alarms.

Using Onboard Failure Logging

Onboard Failure Logging (OBFL) collects and stores boot, environmental, and critical hardware data in the nonvolatile flash memory of the CPU controller card. This information is used for troubleshooting, testing, and diagnosis if a failure or other error occurs. This data provides improved accuracy in hardware troubleshooting and root cause isolation analysis. The data collected includes field-replaceable unit (FRU) serial number, OS version, total run time, boot status, temperature and voltage at boot, temperature and voltage history, and other board specific errors.

Procedure

show logging onboard { fmea | inventory | temperature | uptime | voltage}

Displays OBFL data.

Example:

The following example shows the uptime information. 

sysadmin-vm:0_RP0# show logging onboard uptime

OBFL Uptime Information For : 0/RP0
       * indicates incomplete time-sync while record was written 
       ! indicates time reset backwards while system was running 
  -----------------------------------------------------------------------------------
       UPTIME CARD INFORMATION     
  -----------------------------------------------------------------------------------
       Entity Name                :  Value
  -----------------------------------------------------------------------------------
      Previous Chassis SN         : CAT2311B0C5 
      Current Chassis SN          : CAT2311B0CM 
      Previous R/S/I              : 0/0/0 
      Current R/S/I               : 0/0/0 
      Write Interval              : 15 (min) 
      First Power On TS           : 07/30/2019 07:33:56  
      Last Erase TS               : --/--/---- --:--:-- 
      Rack Change Count           : 8
      Slot Change Count           : 8
   
  -----------------------------------------------------------------------------------
       UPTIME INFORMATION     
  -----------------------------------------------------------------------------------
   Start Time (UTC)    | End Time (UTC)      | Card Uptime info    
   mm/dd/yyyy hh:mm:ss | mm/dd/yyyy hh:mm:ss | Weeks.Days.Hrs.Min.Sec 
  -----------------------------------------------------------------------------------
   10/28/2021 12:23:17 | 11/14/2021 21:09:18 | 2.3.8.46.1 
   11/14/2021 21:09:18 | 11/18/2021 16:31:15 | 0.3.19.21.57 
   11/18/2021 16:31:15 | 11/18/2021 21:10:35 | 0.0.4.39.20 
   11/18/2021 21:10:35 | 11/19/2021 12:40:39 | 0.0.15.30.4 
   11/19/2021 12:40:39 | 11/19/2021 14:16:10 | 0.0.1.35.31 
   11/19/2021 14:16:10 | 11/22/2021 11:49:20 | 0.2.21.33.10 
   11/22/2021 11:49:20 | 11/22/2021 22:51:48 | 0.0.11.2.28 
   11/22/2021 22:51:48 | 11/23/2021 17:17:41 | 0.0.18.25.53 
   11/24/2021 21:22:12 | 11/24/2021 23:11:16 | 0.0.1.49.4 
   11/24/2021 23:11:16 | 11/24/2021 23:39:49 | 0.0.0.28.33 
   11/24/2021 23:39:49 | 11/25/2021 15:25:32 | 0.0.15.45.43 
   11/25/2021 15:25:32 | 11/25/2021 16:10:05 | 0.0.0.44.33 
   11/25/2021 16:10:05 | 11/25/2021 16:25:08 | 0.0.0.15.3 
   11/25/2021 16:25:08 | 11/25/2021 16:37:18 | 0.0.0.12.10 
   11/25/2021 16:37:18 | 11/26/2021 15:08:27 | 0.0.22.31.9 


OBFL Uptime Information For : 0/SC0
       * indicates incomplete time-sync while record was written 
       ! indicates time reset backwards while system was running 
  -----------------------------------------------------------------------------------
       UPTIME CARD INFORMATION     
  -----------------------------------------------------------------------------------
       Entity Name                :  Value
  -----------------------------------------------------------------------------------
      Previous Chassis SN         : ------------ 
      Current Chassis SN          : CAT2311B0CM 
      Previous R/S/I              : -/-/- 
      Current R/S/I               : 0/1/0 
      Write Interval              : 15 (min) 
      First Power On TS           : 06/07/2019 08:52:42  
      Last Erase TS               : --/--/---- --:--:-- 
      Rack Change Count           : 0
      Slot Change Count           : 0
   
  -----------------------------------------------------------------------------------
       UPTIME INFORMATION     
  -----------------------------------------------------------------------------------
   Start Time (UTC)    | End Time (UTC)      | Card Uptime info    
   mm/dd/yyyy hh:mm:ss | mm/dd/yyyy hh:mm:ss | Weeks.Days.Hrs.Min.Sec 
  -----------------------------------------------------------------------------------
   10/24/2021 05:48:29 | 10/24/2021 06:27:51 | 0.0.0.39.22 
   10/24/2021 06:27:51 | 10/24/2021 07:05:24 | 0.0.0.37.33 
   10/24/2021 07:05:24 | 10/26/2021 23:43:32 | 0.2.16.38.8 
   10/26/2021 23:43:32 | 10/26/2021 23:55:49 | 0.0.0.12.17 
   10/26/2021 23:55:49 | 10/27/2021 00:09:49 | 0.0.0.14.0 
   10/27/2021 00:09:49 | 10/27/2021 00:16:08 | 0.0.0.6.19 
   10/27/2021 00:16:08 | 10/27/2021 23:37:51 | 0.0.23.21.43 
   10/27/2021 23:37:51 | 10/27/2021 23:50:33 | 0.0.0.12.42
   11/24/2021 21:22:12 | 11/24/2021 23:11:16 | 0.0.1.49.4 
   11/24/2021 23:11:16 | 11/24/2021 23:39:49 | 0.0.0.28.33 
   11/24/2021 23:39:49 | 11/25/2021 15:25:32 | 0.0.15.45.43 
   11/25/2021 15:25:32 | 11/25/2021 16:10:05 | 0.0.0.44.33 
   11/25/2021 16:10:05 | 11/25/2021 16:25:08 | 0.0.0.15.3 
   11/25/2021 16:25:08 | 11/25/2021 16:37:18 | 0.0.0.12.10 
   11/25/2021 16:37:18 | 11/26/2021 15:09:27 | 0.0.22.32.9

Capture Logs

Procedure

Step 1

show logging

Displays the contents of the logging buffers. You can also view details of FPD upgrade failures.

Example:

RP/0/RP0/CPU0:ios# show logging
Fri Nov 26 15:03:48.886 UTC
Syslog logging: enabled (0 messages dropped, 0 flushes, 0 overruns)
    Console logging: Disabled
    Monitor logging: level debugging, 0 messages logged
    Trap logging: level informational, 0 messages logged
    Buffer logging: level debugging, 1025 messages logged

Log Buffer (2097152 bytes):

RP/0/RP0/CPU0:Nov 25 16:40:28.533 UTC: syslogd[155]: %SECURITY-XR_SSL-6-INFO : XR SSL info: Setting fips register
RP/0/RP0/CPU0:Nov 25 16:40:36.323 UTC: cfgmgr-rp[120]: %MGBL-CONFIG-7-INTERNAL : Configuration Manager was unable to find subtree for 'sh_p_service_role_daemon' partition.  : cfgmgr-rp : (PID=2522) :  -Traceback= 7f1be3f92420 7f1be4bdd0c6 7f1be4bdd208 7f1be4bd74a4 7f1be4bd7e45 7f1be4bdb972 7f1be4bd7f0e 55e025a46170 55e025a42429 55e025a3168f
RP/0/RP0/CPU0:Nov 25 16:40:36.457 UTC: aib[291]: Registering with IM
RP/0/RP0/CPU0:Nov 25 16:40:36.661 UTC: cma_partner[350]: Packet received on undiscovered module 160
RP/0/RP0/CPU0:Nov 25 16:40:37.113 UTC: ifmgr[142]: platform_pfi_ifh_get_if_alloc_info: Setting pic
............
............

  1. logging buffered size

    Configures the size of the logging buffer. The range is from 2097152 to 125000000 bytes.

    Example:

    RP/0/RP0/CPU0:ios(config)#logging buffered 3000000

Step 2

show tech-support ncs1004

Creates a .tgz file that contains the dump of the configuration and show command outputs. This file provides system information for the Cisco Technical Support.

Example:

RP/0/RP0/CPU0:ios# show tech-support ncs1004 
Fri Nov 26 15:05:28.996 UTC
++ Show tech start time: 2021-Nov-26.150529.UTC ++
Fri Nov 26 15:05:30 UTC 2021 Waiting for gathering to complete
..................................................................................................
Fri Nov 26 15:10:38 UTC 2021 Compressing show tech output
Show tech output available at 0/RP0/CPU0 : /harddisk:/showtech/showtech-ncs1004-2021-Nov-26.150529.UTC.tgz
++ Show tech end time: 2021-Nov-26.151040.UTC ++

Step 3

show tech-support alarm-mgr

Collects the Cisco support file for the alarm manager component.

Example:

RP/0/RP0/CPU0:ios#show tech-support alarm-mgr 
Fri Nov 26 15:06:06.916 UTC
++ Show tech start time: 2021-Nov-26.150607.UTC ++
Fri Nov 26 15:06:08 UTC 2021 Waiting for gathering to complete
.....
Fri Nov 26 15:06:23 UTC 2021 Compressing show tech output
Show tech output available at 0/RP0/CPU0 : /harddisk:/showtech/showtech-alarm_mgr-2021-Nov-26.150607.UTC.tgz
++ Show tech end time: 2021-Nov-26.150624.UTC ++

Step 4

admin

Enters system admin EXEC mode.

Example:

RP/0/RP0/CPU0:ios#admin

Step 5

show tech ncs1004-admin

Collects show tech logs that can be copied to IOS XR hard disk.

Example:

sysadmin-vm:0_RP0# show tech ncs1004-admin
Thu May  5  15:22:08.520 UTC+00:00
++ Show tech start time: 2022-May-05.152208.UTC ++
Waiting for gathering to complete
......
Compressing show tech output
Show tech output available at /misc/disk1//showtech/showtech-ncs1004-admin-2022-May-05.152208.UTC.tgz
Please collect show tech-support ctrace in addition to any sysadmin show-tech-support  collection 

sysadmin-vm:0_RP0# run
Thu May  5  15:31:54.352 UTC+00:00
[sysadmin-vm:0_RP0:~]$scp /misc/disk1/showtech/showtech-ncs1004-admin-2022-May-05.152208.UTC.tgz 209.165.200.227:/misc/disk1/
showtech-ncs1004-admin-2022-May-05.152208.UTC.tgz                             100%   13MB  12.8MB/s   00:00    
[sysadmin-vm:0_RP0:~]$exit

What to do next

You should gather the above information before calling the Cisco Technical Assistance Center (TAC).

Verify Process Details and Crash Dump

Procedure

Step 1

show processes

Displays information about active processes.

Example:

The following example shows the output of the show processes command in the EXEC mode. 

RP/0/RP0/CPU0:ios# show processes 
Fri Nov 26 14:59:31.671 UTC
JID    TID  Stack  pri  state        NAME             rt_pri
1      1       0K  20   Sleeping     init             0 
66895  1359    0K  20   Sleeping     oom.sh           0 
66911  1375    0K  20   Sleeping     cgroup_oom.sh    0 
66912  1376    0K  20   Sleeping     oom.sh           0 
66932  1396    0K  0    Sleeping     cgroup_oom       0 
67172  1636    0K  20   Sleeping     app_config_back  0 
67176  1640    0K  20   Sleeping     bash             0 
67203  1667    0K  20   Sleeping     inotifywait      0 
67205  1669    0K  20   Sleeping     bash             0 
67242  1706    0K  20   Sleeping     dbus-daemon      0 
67242  1707    0K  20   Sleeping     dbus-daemon      0 
67260  1724    0K  20   Sleeping     sshd             0 
67271  1735    0K  20   Sleeping     rpcbind          0 
........................................
........................................

The following example shows the output of the show processes command in the system admin EXEC mode. 

sysadmin-vm:0_RP0# show processes all location 0/rp0
Fri Nov  26 15:01:44.450 UTC+00:00
----------------------------------------------------------------------
node:      0/RP0
----------------------------------------------------------------------
LAST STARTED            STATE    RE-     MANDA-  MAINT- NAME(IID) ARGS
                                 START   TORY    MODE
----------------------------------------------------------------------
11/25/2021 16:37:56.000 Run      1                      aaad(0) 
11/25/2021 16:39:12.000 Run      1                      ael_mgbl(0) 
11/25/2021 16:37:56.000 Run      1                      bh_cardmgr(0) -M 3
11/25/2021 16:37:56.000 Run      1                      bh_esd(0) 
11/25/2021 16:37:56.000 Run      1       M              calv_alarm_mgr(0) 
11/25/2021 16:37:56.000 Run      1       M              cm(0) 
11/25/2021 16:37:56.000 Run      1       M              confd_helper(0) -t token -d -w 400 -b 30 -p 600 -r 10 -f 10
11/25/2021 16:39:12.000 Run      1                      debug_agent(0) 
11/25/2021 16:37:56.000 Run      1       M              debug_client(0) 
11/25/2021 16:37:56.000 Run      1                      dr_calv(0) 0
11/25/2021 16:37:56.000 Run      1       M              ds(0) -r
11/25/2021 16:37:56.000 Run      1                      dumper(0) 
11/25/2021 16:37:56.000 Run      1       M              envmon(0) 
11/25/2021 16:39:12.000 Run      1                      envmon_ui(0) 
11/25/2021 16:39:12.000 Run      1                      fit_mgbl(0) 
11/25/2021 16:39:12.000 Run      1                      fpdserv(0) 
11/25/2021 16:39:12.000 Run      1                      gaspp_mgbl(0) 
11/25/2021 16:37:56.000 Run      1       M              inst_agent(0) 
11/25/2021 16:39:13.000 Run      1                      inst_mgr(0) 
11/25/2021 16:37:56.000 Run      1       M              issu_agt(0) 
11/25/2021 16:39:12.000 Run      1                      issu_dir(0) 
11/25/2021 16:39:11.000 Run      2                      led_mgr(0) 
11/25/2021 16:37:56.000 Run      1                      mediasvr(0) 
11/25/2021 16:37:56.000 Run      1                      obfl_mgr(0) 
11/25/2021 16:37:56.000 Run      1                      obfl_show(0) 
11/25/2021 16:37:56.000 Run      1                      pam_manager(0) 
11/25/2021 16:37:56.000 Run      1       M              pm(0) 
11/25/2021 16:37:56.000 Run      1       M              rvm_mgr(0) 
11/25/2021 16:37:56.000 Run      1       M              sdr_mgr(0) 
11/25/2021 16:37:56.000 Run      1                      set_hostname(0) 
11/25/2021 16:37:56.000 Run      1       M              shelf_mgr(0) 
11/25/2021 16:37:56.000 Run      1       M              ship_server(0) 
11/25/2021 16:37:56.000 Run      1                      ssh_key_client(0) 
11/25/2021 16:39:12.000 Run      1                      ssh_key_server(0) 
11/25/2021 16:37:56.000 Run      1       M              syslogd_helper(0) 
11/25/2021 16:37:56.000 Run      1                      syslogd_relay(0) 
11/25/2021 16:37:56.000 Run      1                      tacacsd(0) 
11/25/2021 16:37:56.000 Run      1                      timezone_config(0) 
11/25/2021 16:37:56.000 Run      1       M              vm_manager(0) -W -c -n -v -e  /opt/cisco/qemu-system-x86_64.wrapper
11/25/2021 16:37:56.000 Run      1       M              wd_diskmon(0) 
11/25/2021 16:37:56.000 Run      1       M              wd_memmon(0) 
11/25/2021 16:37:56.000 Run      1       M              wd_sysmon(0) 
11/25/2021 16:37:56.000 Run      1       M              wdmon(0) -q
----------------------------------------------------------------------
Total pcbs: 43
----------------------------------------------------------------------

Step 2

show processes process-name

Displays detailed information about a process.

Example:

RP/0/RP0/CPU0:ios#show processes dsr
Fri Nov 26 15:02:55.728 UTC
                  Job Id: 53
                     PID: 2246
            Process name: dsr
         Executable path: /opt/cisco/XR/packages/ncs1004-iosxr-os-1.0.0.0-r751/all/bin/dsr
              Instance #: 1
              Version ID: 00.00.0000
                 Respawn: ON
           Respawn count: 1
            Last started: Thu Nov 25 16:40:26 2021
           Process state: Run
           Package state: Normal
                    core: DUMPFALLBACK 
               Max. core: 0
                   Level: 11
               Mandatory: ON
           MaintModeProc: ON
               Placement: None
            startup_path: /opt/cisco/XR/packages/ncs1004-iosxr-os-1.0.0.0-r751/all/init.d/dsr.init
                   Ready: 0.375s
        Process cpu time: 2.050 user, 0.930 kernel, 2.980 total
JID    TID  Stack  pri  state        NAME             rt_pri
53     2246    0K  20   Sleeping     dsr              0 
53     2330    0K  20   Sleeping     lwm_service_thr  0 
53     2331    0K  20   Sleeping     qsm_service_thr  0 
53     2336    0K  20   Sleeping     dsr              0 
53     2377    0K  20   Sleeping     dsr              0 
53     2378    0K  20   Sleeping     dsr              0 
53     2379    0K  20   Sleeping     dsr              0 
53     2380    0K  20   Sleeping     dsr              0 
-------------------------------------------------------------------------------

Step 3

show context

Displays information about process crashes.

Example:

RP/0/RP0/CPU0:ios# show context
Fri Nov 26 15:03:26.008 UTC

node: node0_RP0_CPU0
----------------------------------------------------------------
No context

The command output is empty during system upgrade.