Upgrading Single Node Orchestrator

This chapter contains the following sections:

Upgrading Single Node ESX VM

This section describes how to upgrade single node Cisco ACI Multi-Site Orchestrator deployed in an ESX VM. If you are running a 3-node Orchestrator cluster, follow the upgrade procedure described in Upgrading Orchestrator Deployments in VMware ESX instead.

Procedure

Step 1

If you are upgrading from a release prior to Release 2.1(1), configure at least 10GHz CPU cycle reservation for each Orchestrator VM.

This release of Multi-Site Orchestrator requires at least 10GHz CPU cycle reservation for each VM. New deployments of Release 2.1(1) or later apply CPU cycle reservation automatically, however if you're upgrading from an earlier release, you must manually update each Orchestrator VM's settings.

  1. Log in to the vSphere client.

  2. Navigate to the ESX host where your Orchestrator VMs are located.

  3. Shut down one of the VMs.

  4. Right click the VM and choose Edit Settings

  5. In the Virtual Hardware tab, expand the CPU category.

  6. In the Reservation field, enter 10 GHz.

  7. Click OK to save the changes.

  8. Power on the VM and wait for the Orchestrator cluster to stabilize with all nodes healthy.

  9. Repeat the steps for the other Orchestrator VMs.
Step 2

Download the Cisco ACI Multi-Site Orchestrator upgrade image.

  1. Browse to the Software Download link:

    https://software.cisco.com/download/home/285968390/type

  2. Click ACI Multi-Site Software.

  3. Choose the Cisco ACI Multi-Site Orchestrator release version.

  4. Download the ACI Multi-Site Upgrade Image file (msc-<version>.tar.gz) for the release.

Step 3

Copy the image to your Orchestrator node.

The following command copies the image into the /tmp directory on the node. 

# scp msc-<version>.tar.gz <mso-ip-address>:/tmp/
Step 4

Log in to your Orchestrator node and extract the tools directory from the Orchestrator image you copied.

Note 

In the following command, the image filename contains an msc- prefix with a dash, while the second argument has a msc_ prefix with an underscore, because you are extracting a specific directory within the image file. 

# cd /tmp
# tar xfz msc-<version>.tar.gz msc_<version>/tools
Step 5

Copy the upgrade script from the extracted directory into the Orchestrator scripts directory. 

# cp msc_<version>/tools/msc_setup.py /opt/cisco/msc/scripts
Step 6

Change into the Orchestrator scripts directory.

# cd /opt/cisco/msc/scripts
Step 7

Run the upgrade script.

In the following command, provide the image filename you uploaded to the Orchestrator node. 

# python3 msc_setup.py -u -f /tmp/msc-<version>.tar.gz -st
Note 

If the kernel needs to be upgraded, the scripts will perform the upgrade and reboot.

Step 8

If the system restarts due to a kernel upgrade, re-run the upgrade script.

When you re-run the script after a reboot, add the --pass2 argument: 

# python3 msc_setup.py -u -f /tmp/msc-<version>.tar.gz -st --pass2
Step 9

Verify upgrade was successful.

The installation can take up to 15 minutes. After it is completed, use the docker service ls command to verify that all docker REPLICAS are up: 

# docker service ls
ID              NAME                      MODE            REPLICAS      
yljvxfmjt3kb    msc_auditservice          replicated      1/1
kyvaqpehau15    msc_authyldapservice      replicated      1/1
y2fh16599hi5    msc_authytacacsservice    replicated      1/1
6pajp3kjktli    msc_backupservice         replicated      1/1
9a6tnu7wwb6j    msc_cloudsecservice       replicated      1/1
dmwkjl7het8i    msc_consistencyservice    replicated      1/1
l01mbez8j4sy    msc_endpointservice       replicated      1/1
qerrp08i6hsq    msc_executionengine       replicated      1/1
vsitso4b9xu6    msc_jobschedulerservice   replicated      1/1
l1sldx735iut    msc_kong                  replicated      1/1
zk6s5f9h6l93    msc_kongdb                replicated      1/1
t4wbsstsps6r    msc_mongodb               replicated      1/1
qi5aj3zygc2w    msc_pctagvnidservice      replicated      1/1
olxke4nk7me9    msc_platformservice       replicated      1/1
tlsjms2kwl64    msc_policyservice         replicated      1/1
9owa824s83a7    msc_schemaservice         replicated      1/1
zb3dy4d7j775    msc_siteservice           replicated      1/1
miubr6yw135n    msc_syncengine            replicated      1/1
wxle65d6ag1g    msc_ui                    replicated      1/1
jr01hfpmrcbw    msc_userservice           replicated      1/1
Step 10

(Optional) Free up disk space by deleting old images.

Use the following command to display all docker images in the system:

# docker images

You can then delete the old Orchestrator images using the following command: 

# docker system prune -a
Step 11

If you upgraded from a release prior to Release 2.1(1), log in to your Orchestrator GUI and reset the password.

Due to password requirements change in Release 2.1(1), when you first log in to the Orchestrator GUI after upgrading to Release 2.1(1) or later, you will be prompted to update your password. The new password requirements are:

  • At least 12 characters

  • At least 1 letter

  • At least 1 number

  • At least 1 special character (* and space are not allowed

Upgrading Single Node Service Engine VM

This section describes how to upgrade single node Cisco ACI Multi-Site Orchestrator deployed in Cisco Application Service Engine. If you are running a 3-node Orchestrator cluster, follow the upgrade procedure described in Upgrading or Downgrading Orchestrator Deployments in Application Service Engine instead.

Procedure

Step 1

Download the Cisco ACI Multi-Site Orchestrator Image.

  1. Browse to the ACI Multi-Site Orchestrator download page on Cisco DC App Center.

  2. Click Download to download the image.

Step 2

Make the image accessible by the Orchestrator.

Note 

This release supports GUI image upload from an HTTP or HTTPS server only, so you must either make the image available on a web server accessible by the Orchestrator or manually upload the image to the Application Server Engine where the Orchestrator is hosted.

If you have a web server running in your environment, simply host the .aci image you downloaded on that server and proceed to the next step.

Otherwise, to manually upload the image:

  1. Copy the application to the Application Service Engine.

    If your Cisco Application Services Engine is deployed in VMware ESX (.ova), Linux KVM (.qcow), or as a physical appliance (.iso), or you have enabled password-based logins for your AWS (.ami) deployment, use the following command to copy the Orchestrator image into the tmp directory on the Services Engine: 

    # scp <app-local-path> rescue-user@<service-engine-ip>:/tmp/

    However, if your Service Engine is deployed in AWS and you have not enabled password-based login, you must use the certificate (.pem) file that you created during the Application Services Engine deployment: 

    # scp <app-local-path>.aci -i <pem-file-name>.pem rescue-user@<service-engine-ip>:/tmp/

    For example, assuming you're running the scp command from the same directory where you saved the Orchestrator image:

    • For password-based authentication:

      # scp ./cisco-mso-2.2.3c.aci rescue-user@10.30.11.147:/tmp/

    • For PEM-based authentication:

      # scp ./cisco-mso-2.2.3c.aci -i <pem-file-name>.pem rescue-user@10.30.11.147:/tmp/

  2. Log in to your Service Engine as rescue-user.

    If your Cisco Application Service Engine is deployed in VMware ESX (.ova), Linux KVM (.qcow), or as a physical appliance (.iso), simply SSH in using the following command: 

    # ssh rescue-user@<service-engine-ip>

    However, if your Application Service Engine is deployed in AWS (.ami), you must login using the certificate (.pem file) that you created during the Application Service Engine deployment: 

    # ssh -i <pem-file-name>.pem rescue-user@<service-engine-ip>

  3. Add the new image.

    In the following command, replace <application-path> with the full path to the application image you copied in the previous step. 

    # acidiag app install <application-path>

    For example:

    # acidiag app install /tmp/cisco-mso-2.2.3c.aci

  4. Verify that the application was loaded.

    Use the following command to check the operState of the application.

    While the application is loading and installing it will go through a number of operational states, which will be reflected in the operState field, for example 'operState': 'Initialize'. This process can take up to 20 minutes and you must ensure that the state changes to Disabled before proceeding to the next step. 

    # acidiag app show
[   {   'adminState': 'Disabled',
        'apiEntrypoint': '/query',
        'appID': 'MSO',
        'creationTimestamp': '2020-02-10T20:30:36.195960295Z',
        'description': 'Multi-Site Orchestrator application',
        'displayName': 'ACI Multi-Site Orchestrator',
        'id': 'cisco-mso:2.2.3',
        'name': 'cisco-mso',
        'operStage': 'PostInstall',
        'operState': 'Disabled',
        'schemaversion': '',
        'uiEntrypoint': '/ui/app-start.html',
        'vendorID': 'Cisco',
        'version': '2.2.3'}]
Step 3

Log in to your Orchestrator.
Step 4

From the left navigation pane, select Admin > Firmware Management.

Step 5

Add the new image to the Application Service Engine cluster.

Note 

If you manually uploaded the image to the Service Node cluster, the image will be already available and you can skip this step.

  1. In the main window, click Add Image.

    An Add Image window opens.

  2. In the File Path field, provide the URL to the new Orchestrator image.

    For example, https://www.my-web-server.com/mso/cisco-mso-2.2.3c.aci.

  3. Click OK to add the image.

    The image will be uploaded to the Orchestrator's Service Engine nodes, unpacked, processed, and made available for the upgrade. The whole process may take several minutes and you will be able to see the status of the image.

    Wait for the status to change to Available before proceeding to the next step.

Step 6

Activate the new image.

Ensure that the new image's status is Available.

  1. In the main window, click the actions menu next to the image you added.

  2. Then click Activate.

  3. In the Activation Confirmation window, click Continue.

    Wait for the new image to be activated. The page automatically reloads when the process is completed.