Rollback Guide: From SMCP 5.11 to SMCP 5.10

Prerequisites

  • Backup of the SMCP 5.11 management plane

  • Access to SMCP 5.10 artifacts

Rollback Procedure

1. Backup SMCP 5.11 Management Plane

Follow the Backup Guide to create a backup of the current SMCP 5.11 management plane.

2. Destroy SMCP 5.11 Cluster

Remove the existing 1.30-based nodelet cluster:

airctl stop airctl unconfigure-du --force airctl advanced-ddu delete-mgmt

3. Clean Up Configuration

On all master nodes, clean up the certificate directory:

rm -rf /etc/nodelet/airctl-mgmt/certs/*

4. Download and Install SMCP 5.10 Artifacts

Download the SMCP 5.10 artifacts and install them:

wget https://raw.githubusercontent.com/platform9/support-locker/master/edge-stack/download.sh #Download script chmod u+x download.sh # change permission ./download.sh <PF9-SHARED-SECRET> v-5.10.0-3248001 # Download artifacts bash install.sh v-5.10.0-3248001 # Install artifacts

5. Configure SMCP 5.10 Management Cluster

Update configuration files before deployment:

  1. Edit airctl-config.yaml:

    • Update airctlBuildNumber to match the SMCP 5.10 version

  2. Edit nodelet-bootstrap.yaml:

    • Update systemImages to use SMCP 5.10 image files

    • Set userImages to point to SMCP 5.11 image files

    • Add isAirgapped: true for offline installations

Example nodelet-bootstrap.yaml snippet:

isAirgapped: true systemImages: - /opt/pf9/airctl/imgs/kubedu-imgs-v-5.10.0-3248001.tar.gz - /opt/pf9/airctl/imgs/nodelet-imgs-v-5.10.0-3248001.tar.gz userImages: - /opt/pf9/airctl/imgs/kubedu-imgs-v-5.11.1-3501826.tar.gz

6. Deploy SMCP 5.10 Management Cluster

Deploy the new 1.27-based nodelet management cluster:

airctl advanced-ddu create-mgmt

7. Restore SMCP 5.10 Management Plane

Restore the SMCP 5.10 management plane using the backup created in step 1:

Consul Issue:

After restoring, you might see errors in the Consul pod saying ACL not found. These errors are harmless and can be ignored.

Therefore, SMCP currently skips the Consul upgrade in the upgrade step and installs the upgraded Consul during the restore process itself.

8. Verify DU Status

Check that the DU is in a ready state:

airctl status

Wait until all components show as ready before proceeding.

9. Upgrade DU Components (Optional)

If needed, follow the SMCP 5.10 upgrade guide to update the DU components.

10. Verify Rollback in GUI

Log in to the management UI and confirm the version is displayed as 5.10.

11. Push Required Kubernetes Images

Push images for the supported Kubernetes versions (1.27, 1.28, 1.29) to your private registry (if applicable):

sudo airctl advanced-du push-images --img-tar [tar file name]

To push all new images at once:

ls -1 /opt/pf9/airctl/imgs/k8s* | xargs -I{} sh -c "sudo /opt/pf9/airctl/airctl --config /opt/pf9/airctl/conf/airctl-config.yaml advanced-du push-images --img-tar {}"

12. Upgrade Host Components

Upgrade the host-side components to SMCP 5.10:

airctl configure-hosts --reset-pf9-managed-docker-conf --skip-docker-install --skip-docker-img-import --verbose airctl hosts-status airctl upgrade-hosts --verbose

13. Verify Status and Version

After the rollback, confirm the management plane is in a ready state:

airctl status # Check the status of management plane airctl version # The output should be: v-5.10.0-3248001 kubectl get nodes -o wide # Ensure all the nodes are in Ready state