Scaling Down the Management Cluster

In order to scale down the management cluster, a manual PV migration steps is required.

If the nodes to be scaled down host SeaweedFS or Consul deployments, the PVs attached to them must be migrated to other nodes in the cluster before scaling down.

Prerequisites

• Identify which pods are on nodes being removed
• Identify target nodes for migration
• Ensure SSH/rsync access between nodes

Migration Process (Per Pod)

1. Get current state

# Find pod's current node 

$ kubectl get pod <pod-name> -o wide 

​

# Get PV name from PVC 

$ kubectl get pvc <pvc-name> -o jsonpath='{.spec.volumeName}' 

​

# Get current node IP from PV 

$ kubectl get pv <pv-name> -o jsonpath='{.spec.nodeAffinity.required.nodeSelectorTerms[0].matchExpressions[0].values[0]}'

2. Prepare new node directory :

ssh ubuntu@<new-node-ip>

​

sudo mkdir -p /opt/pf9/volumes/csi/<pvc-id>

sudo chown root:ubuntu /opt/pf9/volumes/csi/<pvc-id>

sudo chmod 2775 /opt/pf9/volumes/csi/<pvc-id>

exit

3. Rsync data from old to new node :

# From old node

ssh ubuntu@<old-node-ip>

​

sudo rsync -avzAX --progress \

  --rsync-path="sudo rsync" \

  /opt/pf9/volumes/csi/<pvc-id>/ \

  ubuntu@<new-node-ip>:/opt/pf9/volumes/csi/<pvc-id>/

4. Save PV configuration :

kubectl get pv <pv-name> -o yaml > pv-backup.yaml

5. Edit PV YAML

vim pv-backup.yaml

Changes needed:

• Update nodeAffinity IP: old-node-ip → new-node-ip
• Remove from metadata: creationTimestamp, resourceVersion, uid
• Remove entire status: section

6. Delete PV and PVC :

kubectl apply -f pv-backup.yaml

7. Recreate PV only :

kubectl apply -f pv-backup.yaml

8. Delete pod - StatefulSet will recreate PVC and bind :

kubectl delete pod <pod-name>

9. Verify :

# Verify PV-PVC bind

kubectl get pv,pvc | grep <pvc-name>

​

# Verify pod is on new node

kubectl get pod <pod-name> -o wide