How to Delete Stale Storage Host Entries in PCD

Problem

Unable to launch new VMs using self_hosted_product_name Management plane UI, Cinder-scheduler pods logs show volume service down errors.

Cinder-scheduler.log
    
xxxxxxxxxx
 
WARNING cinder.scheduler.host_manager [None req-ID] volume service is down. (host: [HOST-UUID@STORAGE-NAME])
Copy

The volume service list has multiple volumes in down state:

Command
    
 
$ openstack volume service list+------------------+-----------------------+------+---------+-------+------------+| Binary           | Host                  | Zone | Status | State | Updated At  |+------------------+-----------------------+------+---------+-------+------------+| cinder-scheduler |cinder-scheduler-[UUID]| nova | enabled | down | [Timestamp] |+------------------+-----------------------+------+---------+-------+------------+
Copy

Environment

Self-Hosted Private Cloud Director Virtualization - v2025.4 and Higher.

Procedure

Steps to delete the stale volumes from the self_hosted_product_name:

List the storage volume services, to identify the stale storage host. Execute below command from the Host having the PersistentStorage role applied:

Command
    
xxxxxxxxxx
 
$ LD_LIBRARY_PATH="/opt/pf9/pf9-cindervolume-base/pf9-lib:/opt/pf9/python/pf9-lib:${LD_LIBRARY_PATH}" PYTHONPATH="/opt/pf9/python/lib/python3.6:/opt/pf9/pf9-cindervolume-base/lib/python3.6/site-packages:${PYTHONPATH}" /opt/pf9/pf9-cindervolume-base/bin/cinder-manage --config-dir /opt/pf9/etc/pf9-cindervolume-base/conf.d service list
Copy

Validate the above command output with below command output, capture the information about stale storage host UUID.

Command
    
xxxxxxxxxx
 
$ openstack volume service list
Copy

After confirmation storage volume service can be deleted. Execute the below command from the PersistentStorage role host.

Command
    
xxxxxxxxxx
 
$ LD_LIBRARY_PATH="/opt/pf9/pf9-cindervolume-base/pf9-lib:/opt/pf9/python/pf9-lib:${LD_LIBRARY_PATH}" PYTHONPATH="/opt/pf9/python/lib/python3.6:/opt/pf9/pf9-cindervolume-base/lib/python3.6/site-packages:${PYTHONPATH}" /opt/pf9/pf9-cindervolume-base/bin/cinder-manage --config-dir /opt/pf9/etc/pf9-cindervolume-base/conf.d service remove cinder-scheduler <HOST_UUID>
Copy

Validation

The stale volumes should be removed in the volumes list, It can be verified using:

Export the .rc file and execute below command:

Command
    
xxxxxxxxxx
 
$ openstack volume service list
Copy

From the PersistentStorage role host., execute the below command to ensure the stale entries are removed.

Command
    
xxxxxxxxxx
 
$ LD_LIBRARY_PATH="/opt/pf9/pf9-cindervolume-base/pf9-lib:/opt/pf9/python/pf9-lib:${LD_LIBRARY_PATH}" PYTHONPATH="/opt/pf9/python/lib/python3.6:/opt/pf9/pf9-cindervolume-base/lib/python3.6/site-packages:${PYTHONPATH}" /opt/pf9/pf9-cindervolume-base/bin/cinder-manage --config-dir /opt/pf9/etc/pf9-cindervolume-base/conf.d service list
Copy

Last updated on

Was this page helpful?