Node Connectivity Monitoring

Node Connectivity Monitoring

Rules for the Hostagent

The Prometheus agent is configured to report numerous node-based metrics. Below is the YAML-based rule set that Catapult uses, including alert names, expression, timeframe, annotations which contain the summary and description of the notification as well as the SaaS management plane and hostname.

bash-5.0# cat resmgr.yml
groups:
- name: Host Availability Alerts
rules:
# du_sidekick_last_checkin_seconds will be the time in seconds since a host checked in via sidekick
# or -1 if the host is not reported by sidekick at all. The latter case may occur if
# sidekick has restarted and a host hasn't connected since.
- expr: >-2
(
sum by (host_id,host_name) (
time() - sidekick_host_last_heartbeat_time{job="sidekickserver"} / 1000
)
)
OR
ignoring(host_name) (
sum by (host_id,host_name) (
(resmgr_host_up{job="resmgr"} == 0) - 1
)
)
record: du_sidekick_last_checkin_seconds


- name: Hosts disconnected
rules:
# resmgr says it's been down for at least 10m, sidekick says still reporting


  Last updated by Anagha Pamidi