Ad Widget

Collapse

Clear a problem when the metric no longer exists

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • cnhome1
    Junior Member
    • Feb 2023
    • 4

    #1

    Clear a problem when the metric no longer exists

    Hi,

    I am trying to generate triggers from metrics collected from a prometheus node export. The Triggers works fine its just i cannot get them to auto clear. The problem is from the source of the metrics when the issue clear the data is no longer presented to zabbix via the node exporter. An example is this :-

    [root@v-deployer ~]# curl https://10.169.1.169:8443/metrics/ -k|grep 'chris'
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed
    100 20967 0 20967 0 0 131k 0 --:--:-- --:--:-- --:--:-- 132k
    vsm_pending_message{text="Device not responding",subject="",subjectIdentification="",re sourceType="DEVICE",resourceId="8A298129867A065501 867E6DD42F3907",resourceName="chris",source="DEVIC E_MANAGEMENT",severity="MAJOR",acknowledgedUser="" ,tmnType="COMMUNICATION",user="VSM System",userResetNeeded="false",details="Reason: REST API communication failure: HTTP communication failure. org.apache.http.conn.ConnectTimeoutException: Connect to 10.101.1.2:8443 [/10.101.1.2] failed: connect timed out",correlatedLineupIds="",correlatedLineupNames = "",} 4.0​

    in this example i trigger on vaule>=4 from an item with 'vsm_pending_message{resourceName="chris"}' as a filter but when this is resolved on the host that line is gone

    i did try to clear with nodata but i dont think the trigger gets tested unless there is some metrics that matches meaning, i end up with a problem that never clears.

    Any suggestions gratefully received.
Working...