Escalations, Actions and Events - Zabbix Pre 1.8.3 Rev. 12016
Zabbix Pre 1.8.3 Rev. 12016 issues with escalations, alerts and events
Thought that I would post this in case it helps anyone else out. Noticed that the new pollers cache history, trends, configuration, etc. and now they cache eventid's. Caching of eventid's is good, because it puts less load on your "ids" zabbix database table so that the pollers do not have to keep asking for nextid for eventid. But this will cause eventids created by the browser and the poller to be out of sync sometimes. Usually to get the next event to calculate escalations, age of events, etc. the next event id would do the trick. But since the frontend gui can create event ids, these ids can be requested while the pollers are in the middle of the pollers eventid cache pool (256 ids at a time) and the eventids are not in order sometimes. This caused a lot of issues with our escalations and show events with incorrect ages.
To fix this issue we just sorted on events.clock and not eventids. Also, we added an addition key index to the events table.
Database Fix:
CREATE INDEX events.custom on events (`object`,`objectid`,`clock`);
Poller Fixes:
Frontend Fixes:
Thought that I would post this in case it helps anyone else out. Noticed that the new pollers cache history, trends, configuration, etc. and now they cache eventid's. Caching of eventid's is good, because it puts less load on your "ids" zabbix database table so that the pollers do not have to keep asking for nextid for eventid. But this will cause eventids created by the browser and the poller to be out of sync sometimes. Usually to get the next event to calculate escalations, age of events, etc. the next event id would do the trick. But since the frontend gui can create event ids, these ids can be requested while the pollers are in the middle of the pollers eventid cache pool (256 ids at a time) and the eventids are not in order sometimes. This caused a lot of issues with our escalations and show events with incorrect ages.
To fix this issue we just sorted on events.clock and not eventids. Also, we added an addition key index to the events table.
Database Fix:
CREATE INDEX events.custom on events (`object`,`objectid`,`clock`);
Poller Fixes:
Code:
diff -ruN /root/zabbix-original/src/zabbix_server/events.c /root/zabbix/src/zabbix_server/events.c
--- /root/zabbix-original/src/zabbix_server/events.c 2010-05-15 01:29:59.000000000 -0400
+++ /root/zabbix/src/zabbix_server/events.c 2010-05-17 00:50:57.000000000 -0400
@@ -86,7 +86,7 @@
" where source=%d"
" and object=%d"
" and objectid=" ZBX_FS_UI64
- " order by object desc,objectid desc,eventid desc",
+ " order by object desc,objectid desc,clock desc",
EVENT_SOURCE_TRIGGERS,
EVENT_OBJECT_TRIGGER,
triggerid);
@@ -181,6 +181,9 @@
* if event->trigger_type is not TRIGGER_TYPE_MULTIPLE_TRUE:
* (4) TRUE /UNKNOWN/ TRUE
*/
+
+ event->skip_actions = 0;
+
if (event->value == TRIGGER_VALUE_UNKNOWN) /* (1) */
{
event->skip_actions = 1;
Code:
diff -ruN /root/zabbix-original/frontends/php/api/classes/class.cevent.php /root/zabbix/frontends/php/api/classes/class.cevent.php
--- /root/zabbix-original/frontends/php/api/classes/class.cevent.php 2010-05-15 01:30:00.000000000 -0400
+++ /root/zabbix/frontends/php/api/classes/class.cevent.php 2010-05-17 02:59:30.000000000 -0400
@@ -632,22 +632,22 @@
$sql = ' SELECT eventid, object, objectid'.
' FROM events'.
- ' WHERE eventid < '.$event['eventid'].
+ ' WHERE clock < '.$event['clock'].
' AND objectid = '.$event['objectid'].
' AND value = '.$val.
' AND object = '.EVENT_OBJECT_TRIGGER.
- ' ORDER BY object desc, objectid desc, eventid DESC';
+ ' ORDER BY object desc, objectid desc, clock DESC';
$first = DBfetch(DBselect($sql, 1));
$first_sql = $first ? ' AND e.eventid > '.$first['eventid'] : '';
$sql = ' SELECT eventid, object, objectid'.
' FROM events'.
- ' WHERE eventid > '.$event['eventid'].
+ ' WHERE clock > '.$event['clock'].
' AND objectid = '.$event['objectid'].
' AND value = '.$val.
' AND object = '.EVENT_OBJECT_TRIGGER.
- ' ORDER BY object ASC, objectid ASC, eventid ASC';
+ ' ORDER BY object ASC, objectid ASC, clock ASC';
$last = DBfetch(DBselect($sql, 1));
$last_sql = $last ? ' AND e.eventid < '.$last['eventid'] : '';
diff -ruN /root/zabbix-original/frontends/php/events.php /root/zabbix/frontends/php/events.php
--- /root/zabbix-original/frontends/php/events.php 2010-05-15 01:30:00.000000000 -0400
+++ /root/zabbix/frontends/php/events.php 2010-05-17 02:59:29.000000000 -0400
@@ -236,7 +236,7 @@
// CHECK IF EVENTS EXISTS {{{
$options = array(
'output' => API_OUTPUT_EXTEND,
- 'sortfield' => 'eventid',
+ 'sortfield' => 'clock',
'sortorder' => ZBX_SORT_UP,
'nopermissions' => 1,
'limit' => 1
@@ -273,7 +273,7 @@
'time_from' => $from,
'time_till' => $till,
'output' => API_OUTPUT_SHORTEN,
- 'sortfield' => 'eventid',
+ 'sortfield' => 'clock',
'sortorder' => ZBX_SORT_DOWN,
'limit' => ($config['search_limit']+1)
);
@@ -290,7 +290,7 @@
'select_items' => API_OUTPUT_EXTEND,
);
$dsc_events = CEvent::get($options);
- order_result($dsc_events, 'eventid', ZBX_SORT_DOWN);
+ order_result($dsc_events, 'clock', ZBX_SORT_DOWN);
$objectids = array();
@@ -402,7 +402,7 @@
'time_from' => $from,
'time_till' => $till,
'output' => API_OUTPUT_SHORTEN,
- 'sortfield' => 'eventid',
+ 'sortfield' => 'clock',
'sortorder' => ZBX_SORT_DOWN,
'nopermissions' => 1,
'limit' => ($config['search_limit']+1)
@@ -422,7 +422,7 @@
'nopermissions' => 1
);
$events = CEvent::get($options);
- order_result($events, 'eventid', ZBX_SORT_DOWN);
+ order_result($events, 'clock', ZBX_SORT_DOWN);
foreach($events as $enum => $event){
$trigger = reset($event['triggers']);
diff -ruN /root/zabbix-original/frontends/php/include/acknow.inc.php /root/zabbix/frontends/php/include/acknow.inc.php
--- /root/zabbix-original/frontends/php/include/acknow.inc.php 2010-05-15 01:30:00.000000000 -0400
+++ /root/zabbix/frontends/php/include/acknow.inc.php 2010-05-17 02:59:30.000000000 -0400
@@ -25,7 +25,7 @@
' FROM events '.
' WHERE objectid='.$triggerid.
' and object='.EVENT_OBJECT_TRIGGER.
- ' ORDER BY objectid desc, object desc, eventid desc', 1));
+ ' ORDER BY objectid desc, object desc, clock desc', 1));
if(!$event_data)
return FALSE;
return $event_data;
diff -ruN /root/zabbix-original/frontends/php/include/blocks.inc.php /root/zabbix/frontends/php/include/blocks.inc.php
--- /root/zabbix-original/frontends/php/include/blocks.inc.php 2010-05-15 01:30:00.000000000 -0400
+++ /root/zabbix/frontends/php/include/blocks.inc.php 2010-05-17 02:59:30.000000000 -0400
@@ -330,7 +330,7 @@
'output' => API_OUTPUT_EXTEND,
'nopermissions' => 1,
'limit' => 1,
- 'sortfield' => 'eventid',
+ 'sortfield' => 'clock',
'sortorder' => ZBX_SORT_DOWN
);
@@ -785,7 +785,7 @@
' WHERE e.object='.EVENT_OBJECT_TRIGGER.
' AND e.objectid='.$trigger['triggerid'].
' AND e.value='.TRIGGER_VALUE_TRUE.
- ' ORDER by e.object DESC, e.objectid DESC, e.eventid DESC';
+ ' ORDER by e.object DESC, e.objectid DESC, e.clock DESC';
$res_events = DBSelect($event_sql,1);
while($row_event=DBfetch($res_events)){
$ack = NULL;
diff -ruN /root/zabbix-original/frontends/php/include/events.inc.php /root/zabbix/frontends/php/include/events.inc.php
--- /root/zabbix-original/frontends/php/include/events.inc.php 2010-05-15 01:30:00.000000000 -0400
+++ /root/zabbix/frontends/php/include/events.inc.php 2010-05-17 02:59:30.000000000 -0400
@@ -122,7 +122,7 @@
' FROM events e '.
' WHERE e.objectid='.$row['triggerid'].$sql_cond.
' AND e.object='.EVENT_OBJECT_TRIGGER.
- ' ORDER BY e.object, e.objectid, e.eventid';
+ ' ORDER BY e.object, e.objectid, e.clock';
$res = DBselect($sql,1);
if($rows = DBfetch($res)) return $rows['eventid'];
@@ -134,7 +134,7 @@
' WHERE e.eventid > '.$eventid.
' AND e.objectid='.$row['triggerid'].$sql_cond.
' AND e.object='.EVENT_OBJECT_TRIGGER.
- ' ORDER BY e.object, e.objectid, e.eventid';
+ ' ORDER BY e.object, e.objectid, e.clock';
$res = DBselect($sql,1);
if($rows = DBfetch($res)){
@@ -155,7 +155,7 @@
' AND e.objectid='.$row['triggerid'].
' AND e.object='.EVENT_OBJECT_TRIGGER.
' AND e.value='.$row['value'].
- ' ORDER BY e.object, e.objectid, e.eventid';
+ ' ORDER BY e.object, e.objectid, e.clock';
$res = DBselect($sql,1);
$rows = DBfetch($res);
@@ -189,7 +189,7 @@
' AND e.eventid < '.$row['eventid'].
' AND e.object='.EVENT_OBJECT_TRIGGER.
' AND e.value='.TRIGGER_VALUE_FALSE.
- ' ORDER BY e.object DESC, e.objectid DESC, e.eventid DESC';
+ ' ORDER BY e.object DESC, e.objectid DESC, e.clock DESC';
if($rez = DBfetch(DBselect($sql,1))) $eventz[$rez['value']] = $rez['eventid'];
$sql = 'SELECT e.eventid, e.value '.
@@ -198,7 +198,7 @@
' AND e.eventid < '.$row['eventid'].
' AND e.object='.EVENT_OBJECT_TRIGGER.
' AND e.value='.TRIGGER_VALUE_TRUE.
- ' ORDER BY e.object DESC, e.objectid DESC, e.eventid DESC';
+ ' ORDER BY e.object DESC, e.objectid DESC, e.clock DESC';
if($rez = DBfetch(DBselect($sql,1))) $eventz[$rez['value']] = $rez['eventid'];
@@ -209,7 +209,7 @@
' AND e.eventid < '.$row['eventid'].
' AND e.object='.EVENT_OBJECT_TRIGGER.
' AND e.value='.TRIGGER_VALUE_UNKNOWN.
- ' ORDER BY e.object DESC, e.objectid DESC, e.eventid DESC';
+ ' ORDER BY e.object DESC, e.objectid DESC, e.clock DESC';
if($rez = DBfetch(DBselect($sql,1))) $eventz[$rez['value']] = $rez['eventid'];
}
@@ -237,20 +237,20 @@
$sql = 'SELECT e.eventid, e.value, e.clock '.
' FROM events e'.
' WHERE e.objectid='.$row['objectid'].
- ' AND e.eventid > '.$row['eventid'].
+ ' AND e.clock > '.$row['clock'].
' AND e.object='.EVENT_OBJECT_TRIGGER.
' AND e.value='.$row['value'].
- ' ORDER BY e.object, e.objectid, e.eventid';
+ ' ORDER BY e.object, e.objectid, e.clock';
}
else{
$sql = 'SELECT e.eventid, e.value, e.clock '.
' FROM events e'.
' WHERE e.objectid='.$row['objectid'].
- ' AND e.eventid > '.$row['eventid'].
+ ' AND e.clock > '.$row['clock'].
' AND e.object='.EVENT_OBJECT_TRIGGER.
' AND e.value<>'.$row['value'].
$sql_cond.
- ' ORDER BY e.object, e.objectid, e.eventid';
+ ' ORDER BY e.object, e.objectid, e.clock';
}
$rez = DBfetch(DBselect($sql,1));
@@ -322,7 +322,7 @@
'eventid_till' => $curevent['eventid'],
'select_triggers' => API_OUTPUT_EXTEND,
'output' => API_OUTPUT_EXTEND,
- 'sortfield' => 'eventid',
+ 'sortfield' => 'clock',
'sortorder' => ZBX_SORT_DOWN,
'limit' => 20
);
@@ -391,7 +391,7 @@
' WHERE eventid<='.$eventid.
' AND object='.EVENT_OBJECT_TRIGGER.
' AND objectid='.$triggerid.
- ' ORDER BY eventid DESC';
+ ' ORDER BY object, objectid, clock DESC';
$db_events = DBselect($sql, ZBX_WIDGET_ROWS);
$count = 0;
@@ -495,7 +495,7 @@
' WHERE e.object='.EVENT_OBJECT_TRIGGER.
' AND '.DBcondition('e.objectid', $trigger_list).
$sql_cond.
- ' ORDER BY e.eventid DESC';
+ ' ORDER BY e.object, e.objectid, e.clock DESC';
$result = DBselect($sql,10*($start+$num));
}
diff -ruN /root/zabbix-original/frontends/php/include/triggers.inc.php /root/zabbix/frontends/php/include/triggers.inc.php
--- /root/zabbix-original/frontends/php/include/triggers.inc.php 2010-05-15 01:30:00.000000000 -0400
+++ /root/zabbix/frontends/php/include/triggers.inc.php 2010-05-17 02:59:30.000000000 -0400
@@ -2848,7 +2848,7 @@
' WHERE e.objectid='.$triggerid.
' AND e.object='.EVENT_OBJECT_TRIGGER.
' AND e.clock<'.$period_start.
- ' ORDER BY e.eventid DESC';
+ ' ORDER BY e.object, e.objectid, e.clock DESC';
if($row = DBfetch(DBselect($sql,1))){
$start_value = $row['value'];
$min = $period_start;
@@ -2905,7 +2905,7 @@
' AND object='.EVENT_OBJECT_TRIGGER.
' AND clock>='.$min.
' AND clock<='.$max.
- ' ORDER BY clock ASC, eventid ASC';
+ ' ORDER BY e.object ASC, e.objectid ASC, e.clock ASC';
$result=DBselect($sql);
while($row=DBfetch($result)){
$clock=$row['clock'];