Ad Widget

Collapse

Millisecond resolution (real-time control) is even theoretically possible?

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • jpka
    Junior Member
    • Sep 2013
    • 24

    #1

    Millisecond resolution (real-time control) is even theoretically possible?

    Hi!
    I try to use Zabbix for real-time SNMP-based industrial process control, which definetely need to much less than one second time resolution and reaction delay.
    1. Is there any internal 'heartbeat' currently which limits time domain resolution of triggering and running user scripts ?
    2. If so (looks like it is around one or two seconds), is there any possibility to make it around 1...10 ms ? (i mean when total q'ty of items and triggers is quite low).
    3. Is database capable to store 100 to 1000 pcs. of same value per second, with correspond millisecond timestamp ?
    4. If No, No and No, how much will be resources need, for implement this?
    Thank you so much!
    PS.
    The only real-time related information is '--real-time' key for zabbix_sender, it sounds good, but no any information about what it means: when i use it for snmptraps, they still appear in history (Monotoring - Latest data - select my host - History) with big delay (from 1 to 5 second) and time-stamped with 1 second resolution, which is too coarse.
  • jan.garaj
    Senior Member
    Zabbix Certified Specialist
    • Jan 2010
    • 506

    #2
    3.) Yes, even database is prepared for nanosecond time resolution (table history has timestamp + ns field for real timestamp).

    Zabbix works with nanosecond time resolution, just frontend don't show nanosecond parts.

    IMHO:
    - you can implement your own app, which can run check every ms and then it will send values to proccesing to Zabbix - implement Zabbix agent protocol with ns precision http://zabbix.org/wiki/Docs/protocols/zabbix_agent/2.0
    - I'm very sceptic with real time proccessing of your check - some ms for tcp transferring of values, some ms (maybe sec) for proccesing in Zabbix server
    Devops Monitoring Expert advice: Dockerize/automate/monitor all the things.
    My DevOps stack: Docker / Kubernetes / Mesos / ECS / Terraform / Elasticsearch / Zabbix / Grafana / Puppet / Ansible / Vagrant

    Comment

    • jpka
      Junior Member
      • Sep 2013
      • 24

      #3
      Hi! Thank you for answer.
      At this moment, i suddenly discover that so fundamental and essential problem as make the software realtime, can have simple solution.
      When i make in selfmon.c something like
      Code:
      	if (ZBX_PROCESS_TYPE_CONFSYNCER == process_type)
      	{
      		do
      		{
      //			sleep(1);
      			usleep(1*10000);
      		}
      		while (0 < --sleep_remains);
      	}
      	else
      //		sleep(sleeptime);
      		usleep(sleeptime*10000);
      
      	update_selfmon_counter(ZBX_PROCESS_STATE_BUSY);
      i get almost immediate reaction on traps! (yet can't say how much is processing delay, but always much less than 1 s).
      Yes, it will have negative effect on CPU load, but really almost not change it when 10 ms heartbeat cycle i try as shown. (also, my setup is very tiny, 20 hosts at most).
      And this is quite good news, and exactly what i need.
      (Very few constants also should be corrected in different places of server to preserve its behaviour on housekeeping, etc, but not essential).
      But what looks much easy to implement is increase time resolution displayed by frontend - so hard for me. Is there any hints or example how i can make (at Monitoring->Latest data, for ex.) displayed sub-seconds? Especially if we take into account that
      Zabbix works with nanosecond time resolution
      ?
      Thank you so much!

      Comment

      • jan.garaj
        Senior Member
        Zabbix Certified Specialist
        • Jan 2010
        • 506

        #4
        You have to keep hacking :-) Now frontend.
        It looks that include/func.inc.php function zbx_date2str() is responsible for displaying time values. But
        Code:
        $lastClock = zbx_date2str(DATE_TIME_FORMAT_SECONDS, $lastHistory['clock']);
        input is always timestamp clock (second resolution) and nanosecond part (field ns) is missing. So you should fetch also ns from DB and then process it in zbx_date2str to required string.
        Devops Monitoring Expert advice: Dockerize/automate/monitor all the things.
        My DevOps stack: Docker / Kubernetes / Mesos / ECS / Terraform / Elasticsearch / Zabbix / Grafana / Puppet / Ansible / Vagrant

        Comment

        • jpka
          Junior Member
          • Sep 2013
          • 24

          #5
          Dear jan.garaj, this information was really very helpful. I also discover that all nanosecond functionality hopefully already present in frontend also, even 'ns' value already fetched; just by unknown reason it's not displayed currently.
          I write this (dirty but simplest possible) fix and it works well for me:
          Code:
          diff -bur /usr/share/zabbix/include/classes/screens/CScreenHistory.php /usr/share/zabbix.ns/include/classes/screens/CScreenHistory.php
          --- /usr/share/zabbix/include/classes/screens/CScreenHistory.php	2014-04-07 13:03:42.000000000 +0400
          +++ /usr/share/zabbix.ns/include/classes/screens/CScreenHistory.php	2014-05-28 11:25:48.000000000 +0400
          @@ -214,7 +214,7 @@
           							}
           						}
           
          -						$row = array(nbsp(zbx_date2str(_('Y.M.d H:i:s'), $data['clock'])));
          +						$row = array(nbsp(zbx_date2str(_('Y.M.d H:i:s'), $data['clock'], $data['ns'])));
           
           						if ($isManyItems) {
           							$row[] = $host['name'].NAME_DELIMITER.$item['name_expanded'];
          @@ -251,8 +251,8 @@
           						$historyTable->addRow($newRow);
           					}
           					else {
          -						$output[] = zbx_date2str(HISTORY_LOG_ITEM_PLAINTEXT, $data['clock']);
          -						$output[] = "\t".$data['clock']."\t".htmlspecialchars($data['value'])."\n";
          +						$output[] = zbx_date2str(HISTORY_LOG_ITEM_PLAINTEXT, $data['clock'], $data['ns']);
          +						$output[] = "\t".$data['clock'].$data['ns']."\t".htmlspecialchars($data['value'])."\n";
           					}
           				}
           
          @@ -287,14 +287,14 @@
           						}
           
           						$historyTable->addRow(array(
          -							zbx_date2str(HISTORY_ITEM_DATE_FORMAT, $data['clock']),
          +							zbx_date2str(HISTORY_ITEM_DATE_FORMAT, $data['clock'], $data['ns']),
           							zbx_nl2br($value)
           						));
           					}
           					// plain text
           					else {
          -						$output[] = zbx_date2str(HISTORY_PLAINTEXT_DATE_FORMAT, $data['clock']);
          -						$output[] = "\t".$data['clock']."\t".htmlspecialchars($value)."\n";
          +						$output[] = zbx_date2str(HISTORY_PLAINTEXT_DATE_FORMAT, $data['clock'], $data['ns']);
          +						$output[] = "\t".$data['clock'].$data['ns']."\t".htmlspecialchars($value)."\n";
           					}
           				}
           
          diff -bur /usr/share/zabbix/include/func.inc.php /usr/share/zabbix.ns/include/func.inc.php
          --- /usr/share/zabbix/include/func.inc.php	2014-04-07 13:03:42.000000000 +0400
          +++ /usr/share/zabbix.ns/include/func.inc.php	2014-05-28 11:27:19.000000000 +0400
          @@ -204,7 +204,7 @@
           }
           
           // Convert timestamp to string representation. Retun 'Never' if 0.
          -function zbx_date2str($format, $value = null) {
          +function zbx_date2str($format, $value = null, $value_ns = null) {
           	static $weekdaynames, $weekdaynameslong, $months, $monthslong;
           
           	$prefix = '';
          @@ -220,6 +220,10 @@
           		return _('Never');
           	}
           
          +	if ($value_ns === null) {
          +		$value_ns = 555777999;	// for debug, should not be zero.
          +	}
          +
           	if (!is_array($weekdaynames)) {
           		$weekdaynames = array(
           			0 => _('Sun'),
          @@ -303,7 +307,10 @@
           
           	$output .= (zbx_strlen($part) > 0) ? date($part, $value) : '';
           
          -	return $prefix.$output;
          +//	return $prefix.$output;
          +	$value_ms = intval ($value_ns / 1000000);
          +	return $prefix.$output.'.'.$value_ms;
          +
           }
           
           // calculate and convert timestamp to string representation
          Thank you so much!

          Comment

          Working...