Undefined offset: 35 [httpdetails.php:82 → CScreenHttpTestDetails->get()

[tolland]

Hi,

Update: this issue arises when creating and adding Web Scenarios to Templates, and applying them to hosts using the API. Currently it appears that adding a Web Scenario to a Template, doesn't automatically propagate that to the linked hosts.

(systems details are pasted in at the end of the post - however both zabbix and the agent are running version 3.4.10-1.el7 with MySQL backend)

I've added some Web Monitors and their triggers to a Template called "Web Checks" using the API, and added this template to my main zabbix server "zabbix". However when trying to view this in "Monitoring -> Web" I am seeing PHP errors in the output.... The error seems to be related to when the httpdetails.php screen attempts to call getMinClock() in CHistoryManager.php

I am seeing this error on the Monitoring-Web screen...



Initially the page seems to return ok, but after a few minutes, the error appears and stays there.

Undefined offset: 35 [httpdetails.php:82 → CScreenHttpTestDetails->get() in
                            include/classes/screens/CScreenHttpTestDetails.php:99]
Invalid argument supplied for foreach() [httpdetails.php:82 →
                                  CScreenHttpTestDetails->get() in include/classes/screens/CScreenHttpTestDetails.php:122]
min(): Array must contain at least one element [httpdetails.php:121 →
                      CHistoryManager->getMinClock() → min() in include/classes/api/managers/CHistoryManager.php:728

The template for the web monitoring looks like this

<?xml version="1.0" encoding="UTF-8"?>
<zabbix_export>
<version>3.4</version>
<date>2018-06-30T21:33:42Z</date>
<groups>
<group>
<name>Templates</name>
</group>
<group>
<name>Web Checks</name>
</group>
</groups>
<templates>
<template>
<template>Web Checks</template>
<name>Web Checks</name>
<description/>
<groups>
<group>
<name>Templates</name>
</group>
<group>
<name>Web Checks</name>
</group>
</groups>
<applications/>
<items/>
<discovery_rules/>
<httptests>
<httptest>
<name>somesite.com</name>
<application/>
<delay>1m</delay>
<attempts>1</attempts>
<agent>Zabbix</agent>
<http_proxy/>
<variables/>
<headers/>
<status>0</status>
<authentication>0</authentication>
<http_user/>
<http_password/>
<verify_peer>0</verify_peer>
<verify_host>0</verify_host>
<ssl_cert_file/>
<ssl_key_file/>
<ssl_key_password/>
<steps>
<step>
<name>somesite.com-1</name>
<url>http://somesite.com</url>
<query_fields/>
<posts/>
<variables/>
<headers/>
<follow_redirects>1</follow_redirects>
<retrieve_mode>0</retrieve_mode>
<timeout>15s</timeout>
<required/>
<status_codes/>
</step>
</steps>
</httptest>
</httptests>
<macros/>
<templates/>
<screens/>
</template>
</templates>
<triggers>
<trigger>
<expression>{Web Checks:web.test.error[somesite.com].strlen()}&gt;0</expression>
<recovery_mode>0</recovery_mode>
<recovery_expression/>
<name>Web scenario &quot;Scenario&quot; failed: {ITEM.VALUE} (alt)</name>
<correlation_mode>0</correlation_mode>
<correlation_tag/>
<url/>
<status>0</status>
<priority>3</priority>
<description/>
<type>0</type>
<manual_close>0</manual_close>
<dependencies/>
<tags/>
</trigger>
</triggers>
</zabbix_export>

Background
Running zabbix packages from the draios repos;

# cat /etc/yum.repos.d/draios.repo.repo
[draios]
baseurl = http://download.draios.com/stable/rpm/$basearch
enabled = 1
gpgcheck = 1
name = Draios

Packages are updated...

[root@zabbix ~]# rpm -qa | grep zabbix
zabbix-web-mysql-3.4.10-1.el7.noarch
zabbix-get-3.4.10-1.el7.x86_64
zabbix-agent-3.4.10-1.el7.x86_64
zabbix-web-3.4.10-1.el7.noarch
zabbix-server-mysql-3.4.10-1.el7.x86_64
zabbix-sender-3.4.10-1.el7.x86_64

System is CentOS 7.5

[root@zabbix ~]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)

[tolland]

So examining the code from the error in

/usr/share/zabbix/include/classes/api/managers/CHistoryManager.php

at line 728

it looks like neither of these 2 are returning anything;

$min_clock = [];
if (array_key_exists(ZBX_HISTORY_SOURCE_ELASTIC, $storage_items)) {
$min_clock[] = $this->getMinClockFromElasticsearch($storage_items[ZBX_HISTORY_SOURCE_ELASTIC]);
}
if (array_key_exists(ZBX_HISTORY_SOURCE_SQL, $storage_items)) {
$min_clock[] = $this->getMinClockFromSql($storage_items[ZBX_HISTORY_SOURCE_SQL], $source);
}
$min_clock = min($min_clock); 

presumably this one should be getting something....

    if (array_key_exists(ZBX_HISTORY_SOURCE_SQL, $storage_items)) {
      $min_clock[] = $this->getMinClockFromSql($storage_items[ZBX_HISTORY_SOURCE_SQL], $source);
    }

    $min_clock = min($min_clock); 

from debugging, I can see that $storage_items is empty, because $item_types is an array of empty arrays;

foreach ($item_types as $type => $itemids) {  
error(_s('type is: %1$s.', str_replace("\n",' ',print_r($type, true))));  
error(_s('itemids is: %1$s.', str_replace("\n",' ',print_r($itemids, true))));        
if (!$itemids) {          
     continue;        
}          
    $history_source = self::getDataSourceType($type);        
    $storage_items[$history_source][$type] = $itemids;      
} 

returns;

type is: 0.
itemids is: Array ( ) .
type is: 1.
itemids is: Array ( ) .
type is: 2.
itemids is: Array ( ) .
type is: 3.
itemids is: Array ( ) .
type is: 4.
itemids is: Array ( ) .
storage types is: Array ()

So it looks like that $item_types is setup wrong somehow

... further debugging shows that

    public function getMinClock(array $items) {  
error(_s('item count is: %1$s.', str_replace("\n",' ',print_r(count ($items), true)))); 

outputs...

• item count is: 0.

the $items array passed in is empty....

[kloczek]

You did not provide:
- details in what kind scenario are those errors
- used version of the zabbix
- in which file you see fragment of the code

[tolland]

You did not provide:
- details in what kind scenario are those errors
- used version of the zabbix
- in which file you see fragment of the code

Thanks. Apologies, I had been watched the football, and had a few beers, so I wasn't thinking straight when I made the first post.. :-)

I added the details and some more debugging...

[tolland]

So it looks like public function getMinClock(array $items) is call from line 121 in httpdetails.php

$graph_in->timeline['starttime'] = date(TIMESTAMP_FORMAT, Manager::History()->getMinClock($items)); 

dumping out the query that generates the $items array gives this;

$items = DBfetchArray(DBselect(
    'SELECT i.itemid,i.value_type,i.history,i.trends,i.hostid'.
    ' FROM items i,httpstepitem hi,httpstep hs'.
    ' WHERE i.itemid=hi.itemid'.
        ' AND hi.httpstepid=hs.httpstepid'.
        ' AND hs.httptestid='.zbx_dbstr($httptest['httptestid'])
)); 

SELECT
    i.itemid,
    i.value_type,
    i.history,
    i.trends,
    i.hostid
FROM
    items i,
    httpstepitem hi,
    httpstep hs
WHERE
    i.itemid=hi.itemid
AND hi.httpstepid=hs.httpstepid
AND hs.httptestid='20'

which returns no results in the databases;

Empty result set fetched

So I deleted all the checks except one, and then dumped out all the web checks;

SELECT
    i.itemid,
    i.value_type,
    i.history,
    i.trends,
    i.hostid
FROM
    items i,
    httpstepitem hi,
    httpstep hs
WHERE
    i.itemid=hi.itemid
AND hi.httpstepid=hs.httpstepid

which returns

name                                           hostid itemid value_type history trends hostid
---------------------------------------------- ------ ------ ---------- ------- ------ ------
Download speed for step "$2" of scenario "$1". 10282  29738  0          30d     90d    10282
Response time for step "$2" of scenario "$1".  10282  29739  0          30d     90d    10282
Response code for step "$2" of scenario "$1".  10282  29740  3          30d     90d    10282

so if I create a new Web Scenario and apply it to that template, the SQL is showing results for both the template and the host...

SELECT
    h.host,
    h.hostid,
    i.name,
    i.hostid,
    i.itemid,
    i.value_type,
    i.history,
    i.trends,
    i.hostid
FROM
    hosts h,
    items i,
    httpstepitem hi,
    httpstep hs
WHERE
    h.hostid=i.hostid
AND    i.itemid=hi.itemid
AND hi.httpstepid=hs.httpstepid

host       hostid name                                           hostid itemid value_type history trends hostid
---------- ------ ---------------------------------------------- ------ ------ ---------- ------- ------ ------
Web Checks 10282  Download speed for step "$2" of scenario "$1". 10282  29738  0          30d     90d    10282
Web Checks 10282  Response time for step "$2" of scenario "$1".  10282  29739  0          30d     90d    10282
Web Checks 10282  Response code for step "$2" of scenario "$1".  10282  29740  3          30d     90d    10282
Web Checks 10282  Download speed for step "$2" of scenario "$1". 10282  29750  0          30d     90d    10282
Web Checks 10282  Response time for step "$2" of scenario "$1".  10282  29751  0          30d     90d    10282
Web Checks 10282  Response code for step "$2" of scenario "$1".  10282  29752  3          30d     90d    10282
zabbix     10265  Download speed for step "$2" of scenario "$1". 10265  29756  0          30d     90d    10265
zabbix     10265  Response time for step "$2" of scenario "$1".  10265  29757  0          30d     90d    10265
zabbix     10265  Response code for step "$2" of scenario "$1".  10265  29758  3          30d     90d    10265

so it looks like for that Web Scenario that is giving the error, it was never applied to the host, and the corresponding item, httpstepitem and httpstep wasn't created on the host....

Any ideas why that would happen... it seems intermittent

[kloczek]

In your last comment still there is no details about I've been asking.
Just back to this thread when you will finish watching football

[tolland]

OK, so further debugging suggests that under some circumstances, when using the API to create a web scenario on a template, the api fails to insert a httpstepitem record for the httpstep on host which has the template applied to it.

Unfortunately I have not been able to create a simplified demonstration of the bug, as the simple case of creating a one-shot web scenario works fine, as do my follow up tests from the Web UI

The problem can be seen from the httpstep and httpstepitem records that were created;

SELECT
        h.host hostname,
       i.*

FROM
    hosts h,
    items i,
    httpstep hs,
    httpstepitem hsi


WHERE  h.hostid = i.hostid
AND    i.itemid = hsi.itemid
AND    hsi.httpstepid = hs.httpstepid

produces this list;

hostname   itemid type snmp_community snmp_oid hostid name                                           key_                                                            delay history trends status value_type trapper_hosts units snmpv3_securityname snmpv3_securitylevel snmpv3_authpassphrase snmpv3_privpassphrase formula error lastlogsize logtimefmt templateid valuemapid params ipmi_sensor authtype username password publickey privatekey mtime flags interfaceid port description inventory_link lifetime snmpv3_authprotocol snmpv3_privprotocol state snmpv3_contextname evaltype jmx_endpoint master_itemid
---------- ------ ---- -------------- -------- ------ ---------------------------------------------- --------------------------------------------------------------- ----- ------- ------ ------ ---------- ------------- ----- ------------------- -------------------- --------------------- --------------------- ------- ----- ----------- ---------- ---------- ---------- ------ ----------- -------- -------- -------- --------- ---------- ----- ----- ----------- ---- ----------- -------------- -------- ------------------- ------------------- ----- ------------------ -------- ------------ -------------
Web Checks 29738  9                            10282  Download speed for step "$2" of scenario "$1". web.test.in[biohackspace.org,biohackspace.org-1,bps]            1m    30d     90d    0      0                        Bps                       0                                                                              0                      (null)     (null)                        0                                               0     0     (null)                       0              30d      0                   0                   0                        0                     (null)      
Web Checks 29739  9                            10282  Response time for step "$2" of scenario "$1".  web.test.time[biohackspace.org,biohackspace.org-1,resp]         1m    30d     90d    0      0                        s                         0                                                                              0                      (null)     (null)                        0                                               0     0     (null)                       0              30d      0                   0                   0                        0                     (null)      
Web Checks 29740  9                            10282  Response code for step "$2" of scenario "$1".  web.test.rspcode[biohackspace.org,biohackspace.org-1]           1m    30d     90d    0      3                                                  0                                                                              0                      (null)     (null)                        0                                               0     0     (null)                       0              30d      0                   0                   0                        0                     (null)      
zabbix     29762  9                            10265  Download speed for step "$2" of scenario "$1". web.test.in[biohackspace.org,biohackspace.org-1,bps]            1m    30d     90d    0      0                        Bps                       0                                                                              0                      29738      (null)                        0                                               0     0     (null)                       0              30d      0                   0                   0                        0                     (null)      
zabbix     29763  9                            10265  Response time for step "$2" of scenario "$1".  web.test.time[biohackspace.org,biohackspace.org-1,resp]         1m    30d     90d    0      0                        s                         0                                                                              0                      29739      (null)                        0                                               0     0     (null)                       0              30d      0                   0                   0                        0                     (null)      
zabbix     29764  9                            10265  Response code for step "$2" of scenario "$1".  web.test.rspcode[biohackspace.org,biohackspace.org-1]           1m    30d     90d    0      3                                                  0                                                                              0                      29740      (null)                        0                                               0     0     (null)                       0              30d      0                   0                   0                        0                     (null)      
Web Checks 29810  9                            10282  Download speed for step "$2" of scenario "$1". web.test.in[londonbiological.com,londonbiological.com-1,bps]    1m    30d     90d    0      0                        Bps                       0                                                                              0                      (null)     (null)                        0                                               0     0     (null)                       0              30d      0                   0                   0                        0                     (null)      
Web Checks 29811  9                            10282  Response time for step "$2" of scenario "$1".  web.test.time[londonbiological.com,londonbiological.com-1,resp] 1m    30d     90d    0      0                        s                         0                                                                              0                      (null)     (null)                        0                                               0     0     (null)                       0              30d      0                   0                   0                        0                     (null)      
Web Checks 29812  9                            10282  Response code for step "$2" of scenario "$1".  web.test.rspcode[londonbiological.com,londonbiological.com-1]   1m    30d     90d    0      3                                                  0                                                                              0                      (null)     (null)                        0                                               0     0     (null)                       0              30d      0                   0                   0                        0                     (null)

and you can see from the list of httpstepitem that the records are missing in httpstepitem for the second web scenario for the host that has the template "Web Checks" applied to it;

httpstepitemid httpstepid itemid type
-------------- ---------- ------ ----
238            80         29738  2  
239            80         29739  1  
240            80         29740  0  
253            85         29762  2  
254            85         29763  1  
255            85         29764  0  
286            96         29810  2  
287            96         29811  1  
288            96         29812  0

There should be 12 records there, not 9....



My simplified example is working fine, so it must be something in my setup that is triggering the problem

  #!/bin/bash
    # --trace-ascii -
  authid=$( curl -s --show-error -X POST "http://zabbix/zabbix/api_jsonrpc.php" -H 'Content-Type: application/json' -d"
  {
  \"jsonrpc\": \"2.0\",
  \"method\": \"user.login\",
  \"params\": {
  \"user\": \"${zabbix_user}\",
  \"password\": \"${zabbix_pass}\"
  },
  \"auth\": null,
  \"id\": 0
  }" | jq '.result' | tr -d '"'
  )
    echo $authid
    ## {'hostid': u'10282', 'steps': [{'url': 'http://londonbiological.com', 'name':
  # 'londonbiological.com-1', 'no': 1}], 'name': 'londonbiological.com'}
    curl --trace-ascii - -X POST "http://zabbix/zabbix/api_jsonrpc.php" -H 'Content-Type: application/json' -d @- <<EOF
  {
  "jsonrpc": "2.0",
  "method": "httptest.create",
  "params": {
  "name": "Homepage check",
  "hostid": "10282",
  "steps": [
  {
  "name": "Homepage",
  "url": "http://mycompany.com",
  "no": 1
  }
  ]
  },
  "auth": "$authid",
  "id": 1
  }
  EOF