Automatic graph of all LLD items per host

[mmarkwitzz]

This script is based on zabbix 2.0 api and LLD which is only available in zabbix 2.0

Description

The intended usage of this script is to automatically generate graphs containing all of the items generated by a LLD rule, for all hosts linked to a specified template.

Currently LLD prototypes allow you to define graphs containing prototype items. These graph prototypes can only contain prototype items, there is no option to create a graph containing all discovered items.

For example, for a discovery rule named volume discovery I have the following item prototypes:

WIN Volume "{#VOLNAME} ({#VOLLABEL})" bytes/sec read
WIN Volume "{#VOLNAME} ({#VOLLABEL})" bytes/sec total
WIN Volume "{#VOLNAME} ({#VOLLABEL})" bytes/sec write
WIN Volume "{#VOLNAME} ({#VOLLABEL})" bytes free
WIN Volume "{#VOLNAME} ({#VOLLABEL})" bytes used

And I have the following graph prototypes:

WIN Volume "{#VOLNAME} ({#VOLLABEL})" bytes/sec stack
WIN Volume "{#VOLNAME} ({#VOLLABEL})" bytes free stack

In a real world scenario I end up with one graph per discovered volume, with no option to create a graph containing all discovered volumes.

Volume discovery: WIN Volume "C: (Windows)" bytes/sec stack
Volume discovery: WIN Volume "C: (Windows)" bytes stack
Volume discovery: WIN Volume "D: (Data)" bytes/sec stack
Volume discovery: WIN Volume "D: (Data)" bytes stack
Volume discovery: WIN Volume "F: (Backup)" bytes/sec stack
Volume discovery: WIN Volume "F: (Backup)" bytes stack
Volume discovery: WIN Volume "G: (Storage)" bytes/sec stack
Volume discovery: WIN Volume "G: (Storage)" bytes stack

What I needed was a graph containing all items generated by the same discovery rule. This script allows you to create the following types of graphs:







Usage

Download the archive, extract and give the zabbix user rx permissions on the lld_all_graph.pl script, edit it, specifically the parameters below, run it, test it, cron it.

	$user = "Admin"; ### username
	$password = "zabbix"; ### password	
	$template = "Windows WIN"; ### only add graphs to hosts linked to this template	
	$url = "http://127.0.0.1/api_jsonrpc.php"; ### intenal zabbix url	
	$graph = 'WIN Volume "ALL" bytes stack'; ### create a graph with this name in each host
	$graphtype = 1; ### 0=normal, 1=stacked
	$mintype = 1; ### 0=calculated, 1=fixed
	$maxtype = 0; ### 0=calculated, 1=fixed
	$minvalue = 0;
	$maxvalue = 100;
	$showtriggers = 1;
	$drawtype = 0; ### 0=line, 1=filled, 2=boldline, 3=dot, 4=dashed, 5=gradient
	$calcfunction = 2; ### 1=min, 4=max, 2=avg, 7=all
	### add graph items mathing these regexes, maximum 2
	$regexes[0] = '^WIN\sVolume\s".*"\sbytes\sfree';
	$regexes[1] = '^WIN\sVolume\s".*"\sbytes\sused';  ### delete this line if not needed.

The script connects using $user and $password via json api and enumerates all hosts that are linked to $template. For each host it enumerates the items that match $regexes[0] or $regexes[1]. It adds them all up and creates (for each host enumerated) a graph named $graph. If the graph already existed, then it first deletes it.

Multiple graph options can be set including the graph type (normal, stacked), whether the min/max values are fixed or calculated, the min/max fixed values, if the triggers are shown. Also, multiple item options can be set including the item drawing type (line, filled, bold line, etc.), the y axis side (left, right) and the function used (avg, min, max, etc.). Colors are chosen automatically.

Gotchas

You only need to specify $regexes[0] (delete de line of $regexes[1]) for simple graphs such as WIN Processor "ALL" above. This adds all items matching $regexes[0] to the graph, using dark colors. If there are more than 21 items in the graph, light colors are also used. A number of 42 items per graph is supported.

Specifying both $regexes[0] and $regexes[1] will generate graphs such as WIN Volumes "ALL" above, where items matching $regexes[0] are alternated with items matching $regexes[1], using alternated dark and light colors.
In my example $regexes[0] matches items for free space and use dark colors, and $regexes[1] matches items for used space and use light colors, as in the graph above. A number of 21 items per regex is supported, 42 in total.

The script does not verify the json return value for errors. For example, it will say "graph created" even if zabbix rejected the json api call.

Update 1

The script now automatically exludes prototype items, you no longer have to exclude them by using regex.
Added new colors, 42 now available.
Added global variable for drawtype and calcfunction. Showtriggers now defaults to true.

[abonne01]

Thanks a lot for this script. I was wondering the same thing.

It seems there is a little bug. When created graph with items, it's taking also item in discovery rules if regexexp isn't precise enough. It could be nice that script doesn't take into account item prototypes.

tips for Linux user:
* lld_all_graph.pl, you need perl-JSON.noarch package (centos)
* file is windows encoded. Just perform a "dos2unix lld_all_graph.pl" to be able to execute it.

Does someone know if this feature will come on next zabbix releases?

Thanks and good job!

Ps : on zip file, these is another scirpt with your password

[mmarkwitzz]

Thanks a lot for this script. I was wondering the same thing.

It seems there is a little bug. When created graph with items, it's taking also item in discovery rules if regexexp isn't precise enough. It could be nice that script doesn't take into account item prototypes.

tips for Linux user:
* lld_all_graph.pl, you need perl-JSON.noarch package (centos)
* file is windows encoded. Just perform a "dos2unix lld_all_graph.pl" to be able to execute it.

Does someone know if this feature will come on next zabbix releases?

Thanks and good job!

Ps : on zip file, these is another scirpt with your password

You are right, in my example I use regex to exclude the item prototype. I use {[^#] to exclude items that contain {# which is the beginning of discovery macros (eg. {#FSNAME}), something item prototypes have to contain to have unique names.

Another way of doing it would be to modify the script and look for {# in each of the item keys.. Something on my todo list

Files have linux eol, I double checked. I'll reupload the archive without the pass, thanks for the tip

[abonne01]

Another way of doing it would be to modify the script and look for {# in each of the item keys.. Something on my todo list

there is no way to detect that item returned is a prototype on zabbix API ? This could be a better solution..

Strange for eol. When I launched script the first time, on the first line (#!/bin/perl) contained a \n\r at the end.. Anyway. easy to fix

What is usage of the 2nd script on archive ? it seems it's not necessary, as if it's a old version.

[mmarkwitzz]

there is no way to detect that item returned is a prototype on zabbix API ? This could be a better solution..

Strange for eol. When I launched script the first time, on the first line (#!/bin/perl) contained a \n\r at the end.. Anyway. easy to fix

What is usage of the 2nd script on archive ? it seems it's not necessary, as if it's a old version.

Right now I have no idee if the api allows to distiguish between items and prototype items.

The second script is just a utility that lists all items for all hosts linked to $template. I use it to get a list of all items and then play with regex buddy on it, thought it could come in handy.

[ehermouet]

nice but error

tks for this

malformed JSON string, neither array, object, number, string or atom, at character offset 1 (before "(end of string)") at /var/www/zabbix-2.0.1/frontends/php/win_process_all_bytessec_stack.pl line 62.


but i have this have this error, do you have idea why ?

[mmarkwitzz]

My guess is that the user/pass are wrong or the user does not have api access.

You can add "print $res;" on line 60 of the script to get some more info from the script.

Edit: Another user solved this by changing the loopback address in $url (in the script) with the server fqdn

[ehermouet]

tks for tour reply

i 'm on ubuntu and i do perl win.... it's like this ?

user and pwd are good i try to ip of server but same result.

EDIT: my url was wrong

tks for your help

[ehermouet]

me again

Script said graph created but in zabbix interface i have no this graph. there is time to wait ?

tks

[mmarkwitzz]

The pl script adds the discovered items to the graph. So check first if you have discovered items in your host. If not, restart the agent and check the error status of the discovery rule of the host.

If there is no discovery rule in the template then try updating to zabbix 2.0.1 and php 5.3, download the zip and import again.

[abonne01]

Hello,

I have a graph with many items and I have an issue with tour script:

    HOST: JCQ - Java CRM portal (10091)
        ITEM: Http Sessions -  C3_JCQ_25 C3 server1 (24223)
        ITEM: Http Sessions -  C3_JCQ_25 C3 server2 (24224)
        ITEM: Http Sessions -  J21 C3 server0 (24033)
        ITEM: Http Sessions -  J21 C3 server1 (24034)
        ITEM: Http Sessions -  J21 C3 server2 (24183)
        ITEM: Http Sessions -  J22 BY server0 (24160)
        ITEM: Http Sessions -  J22 BY server0 (24167)
        ITEM: Http Sessions -  J22 BY server1 (24168)
        ITEM: Http Sessions -  J22 BY server2 (24182)
        ITEM: Http Sessions -  J23 C6 server0 (24181)
        ITEM: Http Sessions -  J23 C6 server0 (24029)
        ITEM: Http Sessions -  J23 C6 server1 (24030)
        ITEM: Http Sessions -  J23 C6 server2 (24188)
        ITEM: Http Sessions -  J24 BV server0 (24198)
        WARNING: Not enough colors, switched to another color base
        ITEM: Http Sessions -  J24 BV server0 (24210)
        ITEM: Http Sessions -  J24 BV server1 (24199)
        ITEM: Http Sessions -  J24 BV server1 (24211)
        ITEM: Http Sessions -  J24 BV server2 (24200)
        ITEM: Http Sessions -  J24 BV server2 (24212)
        ITEM: Http Sessions -  J25 C3 server0 (24222)
        ITEM: Http Sessions -  J25 C3 server1 (24231)
        ITEM: Http Sessions -  J25 C3 server2 (24232)
        ITEM: Http Sessions -  JC20 BV server0 (24027)
        ITEM: Http Sessions -  JC20 BV server1 (24028)
        ITEM: Http Sessions -  JC20 BV server2 (24184)
{"jsonrpc":"2.0","error":{"code":-32602,"message":"Invalid params.","data":"Incorrect colour \"\"."},"id":4}

            Graph created: Sessions http ALL

Maybe you can help.

[mmarkwitzz]

Hello,

I have a graph with many items and I have an issue with tour script:

    HOST: JCQ - Java CRM portal (10091)
        ITEM: Http Sessions -  C3_JCQ_25 C3 server1 (24223)
        ITEM: Http Sessions -  C3_JCQ_25 C3 server2 (24224)
        ITEM: Http Sessions -  J21 C3 server0 (24033)
        ITEM: Http Sessions -  J21 C3 server1 (24034)
        ITEM: Http Sessions -  J21 C3 server2 (24183)
        ITEM: Http Sessions -  J22 BY server0 (24160)
        ITEM: Http Sessions -  J22 BY server0 (24167)
        ITEM: Http Sessions -  J22 BY server1 (24168)
        ITEM: Http Sessions -  J22 BY server2 (24182)
        ITEM: Http Sessions -  J23 C6 server0 (24181)
        ITEM: Http Sessions -  J23 C6 server0 (24029)
        ITEM: Http Sessions -  J23 C6 server1 (24030)
        ITEM: Http Sessions -  J23 C6 server2 (24188)
        ITEM: Http Sessions -  J24 BV server0 (24198)
        WARNING: Not enough colors, switched to another color base
        ITEM: Http Sessions -  J24 BV server0 (24210)
        ITEM: Http Sessions -  J24 BV server1 (24199)
        ITEM: Http Sessions -  J24 BV server1 (24211)
        ITEM: Http Sessions -  J24 BV server2 (24200)
        ITEM: Http Sessions -  J24 BV server2 (24212)
        ITEM: Http Sessions -  J25 C3 server0 (24222)
        ITEM: Http Sessions -  J25 C3 server1 (24231)
        ITEM: Http Sessions -  J25 C3 server2 (24232)
        ITEM: Http Sessions -  JC20 BV server0 (24027)
        ITEM: Http Sessions -  JC20 BV server1 (24028)
        ITEM: Http Sessions -  JC20 BV server2 (24184)
{"jsonrpc":"2.0","error":{"code":-32602,"message":"Invalid params.","data":"Incorrect colour \"\"."},"id":4}

            Graph created: Sessions http ALL

Maybe you can help.

You have too many items in your graphs.
As stated in the first post, the script supports a number of 42 items per graph if using only $regexes[0], or 21 items if using both $regexes[0] and $regexes[1]

You are free to add more colors to the script, you only have to edit it and add more values to the $colors.

When adding additional colors, add them in pairs, one value for $colors[0][x] and another for $colors[1][x]
The [0] should be a dark color and [1] should be a ligher color.

The colors are in html format.

EDIT: Now that I stopped and counted the items in your example I recommend that you re-download the script again. The version you use only supports around 24 items (I forgot the exact number but new release has 42 max)

[screeble]

First of all many thanks for your amazing script.

I'm trying to adapt your script to create graphs for my prototypes



Then I've edited the script as you requested:

        #########
        # modify these values accordingly
        #########
        # user
        $user = "Admin";
        # password
        $password = "zabbix";
        # only add graphs to hosts linked to this template
        $template = "Template OS Linux";
        # internal
        $header = "Content-Type:application/json";
        # intenal zabbix url
        $url = "http://127.0.0.1/zabbix/api_jsonrpc.php";
        # create a graph with this name in each host
        $graph = 'IO read on "ALL"';
        $graphtype = 1; ### 0=normal, 1=stacked
        $mintype = 1; ### 0=calculated, 1=fixed
        $maxtype = 0; ### 0=calculated, 1=fixed
        $minvalue = 0;
        $maxvalue = 100;
        $showtriggers = 1;
        $drawtype = 0; ### 0=line, 1=filled, 2=boldline, 3=dot, 4=dashed, 5=gradient
        $calcfunction = 2; ### 1=min, 4=max, 2=avg, 7=all
        # add graph items mathing these regexes, maximum 2
        $regexes[0] = '^IO\sread\son\s"';
#       $regexes[1] = '^WIN\sVolume\s".*"\sbytes/sec\sread';

Script successfully accomplished his work and returned these results:
HOST: host1 (10088)
Graph created: IO read on "ALL"
HOST: host2 (10100)
Graph created: IO read on "ALL"
HOST: host3 (10124)
Graph created: IO read on "ALL"

but no graphs were found in zabbix interface. Could you please guide me where I made mistake?

[screeble]

everything is ok now)

there was problem with the line $regexes[0] = '^IO\sread\son\s"';

I've deleted `"`.

[pietro54]

Hi, maybe someone can help me,
I conversed code by dos2uinx and change lines like password etc.
Im getting this error:

unexpected end of string while parsing JSON string, at character offset 2 (before "ackground-color:#fff...") at ./lld_all_graph.pl line 62

What i am doing wrong?