PATCH: Lua Scripts in Zabbix

[nelsonab]

** Documentation and latest version can be found at: http://trac.red-tux.net/wiki/lua **


This patch is for Zabbix 1.8.3, I will work on a 1.8.4 version soon.

This patch requires no DB changes.

This patch will require a recompile and a reinstall of your php frontend. (same as an upgrade)

You will need to have the lua development libraries installed for this to work.

To install patch
1) Untar zabbix 1.8.3 into a fresh directory
1a) Download patch (see link above)
2) In the new 1.8.3 directory run the following command: patch -p0 < lua-zabbix-1.8.3.patch (if that does not work try patch -p1 < lua-zabbix-1.8.3.patch)
3) Run "configure" with the following option --with-lua
4) compile and install
5) copy frontend directory as you would a normal install/upgrade


So what does this patch do? It allows you to create your own Lua scripts and run them as a normal item. Right now there are only two functions available in the lua script engine, getitem(itemid) (returns val,clock in that order) and zabbix_log(level,message).

Configuring is pretty straight forward as a new item item is added called "Lua Script" (type 16 for those interested). The key just needs to be in the format of lua[unique identifier]. Also it is important that your lua script return a value using the "return" statement.

In addition you can also set up your own lua library. Just point the zabbix_server.conf variable to where you want to load your library for example:
LuaScriptLibrary=/etc/zabbix/lua/zabbix.lua

There is also a configuration option to start more Lua Pollers using
StartLUAPollers, the default value is 1.

In the following example I have created a lua library with the following:

function sum(...)
total=0
for i=1,#arg do
total = total + get_item(arg[i])
end
return total
end
The main goal of this patch is to allow for people to do simple scripting within Zabbix. Imagine if you needed to create a standard deviation function for various items, how would you implement that now? Via an external script which would either query the API or the database directly. Also looking long term I would also like to tie this into triggers, thus allowing for the writing of more complex triggers using Lua. Imagine conditional triggers where the value of one item determines the sensitivity or values a trigger will activate on for other items.

If you have suggestions for the next patch release let me know.

[nelsonab]

Attached to this message is the latest version of the Lua Patch for Zabbix.

To install this patch:
1) Download a fresh Zabbix-1.8.3 and untar
2) Download patch
3) cd to new Zabbix-1.8.3 directory and run the following command
patch -p1 < (path_to_patch)1.8.3-p2-lua.patch
4) run the bootstrap.sh (bash bootstrap.sh), this will regenerate the appropriate autoconf scripts and headers
5) configure with the --with-lua option.

Overall things are similar to the above. In the config file there are now the following options:

StartLUAPollers - (default 1) The number of Lua pollers to fork
LuaScriptLibrary - The location where the lua script library cab be found
LuaCheckPermissions - (0 = off -> default, 1 = on) Turns on permissions checking for database related Lua commands
LuaDefaltUser - (default: guest) The default user to use should the user information not be provided

The following commands are available:
zabbix_log(LogLevel,Message) - displays a message in the zabbix_server log with the prefix of lua_zabbix_log
get_last(itemid) - returns val,clock in that order
now() - returns the current time in seconds since epoch

The available log levels are:
LOG_LEVEL_EMPTY
LOG_LEVEL_CRIT
LOG_LEVEL_ERR
LOG_LEVEL_WARNING
LOG_LEVEL_DEBUG

When creating a new Lua item the Alias of the user used to create the object is written to the item's properties in the database. When get_last is called the alias used to create the item is checked to see if they have read permission to the item. If they do not an error is returned and the item becomes "Not Supported". No permissions has the lowest priority (So far I'm not 100% sure what the order or priority is for Zabbix, though I think no permissions should be highest followed by read only then read/write).

More information can also be found here:
http://trac.red-tux.net/wiki/ZabbixLua

If you have any questions or comments let me know!

Enjoy!

[qix]

Good job on this!
This is a feature zabbix needs in my opinion.
This way we can use small scripts from within Zabbix so we don't need shell access to the server to setup external scripts.

I hope this will eventually mean a merge with the main Zabbix source tree :)

[nelsonab]

Attached is another version of the patch.

Again this is for 1.8.3, for installation instructions see my previous post.

This version adds script timeouts. There is also a new config file option LuaTimeout. The default timeout is 15 seconds, the value can be set from 0 to 60 seconds. When a script times, an error is returned and the item becomes "Not Supported".

[qix]

Timeouts are a good idea. However, I'm not sure about the 'fail once, become unsupported'. Maybe it should be timeout 3 times in row, then become unsupported? I can imagine a script taking a while because of unforeseen external influences :)

[nelsonab]

Unfortunately there's no easy way around that. Zabbix automatically sets something as "Not Supported" the moment there is an error, this is true of all items. I could dive into "fixing" this but that would be a lot of work and I'd like to keep my focus on this in the hopes that one day it may gain inclusion. :-)

[qix]

Fair enough :)

[nelsonab]

Attached is a patch for Zabbix 1.8.4 There are no new features in this patch.

To install see post #2

[richlv]

note that unsupported items are retried every 10 minutes anyway - see administration -> general -> general or somewhere in that dropdown to change that

[clopez]

Hello,

I am testing your patch on Zabbix 1.8.4 but I am not sure how to pass the name of an item to the function get_last.




I have a host with an Zabbix trapper item.


The KEY of this item is "TEMP"

The itemid (on SQL database) of this item is 22182



On the same host I am trying to create a "Lua Script" Item with key "lua[TEMP]"

If I use the following code for this Lua Script:


value = get_last(22182)
return (value)


It works as expected and I see in the Zabbix debug log that it does the following SQL query
Query [txnlev:0] [select clock,value from history_uint where itemid=22182 order by clock desc limit 1]

But If I put the Item name instead of the itemid it don't works:


value = get_last(TEMP)
return (value)


It is not able to translate the key of the item (TEMP) to the itemid (22182).


Query [txnlev:0] [select clock,value from history_uint where itemid=0 order by clock desc limit 1]



What I am missing? Thanks!

[nelsonab]

You are not missing anything. :-) Right now get_last cannot accept anything other than numeric item ids.

Let me see what I can do with regards to writing a lookup routine which can allow you to look up what the itemid is for a host:item combination. You can then take that and feed it to the get_last function. Also I should add some type checking to ensure get_last receives a numeric value.

Thanks for the feedback. :)

[clopez]

Within the item type "Calculated" I simply can use last(TEMP) and Zabbix is able to translate it. It would be very nice that the "lua script" type was able to translate/understand the item keys as the "calculated" item type does.

I guess it is using the function evaluate_LAST() defined in src/libs/zbxserver/evalfunc.c

Thanks for the quick reply!! :)

[clopez]

Perhaps a way of implementing this is defining a new function in src/libs/zbxlua/zbxlua.c like get_itemid(key,host) that would return the itemid of that host:key. The parameter host would be optional and will default to the current host were the "lua script" is defined. What do you think about this?

Thanks again!

[clopez]

Hello,

I have notified that when you return a non defined variable or a nil zabbix-server crashes giving a segmentation fault:

For example. The following script
return (nondefined)
will cause a fatal error on the zabbix-server

Also if you try to return a nil this will happen

return (nil)

[clopez]

However at first sight it only seems to happen on the return statement. If you try to do an operation with an undefined variable

defined = nondefined + 2
return (defined)
It don't crashes the server, and I can see on the logs:

Lua error: [string "defined = nondefined + 2..."]:1: attempt to perform arithmetic on global 'nondefined' (a nil value)
0.000399 seconds to execute Lua script itemid:22273 script:(null)
Item [55AA-1:lua[TEMP]] error: [string "defined = nondefined + 2..."]:1: attempt to perform arithmetic on global 'nondefined' (a nil value)

[nelsonab]

Cool, thanks for the feedback. Lua assigns all variables the value of nil when they are instantiated (first used). Nil is a special type which has no value, and thus cannot be used in arithmetic.

The bug however is that I am not checking for a nil value to be returned.

I hope to have something soon to fix this.

[nelsonab]

Attached is a patch to add some error checking for nil values. This patch is for 1.8.4, and follows the patching methods mentioned previously.

[nelsonab]

Attached is the latest version for 1.8.4.

New in this version is the function get_itemid(host,key) which will return the itemid for the item associated by host:key. Both host and key must be string values.

There are also two global variables now available. zbx_host and zbx_hostid. zbx_host is a string representing the host associated with the currently executing lua script, zbx_hostid is the integer representation of the hostid for the currently executing lua script.

Thus one way to call get_itemid would be:
itemid=get_itemid(zbx_host,"net.if.in[eth1]")

It is recomended not to test this patch in a large distributed environment or an environment with many hosts. There is an issue with the transcription of long long integers (64bit) to Lua. Lua treats all numbers as a double double (64bit floating point) which allows for 52 bits of integer data. This should not be an issue for most people uless your hostids or itemids are greater than (2^53)-1. I will be working to solve this issue in a future version.

[clopez]

Hello

I have just tested the patch 1.8.4-p4-lua.patch and I confirm you that it solves the crashes related to returning nil items

zabbix_server no longer crashs and now I can see on the logs

Lua error: (null)
0.000334 seconds to execute Lua script itemid:22272 script:(null)
Item [HOST:lua[TEMP]] error: Null Value Returned

I am going to test now the -p5 patch

Thanks a lot!

[clopez]

I have tested the -p5 patch and it works like a charm

For postgresql users is need to replace the \" with \' on all DBselect() calls because postgres is more picky with that than mysql (http://wiki.postgresql.org/wiki/Things_to_find_out_about_when_moving_from_MySQL_to _PostgreSQL)

For people willing to use the script on templates note that they can reference the host where the lua script is executed with the global variable zbx_host. For example:


itemid=get_itemid(zbx_host,"TEMP")
value=get_last(itemid)
return (value)


Thanks a lot for this great patch :)

[nelsonab]

I have added the Postgres changes you mentioned int my dev code. I'm working on tackling the 64bit integer problem right now and hope to have something to post soon.

I will also try and test out loading some other Lua libraries such as the Math ones.

Further down the road I will be working to improve the get_last function (or add a new one) to retrieve and manipulate historical sets of data.

[nelsonab]

Attached is the latest version for 1.8.4, p6
See post #2 for installation instructions.

This version now supports 64bit unsigned integers. This means the earlier limitation on item and hostids should be gone. If you need to create an unsigned 64bit ingeger while within Lua use the following statement:
a=uint64(1234)
This will assign the variable a the value of 1234 which is a 64bit unsigned integer.

Also changes advised by clopez for Postgres have been included.

[clopez]

It is recomended not to test this patch in a large distributed environment or an environment with many hosts. There is an issue with the transcription of long long integers (64bit) to Lua. Lua treats all numbers as a double double (64bit floating point) which allows for 52 bits of integer data. This should not be an issue for most people uless your hostids or itemids are greater than (2^53)-1. I will be working to solve this issue in a future version.

I think that 2^53 is a very very very big number. I don't think nobody in this planet would have such crazy number of hostids/itemids. To make a easy comparative: all IPv4 addresses (the entire Internet) is only 2^32. I have been reading (http://stackoverflow.com/questions/945731/what-is-the-maximum-value-of-a-number-in-lua/947763) about the topic and seems that lua can be patched (http://luaforge.net/projects/lnum/) to allow long long integers. But I am not sure if this would be a good idea.

I have tested your new patch and it broke the Lua script that I was using on my Zabbix deployment. The script is the following:


itemid=get_itemid(zbx_host,"VOLTS")
sample=get_last(itemid)
milivolts=sample*(3300/1024)
vwc=2.97e-9*(milivolts^3) - 7.37e-6*(milivolts^2) + 6.69e-3*milivolts - 1.92
vwc=vwc*100
if vwc < 0 then vwc=0 end
return (vwc)


After reading carefully the Zabbix logs and some debugging of the code I realized that you changed the type of itemid from int to zbx_uint64_t withing the function lua_item_get_last() in the file src/libs/zbxlua/zbxlua.c but you forgot to change the checking of the parameter against zbx_uint64_t instead of the previous int. That caused the following error on Zabbix logs:
Lua error: [string "number=uint64(0)..."]:3: get_last expected to receive a number

The following patch solves this problem


--- a/src/libs/zbxlua/zbxlua.c
+++ b/src/libs/zbxlua/zbxlua.c
@@ -295,7 +295,7 @@ int lua_item_get_last(lua_State* L)
luaL_error(L,"One or more arguments expected");
}

- if (0==lua_isnumber(L,-1))
+ if (0==is_lua_uint64(L,-1))
{
luaL_error(L,"%s expected to receive a number","get_last");
}


After applying it my script seems to be working back :)

[nelsonab]

Dude, you freaking rock! I'll integrate and post back a little later.

The main reason for creating the 64 bit unsigned integers was because of distributed monitoring. Distributed monitoring uses the first few digits to denote the node, thus if you have only 20 digits available, and the first 4 of which are taken up by the node id it is theoretically possible that you could hit the 52bit boundary for Lua numbers. And I do hope to improve the chances of getting this included into the mainline one day so to do so I need to fix these design bugs as much as possible.

Well time for the "odometer" to roll over... this makes post #1,000. :-)

[nelsonab]

Attached is Lua patch number 7 for Zabbix 1.8.4.

Feature added:
Parameter passing to Lua scripts
- All items inside the brackets [] are now passed to the Lua script as parametetrs. They can be found in the global variable ARGC and ARGV. ARGC is the argument count, and ARGV is an array (Lua Table) of the arguments. The base index for ARGV is 1.

For example:
key: lua[one,two,three,4,five]
script: return ARGV[3]

Would return "three"

If the above script were changed to "return ARGV[6]" the item would become unsupported as there is no 6th value in the array. Specifically Lua returns a nil value, but Zabbix cannot handle a nil value and thus it is marked as unsupported.

[nelsonab]

I have also been working on updating the Documentation page at:
http://trac.red-tux.net/wiki/lua (http://trac.red-tux.net/wiki/ZabbixLua)

[Yello]

Hi,
Is there a 1.8.5 version of this patch?


Regards,
David

[nelsonab]

Not yet, but I'm assuming this means there is an interest so I'll add it to my list. I'm working to get a new version of Zabcon pushed out first so when that's done I'll get to work on this.

[Yello]

Not yet, but I'm assuming this means there is an interest...

I think this would be a powerful enhancment to zabbix so I'd urge you to press on regardless. Eventually, the penny will drop with those not showing any interest if the end result is decent and the utility clearly explained.


Regards,
David

[nelsonab]

Don't worry, when people show interest and use the various patches/utilities I have written it motivates me to improve them. Right now I'm working on pushing out an update to Zabcon. I'm hoping to have that pushed out within the next week.

[odium4u]

Would love to see an update to this.

Nice work.. keep it up.

:)

[nelsonab]

Would love to see an update to this.

Nice work.. keep it up.

:)

One is in the works for 1.8.5 right now. I need to update the patch in a sequential manner due to the changes in the Zabbix code between versions.

[nelsonab]

I have just completed and uploaded the latest patch for version 1.8.5.

The following link contains links to the patch file along with instructions and information about the patch.

http://trac.red-tux.net/wiki/lua

[ZHANG FELIX]

Through your conversation, I learned a lot,IBM Batteria (http://www.r4-ds-dsi.it/batteria-notebook/ibm-batteria-portatile.html)COMPAQ Batteria (http://www.r4-ds-dsi.it/batteria-notebook/compaq-batteria-portatile.html) it seems here to learn a lot of knowledge is not learned from books, Thank you seniors!

[mihu]

Since this seems to be a only way to contact code gatekeepers, here it comes.

I did a rework of previously posted Lua "patch". Current version can be found at
https://github.com/mihu/zabbix-lua/

This version is based against zabbix svn trunk. The short info about code status can be found here https://github.com/mihu/zabbix-lua/tree/lua-dev/src/libs/zbxlua

The feedback is welcome.