LLD rule processing syslog

[Brambo]

Hi there,

I'm relative new to zabbix but I was managing
I have setup a syslog on the zabbix server and when creating an item to retrieve this file using agent active the content is shown in latest data
Item key then is: log[/var/log/syslog/{$SYSLOGSENDERIP}/messages.log]
The SyslogsenderIP is host macro with the IP adres of the device we expect data from.
So far so good.
With pre-processing steps regex and second step csv -to Json I get a JSON output with 41 fields
The field I expect to use are:
So I setup a Discovery rule Zabbix Agent active
Key like mentioned above, and the preprocessing rule i used for the item.
These steps are all accepted.
Then go to Item prototypes.

Here is where I run into problems
When I choose depending item I can't select my discovery rule AND in the item key I can't used my Macro's.

Could someone push me in the right direction? I tried following this guide: https://smartsolutions.dioss.com/en/...ry-json-lists/ and I can show the last update of the file using the guide but that is nice to have for sure but the Item prototypes I want to create are more important.
Using zabbix 6.4 and I also have a 6.0 available but there is no syslogger active at the moment.

[Brambo]

As extra information the current setup in the 6.4.4, until written otherwise the mentioned steps are working:



^^ i fixed the } bracket of the dirname macro
(above is part of the LLD get files)


Both working and in the collect data the data I want to collect is shown in latest data.

Only the part I want to do know is from that collected data create Items (the data is converted to json like mentioned in the start post)
I can do (@ item prototype) select dependent item > select prototype == Collect data {#FILENAME} but then I'm missing macro's to create items based on the syslog file.

Good to know is that this syslog file contains details of multiple hosts which communicates to this central module by an encrypted protocol, and with this syslog output I'm able to receive it decrypted.

[ISiroshtan]

Hey there.

So in my brain your post ends up all over the place and I sadly fail to grasp the end goal.

Still, I can outline points that dont fit in overal puzzle for me, you can clarify them, and then maybe me or some one else will be able to advise.

First of all, aim of Zabbix discovery is to find semi-static entries that share same characteristic and automatically set them up for monitoring based on common monitoring setups. If you pipe out content of log file as json into discovery, Zabbix will attempt to setup a monitoring of each log entry. Which I'm pretty sure not something you want to do neither is your aim

If you want to monitor a log file content, you dont need to setup a discovery. Instead you need to identify clear list of either log entries you are hunting after or at least their common characteristics. Like 'error' log level. Or 'high' priority. Etc. Than you can use trigger with find function to see if the latest log entry contains the values that warrants trigger firing.


Also, depending on amount of logs shipped, it might be not good idea to pull all of them into Zabbix DB. You might want to right away filter out unneeded entries on stage of log file processing (log item supports filtering, so that agents only sends matching entries into Zabbix for storing in DB and processing for possible triggers, can check here).

[Brambo]

Ok I try to summarize it better:
First we have a syslog output of a device which report all kinds of information of other device. This could be 10 devices, could be 100.
Only a few of those devices have static IP's, others have a fixed hostname (which can't be edited) I have identified the important field and {#ORIGIN} is always the device which reported the fault to the syslog device and based on the name/IP there I would like to create an Item.
When an error occurs the LOGID of the fault is unique and the clear event (Clear / error / critical are based on {#LEVEL} ) has the same LOGID.

So the combination of ORIGIN-LOGID ($.6-$.1)should be a unique key and LEVEL is the indicator if it's an active alert or clear.
After preprocessing and as last step CVS to json the output of messages.log is: (I could replace the original : to different character before doing csv to json but that is not the example output, pre-processing already drops about 2/3 of all messages)

[{"1":"LogId:3XoeCeGQAA","2":"Priority:",
"3":"Origin","4":"","5":"CallId:","6":"address:10.1.1.20",
"7":"UsrId:","8":"Time:2023-07-11 09.34.07","9":"","10":"Receiver:1","11":"",
"12":"Cald:","13":"address:10.1.1.25","14":"UserId:",
"15":"Time:2023-07-11 09.34.07","16":"Status:200","17":"","18":"Receiver:1",
"19":"","20":"CallId:","21":"address:10.1.1.25/error","22":"UserId:",
"23":"Time:2023-07-11 09.34.07","24":"Status:200","25":"","26":":",
"27":"Application:\"MOD:[email protected]\"","28":"Hostname: ",
"29":"name:Zabbix 1","30":":","31":"Level:0","32":"Pers:yes",
"33":"Description:No error","34":"Fault code:300","35":":","36":":",
"37":":","38":":","39":"detailed info: ","40":""}]​

The reason for doing it dynamic (or at least try ) is that system can grow/ units can be replace etc. And I'm willing to setup it up completely different but I'm lacking knowledge on how to do it.

[Brambo]

The state I'm now at is:
In a template I have an item which collect / processes the syslog data . Output is like following:

2023-07-13 17:29:55	{"data":{"LOGID":"3XoeCfuggf+ySZiTQAA","PRIORIT Y": "","ORIGIN":"10.30.177.120/SystemSurvey","TIME":"2023-07-13 16.29.54","APPADDR":"10.30.177.89","HOSTNAME":" ","LEVEL":"0","FAULTCODE":"0-0-0","DESCSHORT":"","DESCLONG":" "}}
2023-07-13 17:29:41	{"data":{"LOGID":"3XoeCfuggf+ySZcTQAA","PRIORIT Y": "","ORIGIN":"10.30.177.120/SystemSurvey","TIME":"2023-07-13 16.29.41","APPADDR":"10.30.177.89","HOSTNAME":" ","LEVEL":"4","FAULTCODE":"3-3-19","DESCSHORT":"IP equipment lost","DESCLONG":""}}
2023-07-13 16:44:45	{"data":{"LOGID":"3XoeCfuggf+ySacTAAA","PRIORIT Y": "","ORIGIN":"10.30.177.120/SystemSurvey","TIME":"2023-07-13 15.44.44","APPADDR":"10.30.177.89","HOSTNAME":" ","LEVEL":"0","FAULTCODE":"0-0-0","DESCSHORT":"","DESCLONG":" "}}
2023-07-13 16:41:48	{"data":{"LOGID":"3XoeCfuggf+ySaMTAAA","PRIORIT Y": "","ORIGIN":"10.30.177.120/SystemSurvey","TIME":"2023-07-13 15.41.48","APPADDR":"10.30.177.89","HOSTNAME":" ","LEVEL":"4","FAULTCODE":"3-3-19","DESCSHORT":"IP equipment lost","DESCLONG":""}}

^^ about is twice the same alarm and clear, the logID is an unique identifier, APPADDR is the item I would like to create with the discovery rule.
The discovery rule > dependent ITEM on the one which output the above text, key syslog and the same MACROs as in the first post. However the Jsonpath a bit different.

Item Prototype:


However I still don't get any Item. When i test the preprocessing steps with the JSON output above I do get the IP adress without /SystemSurvey