How to prevent zabbix re-reading log repeatedly

[trong.nguyen]

I have the same issue when monitoring MT4 log with zabbix 5.0 LTS (I have CentOS 6 server so can not use zabbix 6.0), I found that the mechanism for writing MT4 log file is quite special, the MT4 log file is written continuously so the mtime of log file always changes, however the size of log file does not change. So it always have this log when I checked zabbix_agentd.log:

30931:20230707:155822.971 the modification time of log file "/window/MetaTrader4Server/logs/20230707.log" has been updated without changing its size, try checking again later
30931:20230707:155914.022 after changing modification time the size of log file "/window/mt4reportserver/20230707.report.log" still has not been updated, consider it to be a new file

This make zabbix re-read MT4 log file repeatedly and make noise for my team.

I've tried this option in logrt[] function:

mtime-noreread - non-unique records, reread only if the size changes (ignore modification time change).

But when log file is written continuously, zabbix does not show anything because the file size does not change.

Any one knows how to prevent zabbix re-reading log file, please help me!!!!!

Thanks so much!!!

[ISiroshtan]

Based on your description it sounds like really weird way to produce logs. So whenever max file size is reached instead of rotating the whole log file it removes few lines from start to add few new lines at the end, is my understanding correct?

If that is the case I'm really not sure it's possible to handle such weird log with Zabbix default tools. If you reference documentation about log file monitoring, following important notes are outlined:

• The agent also internally uses inode numbers (on UNIX/GNU/Linux), file indexes (on Microsoft Windows) and MD5 sums of the first 512 log file bytes for improving decisions when logfiles get truncated and rotated.
• The agent starts reading the log file from the point it stopped the previous time.
• The number of bytes already analyzed (the size counter) and last modification time (the time counter) are stored in the Zabbix database and are sent to the agent to make sure the agent starts reading the log file from this point in cases when the agent is just started or has received items which were previously disabled or not supported.

So if the start of the file is changed (few lines removed to be able to add more lines in the end) - Zabbix treats it as a new file. Even if it would not, the data in file would be changed and the place indicated by number of bytes analyzed would actually point to a different data, making totall mess of log file monitoring.

I can think of two ways to approach this situation:
1. Check if you can make program(MT4) to output logs in more traditional way (like produce logs with no limits and have some kind of log rotation mechanism to swap them on set intervals or rotate logs on reaching certain size limit... pretty much what linux logrotate utility does)
2. Write your own log parser that would find specific timestamp (if log has timestamps that is) and processes all new entries after, saving the new timestamp it processed last in some permanent storage. And if any lines matches what you need - send it via Zabbix sender into Zabbix trapper item to raise the alert.

[Peiler]

Hi, did you have any update on this. From my findings for the same issue (MT4 Logs) MT4 creates an initial daily log file with filesize of 4096kb and it starts filling that one. So until log file size reaches over 4096kb or day ends, zabbix keeps reading the same logs as it considers it as new file since date modified changes but size remains the same.