A question about gaps in data for graphing

[Yekim]

Is it possible to change how Zabbix interacts with gaps in space of data? Whether it predicts linearly the missing data, or puts a gap in the graph, or just keeps a horizontal line until the next data point?

Ideally I would like it to keep a horizontal line until a new data-point is found, I'm uploading a lot of historical data into a Zabbix trapper, and to speed up the process I want to eliminate data points that don't change, however if the line predicts linearly it will give me incorrect data. So I was wondering if there is any way to change how the Zabbix graphing predicts in between data-points.

[kloczek]

Usually, gaps in monitoring data are first signs that you just reached the scale of the zabbix stack in which you should stop using the passive agent and switch to active one.

[Yekim]

I have log files on a 1 minute scale for the last 7 years about 120 different servers, I'm trying to get this all uploaded into Zabbix so we can look back at the data in nice graph form. I'm trying to reduce the upload speed of this massive project by not inserting duplicate entries of values where the value doesn't change from the previous. That is why I'm curious about how Zabbix would handle it on a graphing perspective, if there was no new data added for those points. Going forward we will be using an active agent.

[kloczek]

Slow changes in data series can be handled on DB engine side for example by altering table to use row compression.
https://dev.mysql.com/doc/refman/5.7...ion-usage.html
Even more effective by use storage engine which is using columnar compression.
If compression may give you at least 50% reduction of the physical space than columnar compression may give you in very slow changing data series even 1:20-30 compression ratio.
Another indirect solution is to use zfs with transparent compression with large zfs record size (max can be 1MB) which is more effective than altering tables to use compressed rows (because it compresses only rows data). As altering history tables may reduce tables by at least 20% usually zfs gzip-1 compression gives at least 3-4 times compression ratio (reduces physical space by more than 60-75% only using gzip-1 and with lz4 which should be even better to compress slow changing data physical space reduction should be greater).

Compressing history_log table 7y raw data should give you a lot free disk space.

In case of not so extreme when is not needed so long period of keeping raw data it is possible tu use another approach.
If someone really temporary struggling with available disk space you may consider another solution which you may consider is reducing history window size.
Each item has own "history period". You may go to Administration -> General -> Housekeeping -> History -> "Override item history period" to overwrite globally all metrics own raw history data period globally.
If you have already implemented history tables partitioning and if you are using, for example, daily partitions you don't need to change those setting .. just change number of history partitions.

It is yet another "side effect" of using globally setup "history period".
Using exactly the same period makes predictable real disk space necessary to store some monitoring data using only NVPS.
Any variations in real disk space will come only from avg each day string size stored in history_{log,text,str) tables.

IMO the best method compressing zabbix data is use zfs compression because zfs ARC (Adaptive Reclaim cache) caches in memory hold compressed zfs records which means that it not only reduces storage needs but as well RAM footprint necessary to have high hits/misses ration by almost compression ration. It allows, for example, stay way longer on the same hardware if for example by daily volume of the monitoring data will be necessary to have more than 128GB RFAM (usually it is kind of threshold between cheap and more expensive HW)

In other words: none of those solutions need to interpolate any monitoring data. Because those methods are already quite often used in many zabbix stacks there is no risk related to implementing such solutions .. compare to whatever you may be thinking that is Better(tm).
Another conclusion: your problem is not monitoring data interpolation but lack of disk space. You should tell about this you line manager .. storage today is quite cheap.