Zabbix Hardware and architecture recommendation

[alaahil]

Hello everyone,

I am trying to setup monitoring over a high number of machines but I am not sure what is the best way to proceed. The situation is as follows:
a) We have around 100 different physical locations and there are around 30 devices that need to be monitored over each location
b) for each device there are at least 20 items that we need to be monitored
c) there are at least 5 items on each device that will be monitored through powershell scripts
d) We need to have fail over mechanism to be implemented

I have done some research but I could not find a comprehensive guideline on how to go implement so I thought I would write the different information that I have read and would appreciate any potential input I would get to correct the info:
1- We will need to deploy a proxy for each location, the proxy hardware requirements will not be very high as the load is low per each location
2- Install the zabbix servers and the databases on separate machines, recommended 3 servers and 3 databases to implement a failover cluster for each
3- 64 GB RAM will be required for the zabbix server
4- The bottleneck will be in the database.

Would like to know if the above approach (even though described briefly) would be suitable for our setup, and if the above makes sense what would the recommendations be related to the database being the bottleneck?
Regarding failover clustering, would it be recommended to use the standard zabbix HA configuration? or would it be advised to use something like pacemaker while maintaining same IP?

[LenR]

1) Correct
2) I would recommend researching postgres and timescaledb. You want to avoid Zabbix housekeeping on the item data either with mysql partitioning or timescaledb. We weren't that large, but ran the Zabbix server and mysql on the same server, with web frontend on a seperate server. We deployed many years ago (about Zabbix 1.8) so mysql partitioning was the best practice then. We added a lot of ram for DB buffer pools and exploit hugepages. Zabbix recommends SSD, but we were VM's for the past few generations and disk I/O was "OK", depending on the backend technology.

Your performance is mostly influenced by new values per second, collecting 20 items per device every 15 seconds is more costly than collecting 20 items per device every 5 minutes. Adjust the item frequency in a sensible manner, for example disk usage needs to be frequent, total disk size doesn't.

[alaahil]

Thank you for your input it is highly appeciated. Just couple more clarifications:
1- Would you recommend putting the DB and the server on separate machines and should we implement failover clusters on each?
2- In the case where we will not be retaining results for items. Would that be a major enhancement for the DB performance?

[LenR]

#1 We were not HA for either mysql or Zabbix, the single server was acceptable performance. I looked at the HA design, but we didn't choose to implement it, from our past outages, it wouldn't have helped much, so it wasn't worth it. I think the design is to build a "reliable" DB cluster, then Zabbix HA servers all refer to that HA DB. In that design, putting the DB on the Zabbix probably wouldn't work. Putting the Web server on the Zabbix server would probably be OK.


See:

https://assets.zabbix.com/files/events/conference_china_2019/Edmunds_Vesmanis-Zabbix_HA_cluster_setups.pdf

https://www.reddit.com/r/zabbix/comm...qlmariadb_and/

#2 if items are monitored, they are retained for some interval, I don't really understand that question. Retaining items for different intervals relies on Zabbix housekeeping, which in large clusters is bad. Partitioning (and I think timescaledb) treat all items with the same retention, for example, after 30 days, we just delete the partition, we don't process individual items.

[alaahil]

Thanks a lot this was really helpful

[alaahil]

Hello,

I have one question, would you advise with or against the use of load balancers with zabbix?

Regards,

[cyber]

Hello,

I have one question, would you advise with or against the use of load balancers with zabbix?

Regards,

Where would you use them? if you have 1 active server at any time .. one active GUI, one DB... Where would you balance anything?

[alaahil]

Where would you use them? if you have 1 active server at any time .. one active GUI, one DB... Where would you balance anything?

My mistake..