Standing up Zabbix in AWS

[w00tburger]

Hi all. I am entering a PoC to bring Zabbix to AWS utilizing an RDS backend. I have done a handful of reading around The Google to find that it has left me with more questions than answers.

First, I am reading a lot of users are still using Zabbix 3.0 vs the latest (not in development) I see the latest production version 3.4 is not LTS, does it have any plans on supporting 3.4 for LTS, or are they skipping straight to 4.0 which is in development?

Second, I am reading a lot of users that are having problems with performance while utilizing an AWS RDS backend. A lot of them seem to be in regards to its performance during its backup windows when all the Zabbix agents become unavailable, and the Admins are flying blind as to what is happening during this backup window. Can anyone enlighten me on some of the cons, or do they have an positive experience using Zabbix with a RDS backend? We will be looking to support ~600 servers with this solution.

A few of my concerns are that we implement Zabbix 3.2 for the LTS support, and then by the time we migrate fully into AWS it will be time for an overhaul already. I would like to plan for four years out of possible. Lastly, performance on an RDS instance will be huge. My PoC for this will only be 20 to 30 systems at the most, it would be impossible for all 600. Any future insight into possible blockers or potential problems would be appreciated greatly.

Thanks for your time all. Looks like you have a pretty strong community here.

[vesper1978]

Zabbix 4.0 will be the next LTS. The Zabbix Life Cycle changed with Zabbix 3.0 and up so that the x.0 releases are the LTS versions.

https://www.zabbix.com/life_cycle_and_release_policy

RDS should be fine for Zabbix. Just as with setting up your own DB server, be sure to size it properly. With any database solution, you will lose performance during backups. So be sure that you factor in your backups and how much of a performance impact taking a backup will make.

For instance, I run a 3 node MySQL 5.6 Galera Cluster for my Zabbix DB server.
They are m4.2xl instances (8CPU+32GB ). My DB is currently 100GB in size, and I monitor ~150 items per second over ~200 hosts. When my full backup fires off, my iops spike to 1200. However, my monitoring is not affected as the disks I've provisioned are more than able to handle it.

I also have RDS instances in AWS for non-Zabbix usage, and same with them, I've provisioned a beefy instance with dedicated iops so that when it snapshots, I don't have an app performance issues.

You're going to want to over provision your RDS instance, as well as the disk. It's better to have too many resources than not enough. There's no such thing as "just enough".

[w00tburger]

Reposted below

[w00tburger]

Does anyone happen to have a hard date as to the release other than Q1? I am literally at the point where I am standing my PoC up in our AWS using Zabbix 3.0 since its the current LTS. I would rather wait on my PoC if I know that Zabbix 4 is going to be out within the next month or so (or at the very least, use the alpha as our PoC), rather than going through the headache of deploying 3, just to be upgrading once the AWS migration is complete.

Also, I had been working with the auto-discovery with Zabbix and was curious, is there still no way to automatically de-register a host if it scales down in AWS, or will that require custom scripts on the servers and an interaction with the Zabbix API?

[vesper1978]

There's no hard date. Considering 4.0.0 alpha2 was only released in January, and there have been no betas released yet, there's probably a few more months of waiting.

[ik_zelf]

when scaling is needed, you might want to take a look at cockroachDB for the backend database. Zabbix needs a few minor tweaks te be able to use it. Also, when a dbnode crashes, in a 3 node cluster, the rest keeps your application working without interuption. As a matter of fact, as long as more than half of the db nodes is working, the db remains available.

[kloczek]

Does anyone have experience using Zabbix for a webapp deployed with Elastic Beanstalk? Zabbix is a great tool when you have complete control over the servers, but I don't know how to manage it in an auto-scaled environment like Elastic Beanstalk where EC2 instances come and go. Can someone point me in the right direction?

Do not waste a time on elastic search backed. use SQL one.

[jan.garaj]

Do not waste a time on elastic search backed. use SQL one.

Elastic Beanstalk is not Elasticsearch.

Zabbix Server doesn't scale horizontally very well by design. You can deploy to the cloud (AWS in your case), but it must configured as a classic app.
IMHO AWS EKS/ECS will be better option for Zabbix infrastructure.

Inspiration: https://github.com/ThomasVdBerge/zab...cker-terraform

[William138]

They are m4.2xl instances (8CPU+32GB ). My DB is currently 100GB in size, and I monitor ~150 items per second over ~200 hosts. When my full backup fires off, my iops spike to 1200. However, my monitoring is not affected as the disks I've provisioned are more than able to handle it.

I also have RDS instances in AWS for non-Zabbix usage, and same with them, I've provisioned a beefy instance with dedicated iops so that when it snapshots, I don't have an app performance issues.

You're going to want to over provision your RDS instance, as well as the disk. It's better to have too many resources than not enough. There's no such thing as "just enough".

[LinuxEngi]

I am actually in the midst of switching from a MariaDB Backed EC2 instance (Zabbix server running on EC2 t2.medium, with MariaDB running on same instance) to a Single Node MySQL compatible Aurora Cluster (aurora.t2.small) and scaling the host running Zabbix Server down to a t2.small.

Our stats are

Number of hosts (enabled/disabled/templates)	188	121 / 6 / 61
Number of items (enabled/disabled/not supported)	7735	7106 / 338 / 291
Number of triggers (enabled/disabled [problem/ok])	2255	2215 / 40 [54 / 2161]
Number of users (online)	13	2
Required server performance, new values per second	94.29

The only issue we have ever ran into was storage on the instance, just recently but that was due to me forgetting that I temporarily disabled housekeeper and it became not so temporary.

The motivation of switching to RDS backed DB was so if the DB went down the instance was still available and vice-versa; and you really shouldn't monitor the same stack within the stack (we use Zabbix primarily to monitor our other EC2 instances and some on-prem infrastructure.)

Will update on how the stack works once implemented.

[petersmith69]

this is same issue i am facing. i thought that this is because the versions are not updated yet and they might release later.Hope this is helpful to you and do tell me is your issue is resolved.

[kloczek]

Using zabbix on AWS is not a problem.
This still the same zabbix.
Database backemd is not a problem as well. Exact types of RDSess are still MySQL, PostgreSQL. Oracle with some minor differences/limitations (for example in case MySQL is not possible to control replication to the level which is possible in case non-RDS).
In other words on using zabbix on AWS all current knowledge, methodologies etc are still almost 100% relevant.