Zabbix 3.4 (4.0) in AWS with 2k-3k NVPS aiming 4k-5k

[Xalberg]

Greetings,

Could someone, please, give me some suggestions on the following topic.
I need to migrate our current implementation to some cloud because performance is greatly degraded and load (NVPS) expects to grow further.
Just a quick review on the current solution:
- 450k items. (planned 800k - 1M);
- 90 hosts (planned 180-200);
- Current values processed per second are 2k (3k at peaks every 5 minutes);
Current NVPS is now showing correct because majority of the checks are passive (SSH Agents). Yes, This is intended.

One of the critical points - we use Grafana to visualize a lot of stuff. And this hits IOPS greatly.

Now I was looking on AWS and what it could suggest.
And I'm really stuck.
I was thinking about i3.2xlarge instance (8vCPU, 60GB, 2TB NVME - 12k IOPS).
But it turned out that NVME will reset on instance restart, so this solution is not redundant.
Selecting EBS as an io1 type with 10k guaranteed IOPS and 2TB space turned out to be a bit expensive.

To me understanding and please correct me if I'm wrong, I should aim ~ 10k IOPS with my solution.
I would like to omit RDS instance and have one EC2 doing all the stuff. (there will be more instances like this in future). But if RDS is something that is better to do - please, let me know.

In general I would like to hear recommendations, thoughts on this situation.
If someone can share technical recommendations with instance types and best approach - that will be great.

Thank you.

[kloczek]

None of the cloud solutions can solve low latency IOs read issues because your problem is not related to number of IO/s but latency of such IOs.
None of the cloud infrastructures should be used to maintain such workloads. Cloud based based platforms always have some limitations.
If you will look into the descriptions of all RDS and EC2 instances none of them guarantees you exact IO latency.

[Xalberg]

Hello, kloczek,

That's something I haven't thought about.
Still there are options, I assume it might be possible to overcome IO latency with 'metal' instances where you have SSDs directly connected.
In any case having personal private cloud is not an option unfortunately.
So we will have to try something in any case.

Thank you for your reply.

[kloczek]

Hello, kloczek,

That's something I haven't thought about.
Still there are options, I assume it might be possible to overcome IO latency with 'metal' instances where you have SSDs directly connected.
In any case having personal private cloud is not an option unfortunately.
So we will have to try something in any case.

As long as you are allocation EC2 instances which are VMs only virtualization adds own latency to storage or network IOs.
IIRC AWS offers as well dedicated bare metal instances but they are no cheap.
Nevertheless scale few k NVPS it is not rocket science. Are you using EC2 instance to maintain DB backend or AWS RDS ones?

[Xalberg]

Are you using EC2 instance to maintain DB backend or AWS RDS ones?

Planning to use a single EC2 instance for everything in order to easier maintain it in future.
Some installations are not that "big" and RDS is definitely not needed there.
The plan is to have a set of instances of different volume, depending on the amount of hosts there.
So I just stumbled over the biggest one with 1M of items in future in terms of resources.

[doctorbal82]

So I just stumbled over the biggest one with 1M of items in future in terms of resources.

That's a juicy stumble. Where's the reference? :-)