Trying to figure out if zabbix is right for us

[Jason M]

Hello all,

I work for an ISP with a large geographical area but not a huge number of nodes. We are monitoring 4500 nodes of network equipment (cisco, cambium, ubiquiti) via ping and snmp via a large private wan. We monitor customers separately, so this is just for the network devices, basically anything above the customer. Currently we pay for SolarWinds Orion, but we feel the dependency system is lacking for our purpose. It is customizable but extremely cumbersome.

Our big issue is implementing dependencies with our network topology. It is our opinion that SolarWinds is better suited for an enterprise environment like a large building or well defined star structure. Instead we have dual-rings leading to dual-uplink-stars leading to dual-rings leading to stars. When the structure changes like extra trunks, updating dependencies is terrible. I personally haven't had to manage it, but they all have the same complaint enough they asked me to look at other solutions. We need either better dependencies, or if not better, at least easier to manage.

If there is an indication Zabbix is good for our purpose, then I have somewhat of a plan. We have about 20 top level rings, and the newest ring is mixed gear which is great, and it has a bunch of rings inside it which is also great, so it will be a perfect demo. I'd like to set up a central zabbix server and at least one remote poller to install on that ring. Possibly one more poller on one of the inner rings. I hear Zabbix discovery is decent, so I would like to try that. It is also an option to create tools and batch jobs that read from Orion and push to Zabbix. Things like note fields, addresses, gps coordinates, etc.

I emailed Zabbix sales for a demo, but they haven't gotten back to me, and also no one answered the new york phone number when I called. Can anyone here can speak directly to our dependency concerns? Or if not directly to our concerns, then at least to dependencies generally? I tried to research it but without having used Zabbix I find the answers hard to digest. It would be great if I could arrange a demo for my engineer that understands the SolarWinds problem.

Any advice, opinions, and links appreciated

--Jason Miller

[Jason M]

Just checking in but no replies lol. I don't blame you though. That post was a lot.

Since then:

1. I have set up a zabbix VM and did network discovery against the dev network using icmp, and it worked great. I showed it to engineering and they're interested, so I'm being allocated a dev vm on the production wan next week in order to try network discovery against that one ring that I described using snmp so I can get into temperatures and capacity-vs-demand on the network.
2. I've explored the trigger system a bit and I think I understand it. I like the function language for triggers such as (pseudocode) avg($cpu,10m)>80% and then then avg($cpu,30m)<40% to avoid flapping. Brilliant!
3. I was not able to get the proxy working, but I think I know what I did wrong. I'll try again on the dev wan since I have to wait. I asked for 3 vms, maria+zabbix+proxy, but I'll see what they give me.
4. I also tested the API a bit, just user.login, host.get, trigger.get, and user.logout. I see methods to push dependencies as an array during a node update, so that looks extremely promising. The whole API looks great honestly, which is one of the things I hoped for when I found out how mature zabbix is. I'm very happy to see this. Is there a swagger implementation for it?
5. I haven't explored the trigger actions yet. I'm hoping to be able to execute http post or similar api level interactions of some kind in addition to the usual email and queue.
6. I haven't explored reporting yet at all.
7. I haven't explored SLA at all.

So any advice on professional service from Zabbix, or do you use outside consultants?

[tim.mooney]

The existence and coverage of the API was one of the reasons why I chose Zabbix many years ago for my workplace. I'm not familiar with swagger, so no idea if that's a thing for the API.

My environment is a lot less complex than yours, so I didn't feel the need for professional services when I originally deployed long ago. I did spend a lot of time reading the docs and playing around with a test install, though, so I felt like I had a good handle on important concepts (templates, macros, etc) before I deployed.

I'm at a university, though, so we get a little better pricing on support, so we have found value in having support. I've only used it may 6 or 8 times over the years, but it's generally been quite good.

Proper network topology dependencies are one of those things that can really make or break a monitoring deployment. We typically have the "nearest" dependency set at the template level, so when "Datacenter B" gets applied to a host, that automatically encodes the ingress/egress network device for that datacenter as a dependency for the base monitors on a host. That has worked well for our smaller environment.

The only real advice I can offer for your install is to check your network dependencies as frequently as possible (short update interval on whatever 'item' you collect, whether it's ping, SNMP, or whatever). You can use a longer update interval for non-topology dependencies, if that makes sense for your environment, but the topology dependencies should be checked frequently. That will reduce (but not completely eliminate) the chance of a trigger failing on a leaf device, when the actual problem was that the topology dependency failed between the time when it was last checked and when the leaf node was polled.

[Jason M]

Agreed. The more I look at the API the more I like it.

Swagger is a kind of universal front-end product that functions as a rest api client. You may have even seen it and not knew you were looking at it. I think it reads the wsdl or something and auto-generates forms that you then customize. The forms it generates actually execute live against the API, very handy. Or maybe you customize the api details with notes and such and then republish it. I'm not completely clear on that part, but we use products like Calix SMX, Ciena MCP, effectively competitors of Zabbix that have Swagger implementations.

Here I found a decent example. The color represents the HTTP verb, and when you click one of the methods, it opens and you get a form with datatypes and defaults, etc. (green has been clicked and opened)

https://www.novatec-gmbh.de/wp-content/uploads/2017/03/swagger-ui.jpg

I think it's basically just a small web app with an xml file describing your API. I assume the other tricky part is programming it to understand your authentication method. I did some google for it, and I cant find a zabbix swagger, so I'm assuming it doesn't exist.

For such a mature API it would be an amazing feature. But maybe there's is something else? Is there some kind of zabbix api browser? I went straight from docs to test code. Is there a tool somewhere I should use? In between there is where Swagger or similar would fit in.

[Markku]

All Swagger/OpenAPI thingies I've seen have been with REST APIs, and Zabbix is using JSON-RPC. Syntactically they are very different, but the basic idea is eventually the same: an HTTP call triggers some processing in the server and a response is returned.

I'm also not aware of any API browser for Zabbix. Personally I use the API with Python and Pyzabbix package, it's quite convenient for live testing as well. But having an online API browser would be worth enhancement request in https://support.zabbix.com (unless there already is an issue for that).

Markku

[Jason M]

Oh gotcha. I noticed that json-rpc but I couldn't tell what the difference was since it seemed functionally identical. So no wsdl then? Oh I see and the variables are mostly in the payload instead of /{id}/ syntax. I didn't need to use that for testing so I didn't even look. Thank you, that clears it up.

[Markku]

Oh I see and the variables are mostly in the payload instead of /{id}/ syntax.

Yeah that's correct, the URL for Zabbix API calls is always the same (https://server/zabbix/api_jsonrpc.php), all the parameters are in the JSON body.

Markku