Charts page fails to load.

[lordeldor]

I have an installations with a fairly large number of graphs. (Currently 35964)

With the default configuration of "Dropdown first entry" being set to all the graphs page fails to load. If I go to another page, and select a host, then the graphs page will load with that host selected. Choosing all from the hosts dropdown causes the same page load failure again.

At first the php memory_limit was set fairly low. Upon inspecting the apache logs it indicated that the php process was running out of memory. I increased the value incrementally until the out of memory warning stopped, but the page still does not load unless I have selected a host on another page first.

My php memory_limit is currently set to 2048M.

Has anyone else encountered this issue? Is there a way to solve this other than setting the "Dropdown first entry" to none?

We are running zabbix version 3.0.3

Thank you

[lordeldor]

I take it nobody who has viewed this so far has encountered this issue before...

Let me drop some more detail, and bump the thread so it can hopefully get more eyes.

I am running my zabbix installation on ubuntu 14.04 using the packages provided at:

http://repo.zabbix.com/zabbix/3.0/ubuntu/

package versions: 3.0.3-1+trusty

I have a split tier installation.

2 web servers behind load balancers
2 zabbix servers in an active/passive configuration
2 database servers in an active/passive configuration
a few proxy servers.

With the php memory_limit set at 2GB I am seeing interesting behavior from the apache forks that are trying to serve the problem requests. After the page fails to load on the client side the apache process continues to consume memory up to just below the 2GB limit and then hangs there for a while before exiting. I assume the thread hangs out because of keepalive settings, but I am uncertain why it consumes so much memory.

My browser will attempt to load the page again upon failure, so a few of these processes can get stacked up.



If I am not careful I can run the server out of memory completely, and then processes start getting killed. This can happen easily with a number of users trying to access the graphs page with all host groups and all hosts.

When I look into the functionality of the DB server I see a healthy functional system. So I have my doubts that the issue lies there, and I suspect the apache configuration.

I turned on the DB server query log and captured the queries that were being executed on the backend during the graphs page load. The biggest, nastiest query in the bunch was the list of graph names.

SELECT  DISTINCT  g.graphid,g.name FROM graphs g,graphs_items gi,items i,hosts_groups hg,hosts h WHERE hg.groupid='21' AND hg.hostid=i.hostid AND gi.graphid=g.graphid AND i.itemid=gi.itemid AND g.graphid=gi.graphid AND h.hostid=i.hostid AND h.status<>3 AND g.flags IN ('0','4');

35964 rows in set (2.97 sec)

Not bad at all.
All of the other queries run in much less time.

But in the end this is the result I get when I try to load the graphs page with all host groups and all hosts.



There is no additional info if I click details. The client is just reporting empty response.

I can work around this issue by selecting a host on the triggers page and then going to graphs, but I would prefer not to have to do that.

Anyone ever run into anything like this before?

[lordeldor]

I'm still trying to get to the bottom of this issue, but not having much luck.

As a test I decided to take apache out of the equation. So I installed an nginx instance to see if I could resolve this issue.

Sadly the issue seems to be within php so the switch to nginx didn't earn me anything.



Has anyone ever experienced this?