Web Front end having performance issues

[tmroberts]

We are starting to see experience some issues with the web front end of Zabbix server. First the set up:
2 load balanced web servers running on CentOS (aws based)
1 zabbix server running on CentOS (in aws)

From time to time the page response times seem to get very sluggish. I have added Apache monitoring to see if I can catch something in the act and nothing. I have added a bunch of web scenarios to watch the response times but nothing. I just had a user ask if there was something wrong, so on a whim I bounced the httpd service and it cleared right up. I have done this a few times and have spent the last week looking for something that might be causing the issues. Any pointers would help.

[dimir]

How about monitoring memory usage on an Apache host? Is there a chance at some point it's swapping?

[tmroberts]

Its not swapping at all. In fact the available swap hasn't moved from 100% since the day I stood them up. I can't help but wonder if there is something in the HTTP configs that needs to be tweaked but I'm not an Apache/HTTP guy in the least.

[dimir]

I heard nginx does the job better, but that's all I can give here, sorry.

[tmroberts]

Is it even possible to switch over from Apache to nginx once the system has already been up and running?

[ingus.vilnis]

Hi,

Check Apache error logs for any issues - out of configured memory limits, timeouts etc.

But are you sure that the problem is on the web server side? Check your DB performance. In very many occasions the database server and its tuning does not keep up with the load but in that case sluggish web interface is just a consequence.

Nginx (tuned to match the server it is running on) may be faster. Yes, it is possible to change the web server engines after the system is already running bit I really doubt that your problems will be solved with that.

Look at the characteristics of your database, how the basic CPU, memory and swap readings look and start from there. Also very important is how busy your Zabbix server is, and how many users you have connected at the same time.

[tmroberts]

I've been logging slow queries and have just about every metric I can find on the database to make sure the bottle neck isn't the DB. I just recently modified the HTTPD conf to include the mod_deflate module but I'm still seeing some performance issues on the UI. Is there any where that shows how to switch over from Apache to Nginx? Will I get better performance from Nginx? The one question I do have is this... We are running about 3700 hosts right now. Some of the host groups have as many as 600 hosts in them. I know this is a lot of data pull back, but it shouldn't be that much. I do have the History and Trends on all items set to 90 days and 730 days respectively. The database is running about 2.8TB right now. Could it be that I just have too much data?

[ingus.vilnis]

Try enabling Debug mode in Zabbix web interface for your user and see the output of a sluggish page. The first section of debug will show time taken total and by SQL queries separately. What have you got in there?