Needecomendation for 7000 host

[pantera]

Hi
I will add about 7000 host in zabbix.
I need zabbix server configuration suggestion and server specifications
please give me your advice to tune up my zabbix server for best performance

[kloczek]

Hi
I will add about 7000 host in zabbix.
I need zabbix server configuration suggestion and server specifications
please give me your advice to tune up my zabbix server for best performance

More important than number of hosts is number of monitored items and metrics sampling rate.
Try to read https://www.zabbix.com/forum/showthread.php?t=50665

[melpheos]

With that many host, your limitation wont be the server but the storage.
We have 558 host and around 20000 items and this gives us around 300 write iops/sec
4Cpu and 16Go of memory
Memory is full but CPU is doing nothing.
You should be aware of the memory consumption and the SAN you will use.
I dont think you can expect to run that many hosts without a SAN or SSD disk in raid 10 (exept if you do only ping which would be sad)
Found this sizer

https://www.google.fr/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&ved=0CCcQFjABahUKEwi11pHIvpfJAhVJ1SwKHW6-DSA&url=https%3A%2F%2Fwww.zabbix.com%2Fforum%2Fattachment.php%3Fattachmentid%3D3%26d%3D1100520552&usg=AFQjCNFAnBJUkmV7R3C_p4bKSGEh69pJhg&sig2=uZzNTf-bV2N9TTpox-jijw&cad=rja

[kloczek]

With that many host, your limitation wont be the server but the storage.
We have 558 host and around 20000 items and this gives us around 300 write iops/sec
4Cpu and 16Go of memory
Memory is full but CPU is doing nothing.
You should be aware of the memory consumption and the SAN you will use.
I dont think you can expect to run that many hosts without a SAN or SSD disk in raid 10 (exept if you do only ping which would be sad)
Found this sizer
https://www.google.fr/url?sa=t&rct=j...x-jijw&cad=rja

For 20k items this hardware should be OK.

I'm not sure are you using Linux or not and if Linux which one distro.
If you are going to use Linux as DB platform I would recommend to use Oracle Linux. Why? Fo only one very basic reason: this distro provides OOTB dtrace support so with such feature is possible to measure distribution of latency read() and write() syscalls.

Below oneliner can be used on Linux and Solaris. On my DB backend (on Solaris) it shows result like below:

# dtrace -qn 'syscall::write:entry /execname == "mysqld"/ {self->stime = timestamp;} syscall::read:entry /execname == "mysqld"/ {self->stime = timestamp;} syscall::write:return /self->stime != 0/ {@LWrite = quantize(timestamp - self->stime);} syscall::read:return /self->stime != 0/ {@LRead = quantize(timestamp - self->stime);}  tick-10s {printa(@LWrite);} tick-10s {printa(@LRead);}'


           value  ------------- Distribution ------------- count    
            1024 |                                         0        
            2048 |                                         81       
            4096 |@@                                       743      
            8192 |@@@@@@@@@@@@@@@@@@@@@@                   6975     
           16384 |@@@@@@@@@@@@@@                           4463     
           32768 |@                                        395      
           65536 |                                         57       
          131072 |                                         8        
          262144 |                                         0        
          524288 |                                         1        
         1048576 |                                         0        
         2097152 |                                         0        
         4194304 |                                         9        
         8388608 |                                         5        
        16777216 |                                         0        
        33554432 |                                         0        
        67108864 |                                         0        
       134217728 |                                         0        
       268435456 |                                         0        
       536870912 |                                         0        
      1073741824 |                                         2        
      2147483648 |                                         0        



           value  ------------- Distribution ------------- count    
             512 |                                         0        
            1024 |@@                                       1467     
            2048 |@@@@@@@@@@                               7286     
            4096 |@@@@@@@@@@@@@@@@@@@                      13711    
            8192 |                                         311      
           16384 |                                         33       
           32768 |                                         33       
           65536 |@@@                                      2257     
          131072 |@@                                       1102     
          262144 |@                                        841      
          524288 |@                                        456      
         1048576 |@                                        580      
         2097152 |@                                        519      
         4194304 |                                         241      
         8388608 |                                         96       
        16777216 |                                         18       
        33554432 |                                         8        
        67108864 |                                         43       
       134217728 |                                         21       
       268435456 |                                         8        
       536870912 |                                         47       
      1073741824 |                                         2        
      2147483648 |                                         18       
      4294967296 |                                         13       
      8589934592 |                                         0        

^C

This oneliner should work on FreBSD as well.

Of course longer you will keep running this command than more accurate output about latency distribution it will provide.

Observing with what latency are done reads and write operations is quite important on investigate is backend storage is enough or not.
Remember that less read() will cause reading some blocks from storage than more robust would be all inserts and updates queries.

[melpheos]

For 20k items this hardware should be OK.

I'm not sure are you using Linux or not and if Linux which one distro.
If you are going to use Linux as DB platform I would recommend to use Oracle Linux. Why? Fo only one very basic reason: this distro provides OOTB dtrace support so with such feature is possible to measure distribution of latency read() and write() syscalls.

OP is looking for 7000 host so this might be between 70K to 700K items...
We do not have people here who has knowledge of Oracle and the Oracle pricing model do not suits us (vmware farm so pricing could be incredibly high if we do not assign a host for zabbix)
tl;dr : Oracle is too expansive for our monitoring needs

[kloczek]

OP is looking for 7000 host so this might be between 70K to 700K items...
We do not have people here who has knowledge of Oracle and the Oracle pricing model do not suits us (vmware farm so pricing could be incredibly high if we do not assign a host for zabbix)
tl;dr : Oracle is too expansive for our monitoring needs

Oracle Linux as upstream is using Fedora. Fedora is similar to RHEL and CentOS.

On my zabbix I have 150k items and about 2.5k nvps. DB backend hw details you can find in tread which I've posted few days ago. On building such backend on Solaris you can gain much higher performance than on Linux (reason: ZFS) but to use it you must know a little more about Solaris. With Linux probably it will be necessary to add more memory than 16GB which you have now.

OL is free to use. Even DTrace packages are free.
Support is paid separately and you need to buy it only if you need it.
Nevertheless I recommend to buy at least one support license to organize mirror of packages which will be used on on install server. Cost of single support license is the same as for Solaris (£600/year) however in case of OL if you will buy support for 3years you will have 30% discount.
You can buy support license (with discount) over web interface making credit card payment.

https://shop.oracle.com/pls/ostore/f?p=dstore:2:0::NO:RIR,RP,2:PROD_HIER_ID:4510272175861805728468

With support you will have additional set of repositories where are ready to use OpenStack packages, Javas (7,8), different MySQLs (communiuty 5.5, 5.6), access to OVM repos (Oracle Virtual Machine .. which with OpenStack can be used as replacement of VMware)
One license will allow you to raise SRs if you will find some issues ..
Effectively with single support license you can raise issues against many software components.
IMO Oracle support even on Linux is way better than RH support .. but it is my private opinion and some people may not agree

If you have already already Oracle hardware with hardware support (even bronze support) automatically you have Solaris and Oracle Linux support on this hardware without additional costs.

[pantera]

I'm using CentOS 6.6
RAM 3831MB

[melpheos]

Oracle Linux as upstream is using Fedora. Fedora is similar to RHEL and CentOS.

On my zabbix I have 150k items and about 2.5k nvps. DB backend hw details you can find in tread which I've posted few days ago. On building such backend on Solaris you can gain much higher performance than on Linux (reason: ZFS) but to use it you must know a little more about Solaris. With Linux probably it will be necessary to add more memory than 16GB which you have now.

OL is free to use. Even DTrace packages are free.
Support is paid separately and you need to buy it only if you need it.
Nevertheless I recommend to buy at least one support license to organize mirror of packages which will be used on on install server. Cost of single support license is the same as for Solaris (£600/year) however in case of OL if you will buy support for 3years you will have 30% discount.
You can buy support license (with discount) over web interface making credit card payment.

https://shop.oracle.com/pls/ostore/f?p=dstore:2:0::NO:RIR,RP,2:PROD_HIER_ID:4510272175861805728468

With support you will have additional set of repositories where are ready to use OpenStack packages, Javas (7,8), different MySQLs (communiuty 5.5, 5.6), access to OVM repos (Oracle Virtual Machine .. which with OpenStack can be used as replacement of VMware)
One license will allow you to raise SRs if you will find some issues ..
Effectively with single support license you can raise issues against many software components.
IMO Oracle support even on Linux is way better than RH support .. but it is my private opinion and some people may not agree

If you have already already Oracle hardware with hardware support (even bronze support) automatically you have Solaris and Oracle Linux support on this hardware without additional costs.

There is a confusion... I'm not the OP
I have no intent to change my architecture

Pantera is

BTW, there is nothing special about Oracle Linux... There is actualy no Oracle DB in it so yeah, you can put a MySQL in it but there is no benefit over other Linux distro besides having Oracle in the name.
Maybe i was not clear about "Oracle knowledge" but to me Oracle is OracleDB...

[pantera]

most commonly, as I understood, RAM is more needed, CPU has no influence
and increasing DB cache.

Any more suggestions?