Understading Zabbix Data Flow

[rohitist]

Hi,

After debugging, i managed to know that anything displayed on overview page of zabbix's UI are retrieved from database. This call is made from the page item.inc.php.

Now my question is; Can anyone please help me to know how an item collects the data from the agent and insert them into the databse ?

Thanks in advance.

Regards,
Rohit

[aib]

The simple answer: magically.
The more complicated answer: You can download the sources of Zabbix and check the Data Flow by yourself.

Sorry, I don't want to offend you, but I don't think that there are anybody who can understand the whole Data Flow path better that developers. And I'm completely sure that nobody from developers team will spend hours and days to explain you the Data Flow by heart.

[steveboyson]

Have a look at the history* tables in your DB, there is one for each data type (uint, text, log, ...). The structure is almost identical while indexes are not.

Or - even better - have a look at the Zabbix API which wrappes DB queries.

[rohitist]

Thanks Aleksey and Steve for your reply.

I have done several hours of debugging and finally developed a widget which shows top 5 processes running on a particular host.

https://www.dropbox.com/s/tntefd0ky52h5rz/Screenshot.png

I will share the codes when it will be completely tested for multiple hosts monitoring.

I understood that whatever data shown on the frontend are retrieved from the DB. (Like you said in DB about history_* tables).

But moving further, I want to understand how data is pushed into the database from a zabbix agent. Since I have just started my career as a software professional and I have not ever made a project in C , I would be very pleased to know the way to begin the debugging of the source codes written in C so that I can further contribute for Zabbix (right from the source directory to import, preparation and resolution of dependecies, etc).

@Aleksey: You are right. None can explain the vast data flow of zabbix. But this time I just want to know: where to begin and how to begin.

P.S: I am using Eclipse for this on RHEL.

Can anyone please guide me into this.

Many thanks in advance.

Regards,
Rohit

[aib]

If you would like to make some additional programs without crushing through the source code, you can take a look on Zabbix API
API provides you a lot of methods to create data/triggers/events/etc. and extract any data from Zabbix DB.

I hope it will be very interesting challenge.

[rohitist]

Thanks Aleksey but my requirement is different.

The servers which are to be monitored are deployed in the client locations and their firewall rules restricts the outgoing data to our internal servers. You can say, this is our business bottleneck.
Thus, I would like to customize zabbix agent, zabbix server and zabbix sender codes in such a way that zabbix sender sends data to zabbix server in a file because bi-directional file transfer is only allowed between servers. As a result, zabbix server would receive all monitoring data in a file and then it would parse those data and insert into DB periodically.

I have begun to debug the code of zabbix server and zabbix agent to understand the flow. I hope, i could figure out the way to make the most out of zabbix.

Meanwhile, I would be grateful to you if you could assist in this mission

Thanks for your understanding and valuable time.

Regards,
Rohit

[steveboyson]

How do you transfer the file from the sender machine to the zabbix server?
With an USB stick?

By the way, nobody stops you from calling zabbix_sender directly on the zabbix server. So, create your send file, bring it in some way to your zabbix server, launch zabbix-sender there.

I still wonder why someone would to do it that way ...

[steveboyson]

Nota bene: for scenarios like you described THEY have created zabbix-proxy. Your clients talk only to the proxy while only the proxy communicates with your zabbix server. See the docs for that.

[rohitist]

Hi steveboyson,

File transfer is allowed only through ftp between our servers (not USB).
I have tried to make a successful connection between agent and server installtion but that couldn't be established. This is because ports are not allowed through the firewall. Opening ports require several business approavals. Thats why I am looking for a way to send file to the zabbix server from the monitored host periodically.

And about the PROXY installation, I would like to say, this still requires business approval in one of the server of a client. And my organization has several clients. Thus several clients add up to several business approvals again. We have Nagios running on them but we would like to have one spot monitoring advantage of zabbix. That is why I am left with the "file transfer" choice.

I hope you get me this time.

Regards,
Rohit

[steveboyson]

You are aware that regular FTP is plain-text, right? As long as you are not using any VPN/IPSEC/SSL tunnel (or other mechanism), this is by far the most insecure way of transferring data between systems.

I would suggest to think about a different system to connect your zabbix clients.

[aib]

Or to stop experimenting with unsecured connections and file transfers.
It will be a big PITA if somebody realize that you are sending the important information outside of business perimeter.

[rohitist]

Yes, we are using VPN as well as Tunnel to access servers. So, I guess security is not a concern over here.

[paulie]

I have tried to make a successful connection between agent and server installtion but that couldn't be established. This is because ports are not allowed through the firewall. Opening ports require several business approavals. Thats why I am looking for a way to send file to the zabbix server from the monitored host periodically.

And about the PROXY installation, I would like to say, this still requires business approval in one of the server of a client. And my organization has several clients. Thus several clients add up to several business approvals again. We have Nagios running on them but we would like to have one spot monitoring advantage of zabbix. That is why I am left with the "file transfer" choice.

I hope you get me this time.

Regards,
Rohit

Hello,

working in enterprise environment myself, I can completely understand your pain trying to get business approval to change the infrastructure setup.
But it is the only way to get monitoring working reliably. Workarounds like copying files and importing to Zabbix server could work, but creating workarounds only tends to get things more complicated later
and makes maintaining the monitoring changes a nightmare.

Paul

[rohitist]

Thanks Paul for understanding.

I have re-written the codes according to my need i discussed. And, now zabbix is working using SFTP data exchange. . About the complexity you mentioned, I tell you, I have written codes in such a way that ensures the simplicity same as the original Zabbix's TCP/IP PIPE implementation.

Thanks everyone for you valuable thoughts. And especially thanks for the openness of Zabbix.

Regards,
Rohit

[dimir]

Just in case:

https://youtu.be/EqYGnrFeMgw