AS/400 Monitoring solutions

[celeborn81]

Hello everyone.
I am implementing zabbix in the company.
Having AS400 I was very interested in the project.
Following the directions I tried to start the service.
And I get as an error:

java.net.BindException: Address already in use (Bind failed)

The agent I am launching from the same machine where the server is installed.
This could be the problem.
I also tried to change the door.
I report the complete error log:

    
11:20220803:171003.308 Starting Zabbix Agent v0.7.9
    11:20220803:171003.314 using configuration file: [Full Path]zabbix_agentd.conf
    11:20220803:171003.314 Debian Java version "11.0.16"
    11:20220803:171003.314  OpenJDK Runtime Environment (build 11.0.16+8-post-Debian-1deb11u1)
    11:20220803:171003.315  OpenJDK 64-Bit Server VM (build 11.0.16+8-post-Debian-1deb11u1, mixed mode, sharing)
    11:20220803:171003.319  Open Source Software, JTOpen 8.2, codebase 5770-SS1 V7R2M0.00 built=20140313 @KD
    11:20220803:171003.459  Agent hostname: 'as400.organization.intra', System info: IBM OS/400 XXXXXXXX V7R3M0, Debian OpenJDK 64-Bit Server VM (v11.0.16)
    15:20220803:171003.497 agent #1 started [collector]
    16:20220803:171003.502 agent #2 (zabbix.organization.intra) started [active checks #3]
    17:20220803:171003.502 agent #3 started[listener #1]
    19:20220803:171003.506 agent #5 started[listener #3]
    18:20220803:171003.508 agent #4 started[listener #2]
     1:20220803:171003.508 PassiveCheck.init() error: java.net.BindException: Indirizzo già in uso (Bind failed)
    17:20220803:171003.509 agent #3 stopped[listener #1]
    18:20220803:171003.510 agent #4 stopped[listener #2]
    19:20220803:171003.510 agent #5 stopped[listener #3]
    16:20220803:171003.518  no active checks: host [as400.organization.intra] not found
    15:20220803:171003.518  Procstat.updateJobinfoList() error: java.lang.InterruptedException
    16:20220803:171003.518 agent #2 (zabbix.organization.intra) stopped [active checks #3]
     1:20220803:171004.011 Zabbix Agent stopped. v0.7.9
    15:20220803:171006.500 agent #1 stopped [collector]

[Oriola Ferhati]

Hi,

Where can I find the link please , because the link provided by this post is not found.

Thank you

[dm_bry]

https://drive.google.com/file/d/1omiFTEoS83h5Fk5HYRprXfUY-fxowJY9/view

[Oriola Ferhati]

anyone could help me with the installation of zabbix agent ?

[dm_bry]

Hello
Would anyone have an example of omitting message text in qsysopr?

[Kos]

And I get as an error:

java.net.BindException: Address already in use (Bind failed)

The agent I am launching from the same machine where the server is installed.
This could be the problem.
I also tried to change the door.
I report the complete error log:

[...]
1:20220803:171003.508 PassiveCheck.init() error: java.net.BindException: Indirizzo già in uso (Bind failed)
[...]

It seems that the port configured for the Agent is already busy; for example, the usual Zabbix Agent (for OS Linux) is started before and uses the same port number (by default, 10050).

anyone could help me with the installation of zabbix agent ?

The archive contains a detailed documentation; I don't like to copy-paste it here, if you have some more specific questions - you are welcome

Would anyone have an example of omitting message text in qsysopr?

Could you provide a bit more details, please, what do you mean?
Do you want to monitor only some specific messages in QSYSOPR, or just opposite - to monitor all messages excluding some specific ones?

[dm_bry]

Just the opposite, exclude. These can be messages a program might send to QSYSOPR using SNDMSG arriving at severity 80 for example, and therefore no associated MSGID, but only some unique message text. Thank you

[Kos]

Just the opposite, exclude. These can be messages a program might send to QSYSOPR using SNDMSG arriving at severity 80 for example, and therefore no associated MSGID, but only some unique message text. Thank you

In this situation you need some mechanism for negative filtering, i.e. filter all messages _excluding_ matching some regular expression.
Unfortunately, it's not a trivial task in Zabbix; but, however, it still is possible using at least 2 ways:

• you can use global regular expressions in Zabbix: global regular expressions allow "negative matching" logic (and our Zabbix Agent emulator for AS/400 supports global RE for eventlog[...] metric);
• in the last versions of Zabbix you can add a preprocessing step for the appropriate item with type "Does not match regular expression", enter your RE and mark a checkbox "Custom on fail" with action "discard value". In this case Zabbix Agent will process all messages, but some of them will be filtered out on the Zabbix Server side (i.e. in fact - ignored).

[dm_bry]

Many thanks for this advice.

[Anon_Zabbix_User]

Hi everyone, i've read the documentation, and all these forum posts, I haven't fully understand the requeriments for doing it remotely? eg: Virtual machine with windows that has the requirements for the agent simulator (JRE and so on) and a connection with AS400 via network and by user and password? If someone could explain it to me it would be awesome

[Kos]

Anon_Zabbix_User , yes, you are right. This is one of designed ways to run this agent, it described in a documentation.
You need 4 files: 3 JAR-files (ZabbixAgent.jar, jt400.jar and json-simple-1.1.1.jar) and a config file (ZabbixAgent.conf by default), as well as write permissions to directory for the log files.
After editing of config file (at least, "LogFile", "Server", "as400ServerHost" and "as400Password" parameters) you could start this program using command like:

[/pat/to/]java -jar [path/to/]ZabbixAgent.jar [[/path/to/]config_file]

The question "How can I start this command automatically upon machine reboot?" is out of scope, as the answer entirely depends on your operating system.

[Anon_Zabbix_User]

Anon_Zabbix_User , yes, you are right. This is one of designed ways to run this agent, it described in a documentation.
You need 4 files: 3 JAR-files (ZabbixAgent.jar, jt400.jar and json-simple-1.1.1.jar) and a config file (ZabbixAgent.conf by default), as well as write permissions to directory for the log files.
After editing of config file (at least, ) you could start this program using command like:

[/pat/to/]java -jar [path/to/]ZabbixAgent.jar [[/path/to/]config_file]

The question "How can I start this command automatically upon machine reboot?" is out of scope, as the answer entirely depends on your operating system.

Thank you for your quick response Kos, that is really nice and much less intrusive.

[Anon_Zabbix_User]

Anon_Zabbix_User , yes, you are right. This is one of designed ways to run this agent, it described in a documentation.
You need 4 files: 3 JAR-files (ZabbixAgent.jar, jt400.jar and json-simple-1.1.1.jar) and a config file (ZabbixAgent.conf by default), as well as write permissions to directory for the log files.
After editing of config file (at least, "LogFile", "Server", "as400ServerHost" and "as400Password" parameters) you could start this program using command like:

[/pat/to/]java -jar [path/to/]ZabbixAgent.jar [[/path/to/]config_file]

The question "How can I start this command automatically upon machine reboot?" is out of scope, as the answer entirely depends on your operating system.

Hi Kos , Again sorry for any inconvenience, but I haven't found anything in the documentation, what type of permissions does the as400 user need to have and where? Also, how does the agent gather its metrics?

[Kos]

Hi Kos , Again sorry for any inconvenience, but I haven't found anything in the documentation, what type of permissions does the as400 user need to have and where? Also, how does the agent gather its metrics?

As mentioned in the "Implementation notes" section of the documentation, the agent collects its metrics using 2 ways:

• IBM Toolbox for Java API (most of metrics);
• some native IBM system i API (disks/ASP's).

Default permissions for an unprivileged user are usually enough for all standard metrics.
There was only one case in my practice where an AS/400 admin performed an additional hardening of his system; in result this Agent could not obtain a state of needed subsystem (as it could not "see" that subsystem at all), therefore it was needed to add him some additional permissions.

Of course, if you use the "remote" mode (i.e. Agent is working on another machine rather AS/400), you need to ensure that all network communications are available (firewalls, NAT, etc.).

[Anon_Zabbix_User]

As mentioned in the "Implementation notes" section of the documentation, the agent collects its metrics using 2 ways:

• IBM Toolbox for Java API (most of metrics);
• some native IBM system i API (disks/ASP's).

Default permissions for an unprivileged user are usually enough for all standard metrics.
There was only one case in my practice where an AS/400 admin performed an additional hardening of his system; in result this Agent could not obtain a state of needed subsystem (as it could not "see" that subsystem at all), therefore it was needed to add him some additional permissions.

Of course, if you use the "remote" mode (i.e. Agent is working on another machine rather AS/400), you need to ensure that all network communications are available (firewalls, NAT, etc.).

Thank you for all your fast responses, you've been a great help.

Best Regards,

Anon_Zabbix_User

[Alfons0]

Good afternoon,

We have a problem in the detection of alerts concerning loss of communication of disk events, after upgrading Zabbix to version 6.0.13.

The problem is given because the problem appears to us but not to the client and in order to solve it, the client has to restart the agent in the given as400 machine (also has been updated) and some time later the alert is solved.
We have made a summary of how it is configured and the version we are using:

• Trigger: nodata(/HOSTNAME/as400.disk.size[DSK_ID,pused],1h)=1
• System information: IBM OS/400 HOSTNAME V7RM0, IBM Corporation IBM J9 VM (v1.8.0_351)
• Version of zabbix_agent(d) running: 0.7.9

Could you please let us know if there is currently any incompatibility issue with the new version of zabbix regarding the configured as400 alerts or if it is an internal problem?


Thank you very much in advance.

[turki]

Hi,

first I would like to thanks to Kos for it's work.
I always had a problem in continuous monitoring of our IBM i environment, until colleague show me a zabbix, and I discover this agent.
Now I had over a year of data, and that completely change my perspective.

Ok, now to the problem.

We are running SAP on IBM i with iASP, and I recently apply latest PTF's to my test environment, and iASP monitoring goes crazy.

Let me show a problem by an example.
When I test function vfs.fs.discovery it normally returns:

{"data":[
{"{#FSNAME}":"34","{#FSTYPE}":"11"},
{"{#FSNAME}":"1","{#FSTYPE}":"00"},
{"{#FSNAME}":"33","{#FSTYPE}":"11"}
]}

but after applying latest PTF-s it return this:

{"data":[
{"{#FSNAME}":"2","{#FSTYPE}":""},
{"{#FSNAME}":"-261947392","{#FSTYPE}":""},
{"{#FSNAME}":"1","{#FSTYPE}":"00"}
]}

Not mentioning that data is completely garbage for ASP other than 1 (*SYSBAS).

Group PTF's I apply are:

PTF Group  Level  Text
SF99740    23117  CUMULATIVE PTF PACKAGE C3117740
SF99739      108  GROUP HIPER
SF99738       46  GROUP SECURITY
SF99737        8  TECHNOLOGY REFRESH
SF99704       25  DB2 FOR IBM I
SF99675        2  HARDWARE AND RELATED PTFS
SF99667       10  740 TCP/IP PTF
SF99666       14  HIGH AVAILABILITY FOR IBM I
SF99665       18  JAVA
SF99664       38  BACKUP RECOVERY SOLUTIONS
SF99663       12  PERFORMANCE TOOLS
SF99662       28  IBM HTTP SERVER FOR I
​

I monitor my partitions, by using remote connection with running agents on one partition to collect information from all other partitions.
That's my favorite, because there is no need to install and run anything custom on monitored server.
So first I suspect in PTF difference between partition with agents and test partition, so I install agent directly on test partition and use newer jt400.jar, but result is the same.

please help.

best regards,
turki

[Kos]

Ok, now to the problem.

We are running SAP on IBM i with iASP, and I recently apply latest PTF's to my test environment, and iASP monitoring goes crazy.

Let me show a problem by an example.
When I test function vfs.fs.discovery it normally returns:

{"data":[
{"{#FSNAME}":"34","{#FSTYPE}":"11"},
{"{#FSNAME}":"1","{#FSTYPE}":"00"},
{"{#FSNAME}":"33","{#FSTYPE}":"11"}
]}

but after applying latest PTF-s it return this:

{"data":[
{"{#FSNAME}":"2","{#FSTYPE}":""},
{"{#FSNAME}":"-261947392","{#FSTYPE}":""},
{"{#FSNAME}":"1","{#FSTYPE}":"00"}
]}

Not mentioning that data is completely garbage for ASP other than 1 (*SYSBAS).

Hi turki,
check, please, that you use the last version of the Zabbix agent emulator (0.7.9 at the moment).
I provided an announce about this problem and an update some time ago (see link here).
If the problem still persist with this version, contact me directly by e-mail, please.