UTF-8 Encoding Issue Log file Monitoring

[Joby]

Hi,

I am facing an encoding issue while monitoring a tomcat log file. The issue is with Japanese Characters only. The Japanese characters in log file shows as question marks in the alert emails.

I have configured "UTF-8" in the item configuration.

Please help me to fix this issue.

Thanks
Joby

[Atsushi]

Please make sure not in the BOM is marked with the log file.
UTF-8 in the libiconv that are available in the character code conversion is the encoding without BOM.

Please try to be output log in UTF-8 without BOM or ASCII.

[Joby]

Please make sure not in the BOM is marked with the log file.
UTF-8 in the libiconv that are available in the character code conversion is the encoding without BOM.

Please try to be output log in UTF-8 without BOM or ASCII.

Could you please explain how can I do this. I mean the changes that I need to make in the zabbix Configuration. Tomcat log file shows the characters correctly, issue is only with the zabbix alert email.

[Atsushi]

Do the Japanese Characters in the log are displayed correctly at Latest data view?
The mail system do you correctly handle the UTF-8?

[Joby]

Do the Japanese Characters in the log are displayed correctly at Latest data view?
The mail system do you correctly handle the UTF-8?

In zabbix console latest data, Japanese characters are displayed as question marks. Email body also is the same.

Only catalina.out file shows the exact characters.

[Atsushi]

If the latest data are question marks, it thought might be a problem with the environment construction of Zabbix.

There are some checkpoint.

For example, Zabbix server side:

- What are you use the platform? Linux? Windows? Other?
- What version of Zabbix is using? 2.2.13? 3.0.3?
- What DBMS for Zabbix is using? MySQL? PostgreSQL?
- If the database characterset is set to utf8?

Zabbix agent side:

- What are you use the platform? Linux? Windows? Other?
- What version of Zabbix is using? 2.2.13? 3.0.3?
- What detail version for Java and Tomcat?
- What characterset is using to catalina.out?
- Are you specify a characterset, such as LANG during Tomcat startup?


So if you can use Japanese, There is a forum to communication in Japanese at Japan community's site.
If you want to ask questions in Japanese, try to use.

ZABBIX-JP (Unofficial Zabbix Japan Community):
http://www.zabbix.jp/

[Joby]

If the latest data are question marks, it thought might be a problem with the environment construction of Zabbix.

There are some checkpoint.

For example, Zabbix server side:

- What are you use the platform? Linux? Windows? Other?
- What version of Zabbix is using? 2.2.13? 3.0.3?
- What DBMS for Zabbix is using? MySQL? PostgreSQL?
- If the database characterset is set to utf8?

Zabbix agent side:

- What are you use the platform? Linux? Windows? Other?
- What version of Zabbix is using? 2.2.13? 3.0.3?
- What detail version for Java and Tomcat?
- What characterset is using to catalina.out?
- Are you specify a characterset, such as LANG during Tomcat startup?


So if you can use Japanese, There is a forum to communication in Japanese at Japan community's site.
If you want to ask questions in Japanese, try to use.

ZABBIX-JP (Unofficial Zabbix Japan Community):
http://www.zabbix.jp/

I have updated the details in line below.

- What are you use the platform? Linux? Windows? Other - Linux
- What version of Zabbix is using? 2.2.13? 3.0.3? - 2.2.8
- What DBMS for Zabbix is using? MySQL? PostgreSQL? - MySQL
- If the database characterset is set to utf8? - Yes

- What are you use the platform? Linux? Windows? Other? - Linux
- What version of Zabbix is using? 2.2.13? 3.0.3? - 2.2.8
- What detail version for Java and Tomcat? - Java 1.7 and Tomcat7
- What characterset is using to catalina.out? - UTF-8
- Are you specify a characterset, such as LANG during Tomcat startup? -NO

Thanks

[Atsushi]

I tested by Zabbix 2.2.13 + OpenJDK 1.8.0 + Tomcat 7.0.54.
As a result, the Japanese character can be properly monitored.

The value of the key for item that I use is as follows.

logrt[/var/log/tomcat/catalina.*.log]

Please check the character set of the database again.

ex.

# mysql -uzabbix -p zabbix
Enter password:

mysql> show variables like 'char%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

mysql>

[Joby]

I tested by Zabbix 2.2.13 + OpenJDK 1.8.0 + Tomcat 7.0.54.
As a result, the Japanese character can be properly monitored.

The value of the key for item that I use is as follows.

logrt[/var/log/tomcat/catalina.*.log]

Please check the character set of the database again.

ex.

# mysql -uzabbix -p zabbix
Enter password:

mysql> show variables like 'char%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

mysql>

Thanks for all your help. I figured out the issue. Encoding was different.

[B.C.K]

Hello,

I wish you a beautiful Friday,

I checked also my variables. Althoug I configured the Database with utf8 like
"CREATE DATABASE zabbix character set utf8 collate utf8_bin;" with different db name, the variables shows liket this:
MariaDB [(none)]> show variableslike 'char%';
+--------------------------+------------------------------+
| Variable_name | Value |
+--------------------------+------------------------------+
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_database | latin1 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | latin1 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mariadb/charsets/ |
+--------------------------+------------------------------+
8 rows in set (0.001 sec)


It seems because of that I can only use the special characters -_.
The Characters []{}*/\#?`´'!^°"§$%&/()=ß+~äü:;, are not accepted.
By the way I am using Zabbix 5.2.6 with 10.3.27-MariaDB. OS Ist CentOS8.3.2011

Did I forgot something during the setup?
I followed the page https://computingforgeeks.com/how-to...el-8-centos-8/ with its crosslinks to Apache and Maria DB.

Would a change of the variables from latin1 to utf8 brings to success, so that I can use the not accepted characters above?

With best regards