manual:discovery:low_level_discovery

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
manual:discovery:low_level_discovery [2019/02/04 10:06]
martins-v removing some possibly redundant information
manual:discovery:low_level_discovery [2021/01/14 11:36] (current)
marinagen [ZBXNEXT-5518] new XML to JSON preprocessing step
Line 1: Line 1:
-==== 3 Low-level discovery ====+===== 3 Low-level discovery ​=====
  
-=== Overview ===+==== Overview ​====
  
-Low-level discovery provides a way to automatically create items, triggers, and graphs for different entities on a computer. For instance, Zabbix can automatically start monitoring file systems or network interfaces on your machine, without the need to create items for each file system or network interface manually. Additionally it is possible to configure Zabbix to remove unneeded entities automatically based on actual results of periodically performed discovery.+Low-level discovery provides a way to automatically create items, triggers, and graphs for different entities on a computer. For instance, Zabbix can automatically start monitoring file systems or network interfaces on your machine, without the need to create items for each file system or network interface manually. Additionallyit is possible to configure Zabbix to remove unneeded entities automatically based on actual results of periodically performed discovery.
  
 A user can define their own types of discovery, provided they follow a particular JSON protocol. A user can define their own types of discovery, provided they follow a particular JSON protocol.
Line 21: Line 21:
 Built-in discovery keys have been updated to return an array of LLD rows at the root of JSON document. Zabbix will automatically extract a macro and value if an array field uses the {#MACRO} syntax as a key. Any new native discovery checks will use the new syntax without the %%"​%%data%%"​%% elements. When processing a low-level discovery value first the root is located (array at ''​$.''​ or ''​$.data''​). ​ Built-in discovery keys have been updated to return an array of LLD rows at the root of JSON document. Zabbix will automatically extract a macro and value if an array field uses the {#MACRO} syntax as a key. Any new native discovery checks will use the new syntax without the %%"​%%data%%"​%% elements. When processing a low-level discovery value first the root is located (array at ''​$.''​ or ''​$.data''​). ​
  
-While the %%"​%%data%%"​%% element has been removed from all native items related to discovery, for backward compatibility Zabbix will still accept the now deprecated ​JSON notation with a %%"​%%data%%"​%% element. If the JSON contains an object with only one %%"​%%data%%"​%% array element, then it will automatically extract the content of the element using JSONPath ''​$.data''​. Low-level discovery now accepts optional user-defined LLD macros with a custom path specified in JSONPath syntax.+While the %%"​%%data%%"​%% element has been removed from all native items related to discovery, for backward compatibility Zabbix will still accept the JSON notation with a %%"​%%data%%"​%% element, though its use is discouraged. If the JSON contains an object with only one %%"​%%data%%"​%% array element, then it will automatically extract the content of the element using JSONPath ''​$.data''​. Low-level discovery now accepts optional user-defined LLD macros with a custom path specified in JSONPath syntax.
  
 <note warning>​As a result of the changes above, newer agents no longer will be able to work with an older Zabbix server.</​note>​ <note warning>​As a result of the changes above, newer agents no longer will be able to work with an older Zabbix server.</​note>​
  
 See also: [[#​discovered_entities|Discovered entities]] See also: [[#​discovered_entities|Discovered entities]]
-=== Configuring low-level discovery ===+==== Configuring low-level discovery ​====
  
 We will illustrate low-level discovery based on an example of file system discovery. We will illustrate low-level discovery based on an example of file system discovery.
Line 32: Line 32:
 To configure the discovery, do the following: To configure the discovery, do the following:
  
-  * Go to: //​Configuration//​ -> //​Templates//​  +  * Go to: //​Configuration//​ -> //Templates// or //Hosts//  
-  * Click on //​Discovery//​ in the row of an appropriate template+  * Click on //​Discovery//​ in the row of an appropriate template/host
  
 {{manual:​discovery:​low_level_discovery:​fs_templates.png|}} {{manual:​discovery:​low_level_discovery:​fs_templates.png|}}
Line 42: Line 42:
 === Discovery rule === === Discovery rule ===
  
-The discovery rule form contains ​four tabs, representing,​ from left to right, the data flow during discovery:+The discovery rule form contains ​five tabs, representing,​ from left to right, the data flow during discovery:
  
   * //Discovery rule// - specifies, most importantly,​ the built-in item or custom script to retrieve discovery data   * //Discovery rule// - specifies, most importantly,​ the built-in item or custom script to retrieve discovery data
Line 48: Line 48:
   * //LLD macros// - allows to extract some macro values to use in discovered items, triggers, etc   * //LLD macros// - allows to extract some macro values to use in discovered items, triggers, etc
   * //Filters// - allows to filter the discovered values   * //Filters// - allows to filter the discovered values
 +  * //​Overrides//​ - allows to modify items, triggers, graphs or host prototypes when applying to specific discovered objects
  
 The **Discovery rule** tab contains the item key to use for discovery (as well as some general discovery rule attributes): ​ The **Discovery rule** tab contains the item key to use for discovery (as well as some general discovery rule attributes): ​
  
-{{manual:​discovery:​low_level_discovery:​lld_rule_fs0.png?600|}}+{{manual:​discovery:​low_level_discovery:​lld_rule_fs0a.png|}}
  
 All mandatory input fields are marked with a red asterisk. ​ All mandatory input fields are marked with a red asterisk. ​
Line 57: Line 58:
 ^Parameter^Description^ ^Parameter^Description^
 |//​Name// ​ |Name of discovery rule.  | |//​Name// ​ |Name of discovery rule.  |
-|//​Type// ​ |The type of check to perform discovery; should be //Zabbix agent// ​or //Zabbix agent (active)// for file system ​discovery. ​ | +|//​Type// ​ |The type of check to perform discovery.\\ In this example we are using a //Zabbix agent// ​item key.\\ The discovery rule can also be a [[:manual/config/items/itemtypes/dependent_items|dependent item]], depending on a regular item. It cannot depend on another ​discovery ​rule. For a dependent item, select the respective type (//​Dependent item//) and specify the master item in the '​Master item' field. The master item must exist.  | 
-|//​Key// ​ |An item with "​vfs.fs.discovery" ​key is built into Zabbix agent since version 2.0 on many platforms (see [[manual:​appendix:​items:​supported_by_platform|supported ​item key list]] for details), and will return a JSON with the list of file systems present on the computer and their types. ​ | +|//​Key// ​ |Enter the discovery ​item key (up to 2048 characters).\\ For example, you may use the built-in %%"%%vfs.fs.discovery%%"%% item key to return a JSON with the list of file systems present on the computer and their types.\\ Note that another option for filesystem discovery is using discovery results by the %%"​%%vfs.fs.get%%"​%% agent key, supported since Zabbix 4.4.5 (see [[:​manual/​discovery/​low_level_discovery/​examples/​mounted_filesystems|example]]).  | 
-|//Update interval// ​ |This field specifies how often Zabbix performs discovery. In the beginning, when you are just setting up file system discovery, you might wish to set it to a small interval, but once you know it works you can set it to 30 minutes or more, because file systems usually do not change very often.\\ [[:​manual/​appendix/​suffixes|Time suffixes]] are supported, e.g. 30s, 1m, 2h, 1d, since Zabbix 3.4.0.\\ [[:manual/config/macros/usermacros|User macros]] are supported, since Zabbix 3.4.0.\\ //Note//: If set to '​0', ​the item will not be polled. However, if flexible ​interval ​also exists with a non-zero value, the item will be polled during the flexible ​interval duration.\\ ​ //Note// that for an existing discovery rule the discovery can be performed immediately by pushing the //Check now// [[#​form_buttons|button]]. | +|//Update interval// ​ |This field specifies how often Zabbix performs discovery. In the beginning, when you are just setting up file system discovery, you might wish to set it to a small interval, but once you know it works you can set it to 30 minutes or more, because file systems usually do not change very often.\\ [[:​manual/​appendix/​suffixes|Time suffixes]] are supported, e.g. 30s, 1m, 2h, 1d, since Zabbix 3.4.0.\\ [[manual:config:macros:​user_macros|User macros]] are supported, since Zabbix 3.4.0.\\ //​Note//: ​The update interval can only be set to '​0'​ if custom intervals exist with a non-zero value. ​If set to '​0', ​and custom ​interval ​(flexible or scheduled) ​exists with a non-zero value, the item will be polled during the custom ​interval duration.\\ ​ //Note// that for an existing discovery rule the discovery can be performed immediately by pushing the //Check now// [[#​form_buttons|button]]. | 
-|//Custom intervals// ​ |You can create custom rules for checking the item:\\ **Flexible** - create an exception to the //Update interval// (interval with different frequency)\\ **Scheduling** - create a custom polling schedule.\\ For detailed information see [[manual:​config:​items:​item:​custom_intervals|Custom intervals]]. Scheduling is supported since Zabix 3.0.0. ​ | +|//Custom intervals// ​ |You can create custom rules for checking the item:\\ **Flexible** - create an exception to the //Update interval// (interval with different frequency)\\ **Scheduling** - create a custom polling schedule.\\ For detailed information see [[manual:​config:​items:​item:​custom_intervals|Custom intervals]]. Scheduling is supported since Zabbix ​3.0.0. ​ | 
-|//Keep lost resources period// ​ |This field allows you to specify the duration for how long the discovered entity will be retained (won't be deleted) once its discovery status becomes "Not discovered anymore"​ (min 1 hour, max 25 years).\\ [[:​manual/​appendix/​suffixes|Time suffixes]] are supported, e.g. 2h, 1d, since Zabbix 3.4.0.\\ [[:manual/config/macros/usermacros|User macros]] are supported, since Zabbix 3.4.0.\\ //Note:// If set to "​0",​ entities will be deleted immediately. Using "​0"​ is not recommended,​ since just wrongly editing the filter may end up in the entity being deleted with all the historical data.   |+|//Keep lost resources period// ​ |This field allows you to specify the duration for how long the discovered entity will be retained (won't be deleted) once its discovery status becomes ​%%"%%Not discovered anymore%%"%% (between ​1 hour to 25 years; or %%"​%%0%%"​%%).\\ [[:​manual/​appendix/​suffixes|Time suffixes]] are supported, e.g. 2h, 1d, since Zabbix 3.4.0.\\ [[manual:config:macros:​user_macros|User macros]] are supported, since Zabbix 3.4.0.\\ //Note:// If set to %%"%%0%%"%%, entities will be deleted immediately. Using %%"%%0%%"%% is not recommended,​ since just wrongly editing the filter may end up in the entity being deleted with all the historical data.   |
 |//​Description// ​ |Enter a description. ​ | |//​Description// ​ |Enter a description. ​ |
 |//​Enabled// ​ |If checked, the rule will be processed. ​ | |//​Enabled// ​ |If checked, the rule will be processed. ​ |
Line 67: Line 68:
 <​note>​Discovery rule history is not preserved.</​note>​ <​note>​Discovery rule history is not preserved.</​note>​
  
-== Preprocessing ==+=== Preprocessing ​===
  
 The **Preprocessing** tab allows to define transformation rules to apply to the result of discovery. One or several transformations are possible in this step. Transformations are executed in the order in which they are defined. All preprocessing is done by Zabbix server. The **Preprocessing** tab allows to define transformation rules to apply to the result of discovery. One or several transformations are possible in this step. Transformations are executed in the order in which they are defined. All preprocessing is done by Zabbix server.
  
-{{manual:​discovery:​low_level_discovery:​lld_rule_fs1b.png?600|}}+See also:  
 + 
 +  * [[:​manual/​config/​items/​preprocessing/​preprocessing_details|Preprocessing details]] 
 +  * [[:​manual/​config/​items/​preprocessing#​testing|Preprocessing testing]] 
 + 
 +{{manual:​discovery:​low_level_discovery:​lld_fs_b.png|}}
  
 ^Type^Transformation^Description^ ^Type^Transformation^Description^
Line 77: Line 83:
 ^Text  ^^^ ^Text  ^^^
 |   ​|//​Regular expression// ​ |Match the received value to the <​pattern>​ regular expression and replace value with the extracted <​output>​. The regular expression supports extraction of maximum 10 captured groups with the \N sequence.\\ Parameters:​\\ **pattern** - regular expression\\ **output** - output formatting template. An \N (where N=1…9) escape sequence is replaced with the Nth matched group. A \0 escape sequence is replaced with the matched text.\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ | |   ​|//​Regular expression// ​ |Match the received value to the <​pattern>​ regular expression and replace value with the extracted <​output>​. The regular expression supports extraction of maximum 10 captured groups with the \N sequence.\\ Parameters:​\\ **pattern** - regular expression\\ **output** - output formatting template. An \N (where N=1…9) escape sequence is replaced with the Nth matched group. A \0 escape sequence is replaced with the matched text.\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ |
 +|:::​|//​Replace// ​ |Find the search string and replace it with another (or nothing). All occurrences of the search string will be replaced.\\ Parameters:​\\ **search string** - the string to find and replace, case-sensitive (required)\\ **replacement** - the string to replace the search string with. The replacement string may also be empty effectively allowing to delete the search string when found.\\ It is possible to use escape sequences to search for or replace line breaks, carriage return, tabs and spaces %%"\n \r \t \s"%%; backslash can be escaped as %%"​\\"​%% and escape sequences can be escaped as %%"​\\n"​%%. Escaping of line breaks, carriage return, tabs is automatically done during low-level discovery.\\ Supported since 5.0.0. ​ |
 ^Structured data  ^^^ ^Structured data  ^^^
-|   |//JSON Path//  |Path that is used to extract LLD macro value from a LLD row, using JSONPath syntax.\\ For example, ''​$.foo''​ will extract ​%%"​%%bar%%"​%% and %%"​%%baz%%"​%% ​from this JSON: ''​[{%%"​%%foo%%"​%%:​%%"​%%bar%%"​%%},​ {%%"%%foo%%"%%:​%%"​%%baz%%"​%%}]''​\\ ​JSONPath can be specified using the dot notation or the bracket notationBracket notation should be used in case of any special characters and Unicodelike ''​$['​unicode + special chars #1']['​unicode + special chars #2']''​.\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ |+|   |//JSONPath//  |Extract value or fragment from JSON data using [[:​manual/​config/​items/​preprocessing/​jsonpath_functionality|JSONPath functionality]].\\ If you mark the //Custom on fail// checkbox, the item will not become unsupported in case of failed preprocessing step and it is possible ​to specify custom error handling options: either to discard the value, ​set a specified value or set a specified error message. ​ | 
 +|   ​|//​XML XPath// ​ |Extract value or fragment from XML data using XPath functionality.\\ For this option to workZabbix server must be compiled with libxml support.\\ Examples:​\\ ​''​number(/​document/​item/​value)''​ will extract ​''​10'' ​from ''​<​document><​item><​value>​10</​value></​item></​document>''​\\ ''​number(/​document/​item/​@attribute)''​ will extract ''​10''​ from ''<​nowiki><​document><​item attribute="10"></​item></​document></​nowiki>​''​\\ ​''/​document/​item''​ will extract ''<​item><​value>​10</​value></​item>''​ from ''<​document><​item><​value>​10</​value></​item></​document>''​\\ Note that namespaces are not supported.\\ Supported since 4.4.0.\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard ​the value, set a specified value or set a specified error message | 
 +|   ​|//​CSV to JSON// ​ |Convert CSV file data into JSON format.\\ For more informationsee: [[:​manual/​config/​items/​preprocessing/​csv_to_json#csv_header_processing|CSV to JSON preprocessing]].\\ Supported since 4.4.0. ​ | 
 +|:::|//XML to JSON// ​ |Convert data in XML format to JSON.\\ For more information,​ see: [[:​manual/​config/​items/​preprocessing/​javascript/​javascript_objects#serialization_rules|Serialization rules]].\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ | 
 +^Custom scripts ​ ^^^ 
 +|   ​|//​JavaScript// ​ |Enter JavaScript code in the block that appears when clicking in the parameter field or on //Open//.\\ Note that available JavaScript length depends on the [[:​manual/​config/​items/​item#​custom_script_limit|database used]].\\ For more information,​ see: [[:​manual/​config/​items/​preprocessing/​javascript|Javascript preprocessing]] ​ |
 ^Validation ​ ^^^ ^Validation ​ ^^^
-|   ​|//​Does not match regular expression// ​ |Specify a regular expression that a value must not match.\\ E.g. Pattern: ​''​%%Error:​(.*?​)\.%%'​', Output: ''​%%"​Exiting,​ error found: ​'\1'​."​%%''​. If a match is found, error with message specified in //Output// is returned. If no match, unmodified value is processed further.\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ | +|   ​|//​Does not match regular expression// ​ |Specify a regular expression that a value must not match.\\ E.g. ''​%%Error:​(.*?​)\.%%''​\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ | 
-|:::​|//​Check for error in JSON// ​ |Check for an application-level error message located at JSONpath. Stop processing if succeeded and message is not empty; otherwise continue processing with the value that was before this preprocessing step. Note that these external service errors are reported to user as is, without adding preprocessing step information.\\ E.g. ''​$.errors''​. If a JSON like ''​%%{"​errors":"​e1"​}%%''​ is received, the next preprocessing step will not be executed. ​ |+|:::​|//​Check for error in JSON// ​ |Check for an application-level error message located at JSONpath. Stop processing if succeeded and message is not empty; otherwise continue processing with the value that was before this preprocessing step. Note that these external service errors are reported to user as is, without adding preprocessing step information.\\ E.g. ''​$.errors''​. If a JSON like ''​%%{"​errors":"​e1"​}%%''​ is received, the next preprocessing step will not be executed.\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ | 
 +|:::​|//​Check for error in XML//  |Check for an application-level error message located at xpath. Stop processing if succeeded and message is not empty; otherwise continue processing with the value that was before this preprocessing step. Note that these external service errors are reported to user as is, without adding preprocessing step information.\\ No error will be reported in case of failing to parse invalid XML.\\ Supported since 4.4.0.\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message.  |
 ^Throttling ​ ^^^ ^Throttling ​ ^^^
-|   ​|//​Discard unchanged with heartbeat// ​ |Discard a value if it has not changed within the defined time period (in seconds).\\ Positive integer values are supported to specify the seconds (minimum - 1 second). Time suffixes can be used in this field (e.g. 30s, 1m, 2h, 1d). User macros and low-level discovery macros can be used in this field.\\ Only one throttling option can be specified for a discovery item.\\ E.g. ''​10''​. If identical text is passed into this rule twice within ​10 seconds, ​further processing ​will not be executed.  |+|   ​|//​Discard unchanged with heartbeat// ​ |Discard a value if it has not changed within the defined time period (in seconds).\\ Positive integer values are supported to specify the seconds (minimum - 1 second). Time suffixes can be used in this field (e.g. 30s, 1m, 2h, 1d). User macros and low-level discovery macros can be used in this field.\\ Only one throttling option can be specified for a discovery item.\\ E.g. ''​1m''​. If identical text is passed into this rule twice within ​60 seconds, ​it will be discarded.\\ //Note//: Changing item prototypes does not reset throttling. Throttling is reset only when preprocessing steps are changed. ​ | 
 +^Prometheus ​ ^^^ 
 +|   ​|//​Prometheus to JSON// ​ |Convert required Prometheus metrics to JSON.\\ See [[:​manual/​config/​items/​itemtypes/​prometheus|Prometheus checks]] for more details.  |
  
 Note that if the discovery rule has been applied to the host via template then the content of this tab is read-only. Note that if the discovery rule has been applied to the host via template then the content of this tab is read-only.
  
 +=== Custom macros ===
 +
 +The **LLD macros** tab allows to specify custom low-level discovery macros.
 +
 +Custom macros are useful in cases when the returned JSON does not have the required macros already defined. So, for example:
  
-== Custom ​macros ​==+  * The native ''​vfs.fs.discovery''​ key for filesystem discovery returns a JSON with some pre-defined LLD macros ​such as {#FSNAME}, {#FSTYPE}. These macros can be used in item, trigger prototypes (see subsequent sections of the page) directly; defining custom macros is not needed; 
 +  * The ''​[[:​manual/​discovery/​low_level_discovery/​examples/​mounted_filesystems|vfs.fs.get]]''​ agent item also returns a JSON with filesystem data, but without any pre-defined LLD macros. In this case you may define the macros yourself, and map them to the values in the JSON using JSONPath:
  
-The **LLD macros** tab allows specify low-level ​discovery ​macro mappings with a custom JSONPath, allowing to extract discovery macro values to use in discovered items, triggers, etc. Note that this action will be applied to the result of discovery and the preprocessing applied so far (see above).+{{manual:discovery:​low_level_discovery:​lld_fs_c.png|}}
  
-{{manual:discovery:​low_level_discovery:​lld_rule_fs0c.png|}}+The extracted values can be used in discovered items, triggers, etc. Note that values will be extracted from the result of discovery ​and any preprocessing steps so far.
  
 ^Parameter^Description^ ^Parameter^Description^
 |//LLD macro// ​ |Name of the low-level discovery macro, using the following syntax: {#​MACRO}. ​ | |//LLD macro// ​ |Name of the low-level discovery macro, using the following syntax: {#​MACRO}. ​ |
-|//​JSONPath// ​ |Path that is used to extract LLD macro value from a LLD row, using JSONPath syntax.\\ For example, ''​$.foo''​ will extract %%"​%%bar%%"​%% and %%"​%%baz%%"​%% from this JSON: ''​[{%%"​%%foo%%"​%%:​%%"​%%bar%%"​%%},​ {%%"​%%foo%%"​%%:​%%"​%%baz%%"​%%}]''​\\ The values extracted from the defined ​JSON path are used to replace the LLD macros in item, trigger, etc. prototype fields.\\ JSONPath can be specified using the dot notation or the bracket notation. Bracket notation should be used in case of any special characters and Unicode, like ''​$['​unicode + special chars #​1'​]['​unicode + special chars #​2'​]''​. ​ |+|//​JSONPath// ​ |Path that is used to extract LLD macro value from a LLD row, using JSONPath syntax.\\ For example, ''​$.foo''​ will extract %%"​%%bar%%"​%% and %%"​%%baz%%"​%% from this JSON: ''​[{%%"​%%foo%%"​%%:​%%"​%%bar%%"​%%},​ {%%"​%%foo%%"​%%:​%%"​%%baz%%"​%%}]''​\\ The values extracted from the returned ​JSON are used to replace the LLD macros in item, trigger, etc. prototype fields.\\ JSONPath can be specified using the dot notation or the bracket notation. Bracket notation should be used in case of any special characters and Unicode, like ''​$['​unicode + special chars #​1'​]['​unicode + special chars #​2'​]''​. ​ |
  
-== Filter ==+=== Filter ​===
  
-The **Filters** tab contains discovery rule filter definitions allowing to filter discovery values: ​+A filter can be used to generate real items, triggers, and graphs only for entities that match the criteria. ​The **Filters** tab contains discovery rule filter definitions allowing to filter discovery values: ​
  
-{{manual:​discovery:​low_level_discovery:​lld_rule_fs0d.png?600|}}+{{manual:​discovery:​low_level_discovery:​lld_fs_d.png|}}
  
 ^Parameter^Description^ ^Parameter^Description^
 |//Type of calculation// ​ |The following options for calculating filters are available:​\\ **And** - all filters must be passed;\\ **Or** - enough if one filter is passed;\\ **And/Or** - uses //And// with different macro names and //Or// with the same macro name;\\ **Custom expression** - offers the possibility to define a custom calculation of filters. The formula must include all filters in the list. Limited to 255 symbols. ​ | |//Type of calculation// ​ |The following options for calculating filters are available:​\\ **And** - all filters must be passed;\\ **Or** - enough if one filter is passed;\\ **And/Or** - uses //And// with different macro names and //Or// with the same macro name;\\ **Custom expression** - offers the possibility to define a custom calculation of filters. The formula must include all filters in the list. Limited to 255 symbols. ​ |
-|//​Filters// ​ |filter ​can be used to generate real itemstriggersand graphs only for certain file systemsIt expects ​a [[https://​en.wikipedia.org/​wiki/​Perl_Compatible_Regular_Expressions|Perl Compatible Regular Expression]] (PCRE). For instance, if you are only interested in C:, D:, and E: file systems, you could put {#FSNAME} into "​Macro"​ and <​nowiki>"​^C|^D|^E"</​nowiki>​ regular expression into "​Regular expression"​ text fields. Filtering is also possible by file system types using {#FSTYPE} macro (e.g. <​nowiki>"​^ext|^reiserfs"</​nowiki>​) and by drive types (supported only by Windows agent) using {#​FSDRIVETYPE} macro (e.g., <​nowiki>"​fixed"</​nowiki>​).\\ You can enter a regular expression or reference a global [[manual:​regular_expressions|regular expression]] in "​Regular expression"​ field.\\ In order to test a regular expression you can use "grep -E", for example: <code bash>for f in ext2 nfs reiserfs smbfs; do echo $f | grep -E '​^ext|^reiserfs'​ || echo "SKIP: $f"; done</​code>​{#​FSDRIVETYPE} macro on Windows is supported since Zabbix **3.0.0**.\\ Defining several filters is supported since Zabbix **2.4.0**.\\ Note that if some macro from the filter is missing in the response, the found entity will be ignored.\\ Filter drop-down offers two values to specify whether ​macro matches a regular expression or does not match. |+|//​Filters// ​ | The following ​filter ​condition operators are available: //matches////does not match////exists//, //does not exist//\\  //Matches// and //does not match// operators expect ​a [[https://​en.wikipedia.org/​wiki/​Perl_Compatible_Regular_Expressions|Perl Compatible Regular Expression]] (PCRE). For instance, if you are only interested in C:, D:, and E: file systems, you could put {#FSNAME} into "​Macro"​ and <​nowiki>"​^C|^D|^E"</​nowiki>​ regular expression into "​Regular expression"​ text fields. Filtering is also possible by file system types using {#FSTYPE} macro (e.g. <​nowiki>"​^ext|^reiserfs"</​nowiki>​) and by drive types (supported only by Windows agent) using {#​FSDRIVETYPE} macro (e.g., <​nowiki>"​fixed"</​nowiki>​).\\ You can enter a regular expression or reference a global [[manual:​regular_expressions|regular expression]] in "​Regular expression"​ field.\\ In order to test a regular expression you can use "grep -E", for example: <code bash>for f in ext2 nfs reiserfs smbfs; do echo $f | grep -E '​^ext|^reiserfs'​ || echo "SKIP: $f"; done</​code>​{#​FSDRIVETYPE} macro on Windows is supported since Zabbix **3.0.0**. \\ \\ //Exists// and //does not exist// operators allow to filter entities based on the presence or absense of the specified LLD macro in the response (supported since version 5.4.0). \\ Defining several filters is supported since Zabbix **2.4.0**.\\ Note that if macro from the filter is missing in the response, the found entity will be ignored, unless ​"does not exist" condition is specified for this macro. |
  
-<note warning>​A mistake or typo in the regular expression used in LLD rule may cause deleting thousands of configuration elements, historical values and events ​for many hosts. For example, an incorrect %%"​%%File systems for discovery%%"​%% regular expression may cause deleting ​thousands of items, triggers, historical values and events.</​note>​+<note warning>​A mistake or typo in the regular expression used in the LLD rule (for example, an incorrect %%"​%%File systems for discovery%%"​%% regular expressionmay cause deletion of thousands of configuration elements, historical valuesand events ​for many hosts. </​note>​
  
 <note important>​Zabbix database in MySQL must be created as case-sensitive if file system names that differ only by case are to be discovered correctly.</​note>​ <note important>​Zabbix database in MySQL must be created as case-sensitive if file system names that differ only by case are to be discovered correctly.</​note>​
  
-== Form buttons ​==+=== Override ​=== 
 +The **Override** tab allows setting rules to modify the list of item, trigger, graph and host prototypes or their attributes for discovered objects that meet given criteria. ​
  
-Buttons at the bottom of the form allow to perform several operations.+{{manual:​discovery:​low_level_discovery:​lld_fs_e.png|}}
  
-|{{manual:​config:​button_add.png|}} ​ |Add discovery rule. This button is only available for new discovery rules.  ​| +Overrides (if any) are displayed in reorderable drag-and-drop list and executed in the order in which they are defined.   
-|{{manual:​config:​button_update.png|}} ​ |Update the properties ​of a discovery rule. This button is only available for existing discovery rules. ​ | +To configure details ​of a new override, click on {{:​manual:​config:​add_link.png|}} ​in the //Overrides// blockTo edit an existing overrideclick on the override name. A popup window ​will open allowing ​to edit the override ​rule details.
-|{{manual:config:​button_clone.png|}} ​ |Create another discovery rule based on the properties of the current discovery rule.  | +
-|{{manual:​config:​button_check_now.png|}} ​ ​|Perform discovery based on the discovery rule immediately. The discovery rule must already exist. See [[:manual/config/items/check_now|more details]].\\ //Note// that when performing discovery immediatelyconfiguration cache is not updated, thus the result ​will not reflect very recent changes ​to discovery rule configuration. ​ | +
-|{{manual:​config:​button_delete.png|}} ​ |Delete ​the discovery ​rule.  | +
-|{{manual:​config:​button_cancel.png|}} ​ |Cancel the editing of discovery rule properties. ​ |+
  
-=== Item prototypes ===+{{manual:​discovery:​low_level_discovery:​lld_override.png|}}
  
-Once a rule is created, go to the items for that rule and press "​Create prototype"​ to create an item prototype. Note how macro {#FSNAME} is used where a file system name is required. When the discovery rule is processed, this macro will be substituted ​with the discovered file system.+All mandatory parameters are marked ​with red asterisks
  
-{{manual:discovery:low_level_discovery:​item_prototype_fs1.png|}}+^Parameter ​ ^Description ​ ^  
 +|//​Name// ​ |A unique (per LLD rule) override name. | 
 +|//If filter matches// ​ |Defines whether next overrides should be processed when filter conditions are met\\ **Continue overrides** - subsequent overrides will be processed. \\ **Stop processing** - operations from preceding (if any) and this override will be executed, subsequent overrides will be ignored for matched LLD rows.  | 
 +|//​Filters// ​ | Determines to which discovered entities the override should be applied. Override filters are processed after discovery ​rule [[low_level_discovery#​filter|filters]] and have the same functionality. | 
 +|//​Operations// ​ | Override operations are displayed with these details\\ **Condition** - an object type (item prototype/​trigger prototype/​graph prototype/​host prototype) and a condition to be met (equals/​does not equal/​contains/​does not contain/​matches/​does not match) \\ **Action** - links for editing and removing an operation are displayed |
  
-Low-level discovery [[:​manual/​config/​macros/​lld_macros|macros]] and user [[:​manual/​appendix/​macros/​supported_by_location_user|macros]] may be used in item prototype configuration and item value preprocessing [[:​manual/​config/​items/​item#​item_value_preprocessing|parameters]].+**Configuring an operation**
  
-<​note>​Context-specific escaping ​of low-level discovery macros is performed for safe use in regular expression and XPath preprocessing parameters.</​note>​+To configure details ​of a new operation, click on {{:​manual:​config:​add_link.png|}} ​in the Operations block. To edit an existing operation, click on {{:​manual:​config:​edit_link.png|}} next to the operation. A popup window where you can edit the operation details will open.
  
-Attributes that are specific for item prototypes:+{{manual:discovery:​low_level_discovery:​lld_override_op.png|}}
  
-^Parameter^Description^ +^Parameter ​ ^^^^Description ​ ^  
-|//New application ​prototype// ​ |You may define ​new application ​prototype.\\ ​In application prototypes you can use low-level discovery macros thatafter discovery, will be substituted with real values ​to create applications that are specific ​for the discovered entitySee also [[manual:discovery:low_level_discovery:notes|application discovery notes]] for more specific ​information. ​ | +|//Object// ​ |||| Four types of objects are available: \\ Item prototype ​\\ Trigger prototype \\ Graph prototype \\ Host prtotype ​  | 
-|//Application prototypes//  |Select from the existing application prototypes.  | +|//​Condition//  ||||Allows filtering entities to which the operation should be applied. | 
-|//Create enabled// ​ |If checked ​the item will be added in an enabled state.\\ ​If unchecked, ​the item will be added to a discovered entity, but in a disabled state. ​ |+^ |Operator ​ ||| Supported operators:​\\ **equals** - apply to this prototype \\ **does not equal** - apply to all prototypes, except this \\ **contains** - apply, if prototype name contains this string \\ **does not contain** - apply, if prototype name does not contain this string \\ **matches** - apply, if prototype name matches regular expression \\ **does not match** - apply, if prototype name does not match regular expression \\ | 
 +|:::​|Pattern ​ ||| A [[:​manual/​regular_expressions|regular expression]] or string to search for. | 
 +|  ||||| 
 +|  ^Object: //​Item ​prototype//  |||| 
 +|:::​|//​Create enabled// ​ ||| When the checkbox is marked, the buttons will appear, allowing to override original item prototype settings: \\ //Yes// - the item will be added in an enabled state.\\ //​No// ​the item will be added to a discovered entity but in a disabled state. | 
 +|:::​|//​Discover// ​ |||When the checkbox is markedthe buttons will appearallowing to override original item prototype settings: \\ //Yes// - the item will be added.\\ //No// - the item will not be added. | 
 +|:::​|//​Update interval// ​ |||When the checkbox is marked, two options will appear, allowing ​to set different interval ​for the item: \\ //Delay// - Item update interval. [[manual:config:macros:user_macros|User macros]] and [[:​manual/​appendix/​suffixes|time suffixes]] (e.g. 30s, 1m, 2h, 1d) are supported. Should be set to 0 if //Custom interval// is used. \\ //Custom interval // - click {{:​manual:​config:​add_link.png|}} to specify flexible/​sheduling intervals. For detailed ​information ​see [[manual:​config:​items:​item:​custom_intervals|Custom intervals]]
 +|:::​|//​History storage period// ​ |||When the checkbox is marked, the buttons will appear, allowing to set different history storage period for the item: \\ //Do not keep history// - if selected, the history will not be stored. \\ //Storage period// - if selected, an input field for specifying storage period will appear to the right. [[manual:​config:​macros:​user_macros|User macros]] and [[manual/​config/​macros/​lld_macros|LLD macros]] are supported. ​
 +|:::|//Trend storage period//  |||When ​the checkbox is marked, the buttons will appear, allowing to set different trend storage period for the item: \\ //Do not keep trends// - if selected, the trends will not be stored\\ //Storage period// - if selected, an input field for specifying storage period will appear to the right. [[manual:​config:​macros:​user_macros|User macros]] and [[manual/​config/​macros/​lld_macros|LLD macros]] are supported. | 
 + ||||
 +|   ​^Object:​ //Trigger prototype// ​ |||| 
 +|:::|//Create enabled// ​ ||| When the checkbox is marked, the buttons will appear, allowing to override original trigger prototype settings: \\ //Yes// - the trigger ​will be added in an enabled state.\\ ​//No// - the trigger ​will be added to a discovered entity, but in a disabled state. ​
 +|:::​|//​Discover// ​ |||When the checkbox is marked, the buttons will appear, allowing to override original trigger prototype settings: \\ //Yes// - the trigger will be added.\\ //No// - the trigger will not be added. | 
 +|:::​|//​Severity// ​ |||When the checkbox is marked, trigger severity buttons will appear, allowing to modify trigger severity. | 
 +|:::​|//​Tags// ​ |||When the checkbox is marked, a new block will appear, allowing to specify tag-value pairs. \\ All trigger prototype tags will be replaced by tags from this override. | 
 +|  ||||| 
 +|   ​^Object:​ //Graph prototype// ​ |||| 
 +|:::​|//​Discover// ​ |||When the checkbox is marked, the buttons will appear, allowing to override original graph prototype settings: \\ //Yes// - the graph will be added.\\ //No// - the graph will not be added. | 
 +|  ||||| 
 +|   ​^Object:​ //Host prototype// ​ |||| 
 +|:::​|//​Create enabled// ​ ||| When the checkbox is marked, the buttons will appear, allowing to override original host prototype settings: \\ //Yes// - the host will be created in an enabled state.\\ //No// - the host will be created in a disabled state. | 
 +|:::​|//​Discover// ​ |||When the checkbox is marked, the buttons will appear, allowing to override original host prototype settings: \\ //Yes// - the host will be discovered.\\ //No// - the host will not be discovered. | 
 +|:::|//Link templates// ​ |||When the checkbox is marked, an input field for specifying templates will appear. Start typing the template name or click on //Select// next to the field and select templates from the list in a popup window. \\ All templates linked to a host prototype will be replaced by templates from this override. | 
 +|:::​|//​Tags// ​ |||When the checkbox is marked, a new block will appear, allowing to specify tag-value pairs. \\ All host prototype tags will be replaced by tags from this override. | 
 +|:::|//Host inventory// ​ |||When the checkbox is marked, the buttons will appear, allowing to select different inventory [[:​manual/​config/​hosts/​inventory|mode]] for the host prototype: \\ //​Disabled//​ - do not populate host inventory \\ //Manual// - provide details manually \\ //​Automated//​ - auto-fill host inventory data based on collected metrics. ​|
  
-We can create several item prototypes for each file system metric we are interested in:+=== Form buttons ===
  
-{{manual:​discovery:​low_level_discovery:​item_prototypes_fs.png|}}+Buttons at the bottom of the form allow to perform several operations.
  
-//[[:manual/config/​items/​itemupdate#​using_mass_update|Mass update]]// option ​is available ​if you want to update properties of several item prototypes at once+|{{manual:config:​button_add.png|}}  |Add a discovery rule. This button ​is only available ​for new discovery rules | 
- +|{{manual:config:button_update.png|}} ​ ​|Update the properties of a discovery rule. This button is only available ​for existing discovery rules. ​ | 
-=== Trigger prototypes === +|{{manual:​config:​button_clone.png|}} ​ |Create another discovery rule based on the properties of the current discovery rule.  | 
- +|{{manual:​config:​button_check_now.png|}} ​ |Perform discovery based on the discovery rule immediatelyThe discovery ​rule must already existSee [[:​manual/​config/​items/check_now|more details]].\\ //Note// that when performing discovery immediately,​ configuration cache is not updated, thus the result will not reflect very recent changes to discovery rule configuration | 
-We create trigger prototypes in a similar way as item prototypes:​ +|{{manual:config:button_delete.png|}} ​ ​|Delete the discovery ​rule 
- +|{{manual:config:button_cancel.png|}} ​ ​|Cancel the editing of discovery rule properties |
-{{manual:discovery:​low_level_discovery:trigger_prototype_fs.png|}} +
- +
-Attributes that are specific ​for trigger prototypes:​ +
- +
-^Parameter^Description^ +
-|//Create enabled// ​ |If checked ​the trigger will be added in an enabled state.\\ If unchecked, ​the trigger will be added to a discovered entity, but in a disabled state.  | +
- +
-When real triggers are created from the prototypes, there may be a need to be flexible as to what constant ('​20'​ in our example) is used for comparison in the expressionSee how [[:manual/discovery/​low_level_discovery#​using_lld_macros_in_user_macro_contexts|user macros with context]] can be useful to accomplish such flexibility. +
- +
-You can define ​[[:​manual/​config/​triggers/dependencies|dependencies]] between trigger prototypes as well (supported since Zabbix 3.0). To do that, go to the //Dependencies// tab. A trigger prototype may depend on another trigger prototype from the same low-level ​discovery ​(LLD) rule or on a regular trigger. A trigger prototype may not depend on a trigger prototype from a different LLD rule or on a trigger created from trigger prototype. Host trigger prototype cannot depend on a trigger from a template+
- +
-{{manual:discovery:​low_level_discovery:trigger_prototypes_fs.png|}} +
- +
-=== Graph prototypes === +
- +
-We can create graph prototypes, too: +
- +
-{{manual:discovery:​low_level_discovery:​graph_prototype_fs.png|}} +
- +
-{{manual:discovery:​low_level_discovery:graph_prototypes_fs.png|}} +
- +
-Finally, we have created a discovery rule that looks like shown belowIt has five item prototypes, two trigger prototypes, and one graph prototype. +
- +
-{{manual:​discovery:​low_level_discovery:​lld_rules_fs.png|}}+
  
-//Note//: For configuring host prototypes, see the section about [[:​manual/​vm_monitoring#​host_prototypes|host prototype]] configuration in virtual machine monitoring. 
  
-=== Discovered entities ===+==== Discovered entities ​====
  
 The screenshots below illustrate how discovered items, triggers, and graphs look like in the host's configuration. Discovered entities are prefixed with an orange link to a discovery rule they come from. The screenshots below illustrate how discovered items, triggers, and graphs look like in the host's configuration. Discovered entities are prefixed with an orange link to a discovery rule they come from.
Line 183: Line 204:
 {{manual:​discovery:​low_level_discovery:​discovered_items1.png|}} {{manual:​discovery:​low_level_discovery:​discovered_items1.png|}}
  
-Note that discovered entities will not be created in case there are already existing entities with the same uniqueness criteria, for example, an item with the same key or graph with the same name. An error message is displayed in this case that the low-level discovery rule could not create certain entities.+Note that discovered entities will not be created in case there are already existing entities with the same uniqueness criteria, for example, an item with the same key or graph with the same name. An error message is displayed in this case in the frontend ​that the low-level discovery rule could not create certain ​entities. The discovery rule itself, however, will not turn unsupported because some entity could not be created and had to be skipped. The discovery rule will go on creating/​updating other entities.
  
 Items (similarly, triggers and graphs) created by a low-level discovery rule will be deleted automatically if a discovered entity (file system, interface, etc) stops being discovered (or does not pass the filter anymore). In this case the items, triggers and graphs will be deleted after the days defined in the //Keep lost resources period// field pass. Items (similarly, triggers and graphs) created by a low-level discovery rule will be deleted automatically if a discovered entity (file system, interface, etc) stops being discovered (or does not pass the filter anymore). In this case the items, triggers and graphs will be deleted after the days defined in the //Keep lost resources period// field pass.
Line 199: Line 220:
 {{manual:​discovery:​low_level_discovery:​discovered_graphs1.png|}} {{manual:​discovery:​low_level_discovery:​discovered_graphs1.png|}}
  
-=== Other types of discovery ===+==== Other types of discovery ​====
  
 More detail and how-tos on other types of out-of-the-box discovery is available in the following sections: More detail and how-tos on other types of out-of-the-box discovery is available in the following sections:
  
-  * discovery of [[:​manual/​discovery/​low_level_discovery/​network_interfaces|network interfaces]];​ +  * discovery of [[:​manual/​discovery/​low_level_discovery/examples/​network_interfaces|network interfaces]];​ 
-  * discovery of [[:​manual/​discovery/​low_level_discovery/​cpu|CPUs and CPU cores]]; +  * discovery of [[:​manual/​discovery/​low_level_discovery/examples/cpu|CPUs and CPU cores]]; 
-  * discovery of [[:​manual/​discovery/​low_level_discovery/​snmp_oids|SNMP OIDs]]; +  * discovery of [[:​manual/​discovery/​low_level_discovery/examples/​snmp_oids|SNMP OIDs]]; 
-  * discovery of [[:​manual/​discovery/​low_level_discovery/​jmx|JMX objects]];​ +  * discovery of [[:​manual/​discovery/​low_level_discovery/examples/jmx|JMX objects]];​ 
-  * discovery using [[:​manual/​discovery/​low_level_discovery/​sql_queries|ODBC SQL queries]];​ +  * discovery using [[:​manual/​discovery/​low_level_discovery/examples/​sql_queries|ODBC SQL queries]];​ 
-  * discovery of [[:​manual/​discovery/​low_level_discovery/​windows_services|Windows services]];​ +  * discovery of [[:​manual/​discovery/​low_level_discovery/examples/​windows_services|Windows services]];​ 
-  * discovery of [[:​manual/​discovery/​low_level_discovery/​host_interfaces|host interfaces]] in Zabbix.+  * discovery of [[:​manual/​discovery/​low_level_discovery/examples/​host_interfaces|host interfaces]] in Zabbix.
  
 For more detail on the JSON format for discovery items and an example of how to implement your own file system discoverer as a Perl script, see [[#​creating_custom_lld_rules|creating custom LLD rules]]. For more detail on the JSON format for discovery items and an example of how to implement your own file system discoverer as a Perl script, see [[#​creating_custom_lld_rules|creating custom LLD rules]].
  
-=== Data limits for return values === 
  
-There is no limit for low-level discovery rule JSON data if it is received directly by Zabbix server, because return values are processed without being stored in a database. There'​s also no limit for custom low-level discovery rules, however, if it is intended to acquire custom LLD data using a user parameter, then user parameter return value limit applies (512 KB). +==== Creating custom LLD rules ====
- +
-If data has to go through Zabbix proxy it has to store this data in database so [[:​manual/​config/​items/​item#​text_data_limits|database limits]] apply, for example, 2048 bytes on a Zabbix proxy run with IBM DB2 database. +
- +
-=== Multiple LLD rules for same item === +
- +
-Since Zabbix agent version 3.2 it is possible to define several low-level discovery rules with the same discovery item.  +
- +
-To do that you need to define the Alias agent [[manual/​appendix/​config/​zabbix_agentd|parameter]],​ allowing to use altered discovery item keys in different discovery rules, for example ''​vfs.fs.discovery[foo]'',​ ''​vfs.fs.discovery[bar]'',​ etc. +
-=== Creating custom LLD rules ===+
  
 It is also possible to create a completely custom LLD rule, discovering any type of entities - for example, databases on a database server. It is also possible to create a completely custom LLD rule, discovering any type of entities - for example, databases on a database server.
Line 324: Line 335:
  
 Note that, if using a user parameter, the return value is limited to 512 KB. For more details, see [[:​manual/​discovery/​low_level_discovery#​data_limits_for_return_values|data limits for LLD return values]]. Note that, if using a user parameter, the return value is limited to 512 KB. For more details, see [[:​manual/​discovery/​low_level_discovery#​data_limits_for_return_values|data limits for LLD return values]].
- 
-=== Using LLD macros in user macro contexts === 
- 
-User macros [[:​manual/​config/​macros/​usermacros#​user_macro_context|with context]] can be used to accomplish more flexible thresholds in trigger expressions. Different thresholds may be defined on user macro level and then used in trigger constants depending on the discovered context. Discovered context appears when the [[:​manual/​config/​macros/​lld_macros|low-level discovery macros]] used in the macros are resolved to real values. 
- 
-To illustrate we can use data from the example above and assume that the following file systems will be discovered: ''/'',​ ''/​home'',​ ''/​tmp'',​ ''/​usr'', ​ ''/​var''​. 
- 
-We may define a free-disk-space trigger prototype for a host, where the threshold is expressed by a user macro with context: 
- 
-''​{host:​vfs.fs.size[{#​FSNAME},​pfree].last()}<​**{$LOW_SPACE_LIMIT:<​nowiki>"</​nowiki>​{#​FSNAME}<​nowiki>"</​nowiki>​}**''​ 
- 
-Then add user macros: 
-  * ''​{$LOW_SPACE_LIMIT}''​ **10** 
-  * ''​{$LOW_SPACE_LIMIT:/​home}''​ **20** 
-  * ''​{$LOW_SPACE_LIMIT:/​tmp}''​ **50** 
- 
-Now, once the file systems are discovered, events will be generated if ''/'',​ ''/​usr''​ and ''/​var''​ filesystems have less than **10**% of free disk space, the ''/​home''​ filesystem - less than **20**% of free disk space or the ''/​tmp''​ filesystem - less than **50**% of free disk space.