Zabbix Documentation 5.0

3.04.04.4 (current)| In development:5.0 (devel)| Unsupported:1.82.02.22.43.23.44.2Guidelines

User Tools

Site Tools


manual:discovery:low_level_discovery

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
Next revision Both sides next revision
manual:discovery:low_level_discovery [2015/06/15 06:35]
Igors H [3.6 Discovery of Windows services] added info about windows services
manual:discovery:low_level_discovery [2019/03/28 13:45]
martins-v formatting fix
Line 1: Line 1:
-==== - #3 Low-level discovery ====+==== 3 Low-level discovery ====
  
 === Overview === === Overview ===
  
 Low-level discovery provides a way to automatically create items, triggers, and graphs for different entities on a computer. For instance, Zabbix can automatically start monitoring file systems or network interfaces on your machine, without the need to create items for each file system or network interface manually. Additionally it is possible to configure Zabbix to remove unneeded entities automatically based on actual results of periodically performed discovery. Low-level discovery provides a way to automatically create items, triggers, and graphs for different entities on a computer. For instance, Zabbix can automatically start monitoring file systems or network interfaces on your machine, without the need to create items for each file system or network interface manually. Additionally it is possible to configure Zabbix to remove unneeded entities automatically based on actual results of periodically performed discovery.
- 
-In Zabbix, five types of item discovery are supported out of the box: 
- 
-  * discovery of file systems; 
-  * discovery of network interfaces; 
-  * discovery of CPUs and CPU cores; 
-  * discovery of SNMP OIDs; 
-  * discovery using ODBC SQL queries. 
-  * discovery of Windows services 
  
 A user can define their own types of discovery, provided they follow a particular JSON protocol. A user can define their own types of discovery, provided they follow a particular JSON protocol.
Line 20: Line 11:
 First, a user creates a discovery rule in "​Configuration"​ -> "​Templates"​ -> "​Discovery"​ column. A discovery rule consists of (1) an item that discovers the necessary entities (for instance, file systems or network interfaces) and (2) prototypes of items, triggers, and graphs that should be created based on the value of that item. First, a user creates a discovery rule in "​Configuration"​ -> "​Templates"​ -> "​Discovery"​ column. A discovery rule consists of (1) an item that discovers the necessary entities (for instance, file systems or network interfaces) and (2) prototypes of items, triggers, and graphs that should be created based on the value of that item.
  
-An item that discovers the necessary entities is like a regular item seen elsewhere: the server asks a Zabbix agent (or whatever the type of the item is set to) for a value of that item, the agent responds with a textual value. The difference is that the value the agent responds with should contain a list of discovered entities in a specific ​JSON format. While the details of this format are only important for implementers of custom discovery checks, it is necessary to know that the returned value contains a list of macro -> value pairs. For instance, item "​net.if.discovery"​ might return two pairs: "​{#​IFNAME}"​ -> "​lo"​ and "​{#​IFNAME}"​ -> "​eth0"​.+An item that discovers the necessary entities is like a regular item seen elsewhere: the server asks a Zabbix agent (or whatever the type of the item is set to) for a value of that item, the agent responds with a textual value. The difference is that the value the agent responds with should contain a list of discovered entities in a JSON format. While the details of this format are only important for implementers of custom discovery checks, it is necessary to know that the returned value contains a list of macro -> value pairs. For instance, item "​net.if.discovery"​ might return two pairs: "​{#​IFNAME}"​ -> "​lo"​ and "​{#​IFNAME}"​ -> "​eth0"​.
  
-<​note>​Low-level discovery items "​vfs.fs.discovery" ​and "​net.if.discovery" ​are supported since Zabbix agent version 2.0.\\ Discovery item "​system.cpu.discovery"​ is supported since Zabbix agent version 2.4.\\ Discovery ​of SNMP OIDs is supported since Zabbix server and proxy version 2.0.\\ Discovery ​using ODBC SQL queries is supported since Zabbix server and proxy version 3.0.</​note>​+These macros are used in names, keys and other prototype fields where they are then substituted with the received values for creating real items, triggers, graphs or even hosts for each discovered entitySee the full list of [[:​manual/​config/​macros/​lld_macros|options]] for using LLD macros.
  
-<​note>​Return values of low-level ​discovery ​rule are limited to 2048 bytes on a Zabbix proxy run with IBM DB2 databaseThis limit does not apply to Zabbix ​server ​as return values are processed without being stored in a database.</​note>​+When the server receives a value for a discovery ​item, it looks at the macro -> value pairs and for each pair generates real items, triggers, and graphs, based on their prototypes. In the example ​with "net.if.discovery"​ above, the server ​would generate one set of items, triggers, and graphs for the loopback interface "​lo",​ and another set for interface "​eth0"​.
  
-These macros are then used in nameskeys, and other prototype fields ​that are basis for creating real items, triggers, and graphs for each discovered entityThese macros can be used:+Note that since **Zabbix 4.2**the format of the JSON returned by low-level discovery rules has been changed. It is no longer expected ​that the JSON will contain the %%"​%%data%%"​%% object. Low-level discovery will now accept a normal JSON containing an arrayin order to support new features such as the item value preprocessing ​and custom paths to low-level discovery macro values in a JSON document.
  
-  * for item prototypes ​in +Built-in discovery keys have been updated to return an array of LLD rows at the root of JSON document. Zabbix will automatically extract a macro and value if an array field uses the {#MACRO} syntax ​as key. Any new native discovery checks will use the new syntax without the %%"​%%data%%"​%% elementsWhen processing a low-level discovery value first the root is located ​(array at ''​$.''​ or ''​$.data''​)
-    * names +
-    * key parameters +
-    * IPMI sensor +
-    * SNMP OIDs +
-    * units +
-    * calculated item formulas +
-    * SSH and Telnet scripts +
-    * database monitor item parameters +
-    * descriptions (supported since 2.2.0) +
-  * for trigger prototypes in +
-    * names +
-    * expressions (insofar ​as when referencing an item key prototype and as standalone constants)  +
-    * URLs (supported since 3.0.0) +
-    * descriptions ​(supported since 2.2.0) +
-  * for graph prototypes in +
-    * names+
  
-When the server receives a value for a discovery ​itemit looks at the macro -> value pairs and for each pair generates real items, triggers, and graphsbased on their prototypesIn the example ​with "net.if.discovery" ​above, the server would generate one set of items, triggers, and graphs for the loopback interface "​lo"​and another set for interface "​eth0"​.+While the %%"​%%data%%"​%% element has been removed from all native items related to discovery, for backward compatibility Zabbix will still accept the JSON notation with a %%"​%%data%%"​%% elementthough its use is discouragedIf the JSON contains an object ​with only one %%"%%data%%"%% array elementthen it will automatically extract ​the content ​of the element using JSONPath ''​$.data''​. Low-level discovery now accepts optional user-defined LLD macros with a custom path specified in JSONPath syntax. 
 + 
 +<note warning>​As a result of the changes abovenewer agents no longer will be able to work with an older Zabbix server.</​note>​
  
-The following sections illustrate the process described above in detail and serve as a how-to for performing all types of discovery ​mentioned above. The last section describes the JSON format for discovery items and gives an example of how to implement your own file system discoverer as a Perl script.+See also: [[#​discovered_entities|Discovered entities]] 
 +=== Configuring low-level discovery ​===
  
-=== Discovery ​of file systems ===+We will illustrate low-level discovery based on an example ​of file system discovery.
  
-To configure the discovery ​of file systems, do the following:+To configure the discovery, do the following:
  
   * Go to: //​Configuration//​ -> //​Templates// ​   * Go to: //​Configuration//​ -> //​Templates// ​
Line 60: Line 38:
  
   * Click on //Create discovery rule// in the upper right corner of the screen   * Click on //Create discovery rule// in the upper right corner of the screen
-  * Fill in the form with the following ​details+  * Fill in the discovery rule form with the required ​details
  
-The **Discovery rule** tab contains general discovery rule attributes: ​+=== Discovery rule ===
  
-{{manual:​discovery:​low_level_discovery:​lld_rule_fs.png|}}+The discovery rule form contains four tabs, representing,​ from left to right, the data flow during discovery:​ 
 + 
 +  * //Discovery rule// - specifies, most importantly,​ the built-in item or custom script to retrieve discovery data 
 +  * //​Preprocessing//​ - applies some preprocessing to the discovered data 
 +  * //LLD macros// - allows to extract some macro values to use in discovered items, triggers, etc 
 +  * //Filters// - allows to filter the discovered values 
 + 
 +The **Discovery rule** tab contains the item key to use for discovery (as well as some general discovery rule attributes):​  
 + 
 +{{manual:​discovery:​low_level_discovery:​lld_rule_fs0.png?600|}} 
 + 
 +All mandatory input fields are marked with a red asterisk. ​
  
 ^Parameter^Description^ ^Parameter^Description^
 |//​Name// ​ |Name of discovery rule.  | |//​Name// ​ |Name of discovery rule.  |
-|//​Type// ​ |The type of check to perform discovery; should be //Zabbix agent// or //Zabbix agent (active)// for file system discovery. ​ | +|//​Type// ​ |The type of check to perform discovery; should be //Zabbix agent// or //Zabbix agent (active)// for file system discovery.\\ The discovery rule can also be a [[:​manual/​config/​items/​itemtypes/​dependent_items|dependent item]], depending on a regular item. It cannot depend on another discovery rule. For a dependent item, select the respective type (//​Dependent item//) and specify the master item in the '​Master item' field. The master item must exist.  | 
-|//​Key// ​ |An item with "​vfs.fs.discovery"​ key is built into the Zabbix agent on many platforms (see [[manual:​appendix:​items:​supported_by_platform|supported item key list]] for details), and will return a JSON with the list of file systems present on the computer and their types. ​ | +|//​Key// ​ |An item with "​vfs.fs.discovery"​ key is built into Zabbix agent on many platforms (see [[manual:​appendix:​items:​supported_by_platform|supported item key list]] for details), and will return a JSON with the list of file systems present on the computer and their types. ​ | 
-|//Update interval ​(in sec)//  |This field specifies how often Zabbix performs discovery. In the beginning, when you are just setting up file system discovery, you might wish to set it to a small interval, but once you know it works you can set it to 30 minutes or more, because file systems usually do not change very often.\\ //Note//: If set to '​0',​ the item will not be polled. However, if a flexible interval also exists with a non-zero value, the item will be polled during the flexible interval duration. ​ | +|//Update interval// ​ |This field specifies how often Zabbix performs discovery. In the beginning, when you are just setting up file system discovery, you might wish to set it to a small interval, but once you know it works you can set it to 30 minutes or more, because file systems usually do not change very often.\\ [[:​manual/​appendix/​suffixes|Time suffixes]] are supported, e.g. 30s, 1m, 2h, 1d, since Zabbix 3.4.0.\\ [[:​manual/​config/​macros/​usermacros|User macros]] are supported, since Zabbix 3.4.0.\\ //Note//: If set to '​0',​ the item will not be polled. However, if a flexible interval also exists with a non-zero value, the item will be polled during the flexible interval duration.\\  //Note// that for an existing discovery rule the discovery can be performed immediately by pushing the //Check now// [[#​form_buttons|button]]. ​
-|//Flexible ​intervals// ​ |You can create ​exceptions ​to //Update interval//. For example:\\ Interval: ​**0**, Period: **6-7,​00:​00-24:​00** - will disable the polling ​at the weekend. Otherwise default update interval will be used.\\ Up to seven flexible intervals can be defined.\\ If multiple flexible intervals overlap, the smallest //​Interval//​ value is used for the overlapping period.\\ See [[manual:appendix:time_period|Time period specification]] page for description of the //Period// format.\\ //Note//: If set to '0', the item will not be polled during the flexible interval duration and will resume polling according to the //Update interval// once the flexible interval period is over.  | +|//Custom ​intervals// ​ |You can create ​custom rules for checking the item:\\ **Flexible** - create an exception ​to the //Update interval// ​(interval with different frequency)\\ **Scheduling** - create a custom ​polling ​schedule.\\ For detailed information see [[manual:config:items:​item:​custom_intervals|Custom intervals]]. Scheduling is supported since Zabix 3.0.0.  | 
-|//Keep lost resources period ​(in days)//  |This field allows you to specify for how many days the discovered entity will be retained (won't be deleted) once its discovery status becomes "Not discovered anymore"​ (max 3650 days). \\ //Note:// If set to "​0",​ entities will be deleted immediately. Using "​0"​ is not recommended,​ since just wrongly editing the filter may end up in the entity being deleted with all the historical data.   |+|//Keep lost resources period// ​ |This field allows you to specify ​the duration ​for how long the discovered entity will be retained (won't be deleted) once its discovery status becomes ​%%"%%Not discovered anymore%%"%% (between 1 hour to 25 years; or %%"​%%0%%"​%%).\\ [[:​manual/​appendix/​suffixes|Time suffixes]] are supported, e.g. 2h, 1d, since Zabbix 3.4.0.\\ [[:​manual/​config/​macros/​usermacros|User macros]] are supported, since Zabbix 3.4.0.\\ //Note:// If set to %%"%%0%%"%%, entities will be deleted immediately. Using %%"%%0%%"%% is not recommended,​ since just wrongly editing the filter may end up in the entity being deleted with all the historical data.   |
 |//​Description// ​ |Enter a description. ​ | |//​Description// ​ |Enter a description. ​ |
 |//​Enabled// ​ |If checked, the rule will be processed. ​ | |//​Enabled// ​ |If checked, the rule will be processed. ​ |
  
-The **Filters** tab contains discovery ​rule filter definitions: ​+<​note>​Discovery ​rule history is not preserved.</​note>​
  
-{{manual:​discovery:​low_level_discovery:​lld_rule_fs2.png|}}+== Preprocessing ==
  
-^Parameter^Description^ +The **Preprocessing** tab allows ​to define ​transformation rules to apply to the result of discoveryOne or several transformations ​are possible ​in this stepTransformations are executed ​in the order in which they are definedAll preprocessing ​is done by Zabbix ​server.
-|//Type of calculation// ​ |The following options for calculating filters are available:​\\ ​**And** - all filters must be passed;\\ **Or** - enough if one filter is passed;\\ **And/Or** - uses //And// with different macro names and //Or// with the same macro name;\\ **Custom expression** - offers the possibility ​to define ​a custom calculation of filters. The formula must include all filters in the list. Limited ​to 255 symbols. ​ | +
-|//​Filters// ​ |A filter can be used to generate real items, triggers, and graphs only for certain file systemsIt expects a [[manual:​regular_expressions|POSIX Extended Regular Expression]]. For instance, if you are only interested ​in C:, D:, and E: file systems, you could put {#FSNAME} into "​Macro"​ and <​nowiki>"​^C|^D|^E"</​nowiki>​ regular expression into "​Regular expression"​ text fieldsFiltering is also possible by file system types using {#FSTYPE} macro (e.g. <​nowiki>"​^ext|^reiserfs"</​nowiki>​) and by drive types (supported only by Windows agent) using {#​FSDRIVETYPE} macro (e.g., <​nowiki>"​fixed"</​nowiki>​).\\ You can enter a regular expression or reference a global [[manual:​regular_expressions|regular expression]] ​in "​Regular expression"​ field.\\ In order to test a regular expression you can use "grep -E", for example: <code bash>for f in ext2 nfs reiserfs smbfs; do echo $f | grep -E '​^ext|^reiserfs'​ || echo "SKIP: $f"; done</​code>​{#​FSDRIVETYPE} macro on Windows is supported since Zabbix **3.0.0**.\\ Defining several filters ​is supported since Zabbix ​**2.4.0**.\\ Note that if some macro from the filter is missing in the response, the found entity will be ignored |+
  
-<note important>​Zabbix database in MySQL must be created as case-sensitive if file system names that differ only by case are to be discovered correctly.</​note>​+{{manual:​discovery:​low_level_discovery:​lld_rule_fs2b.png?600|}}
  
-<note>Discovery rule history ​is not preserved.</note>+^Type^Transformation^Description^ 
 +|  ||| 
 +^Text  ^^^ 
 +|   ​|//​Regular expression// ​ |Match the received value to the <patternregular expression and replace value with the extracted <​output>​. The regular expression supports extraction of maximum 10 captured groups with the \N sequence.\\ Parameters:​\\ **pattern** - regular expression\\ **output** - output formatting template. An \N (where N=1…9) escape sequence ​is replaced with the Nth matched group. A \0 escape sequence is replaced with the matched text.\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ | 
 +^Structured data  ^^^ 
 +|   ​|//​JSONPath// ​ |Path that is used to extract LLD macro value from a LLD row, using JSONPath syntax.\\ For example, ''​$.foo''​ will extract %%"​%%bar%%"​%% and %%"​%%baz%%"​%% from this JSON: ''​[{%%"​%%foo%%"​%%:​%%"​%%bar%%"​%%},​ {%%"​%%foo%%"​%%:​%%"​%%baz%%"​%%}]''​\\ JSONPath can be specified using the dot notation or the bracket notation. Bracket notation should be used in case of any special characters and Unicode, like ''​$['​unicode + special chars #​1'​]['​unicode + special chars #​2'​]''​.\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ | 
 +^Custom scripts ​ ^^^ 
 +|   ​|//​JavaScript// ​ |Enter JavaScript code in the block that appears when clicking in the parameter field or on //Open//.\\ Note that available JavaScript length depends on the [[:​manual/​config/​items/​item#​custom_script_limit|database used]]. ​ | 
 +^Validation ​ ^^^ 
 +|   ​|//​Does ​not match regular expression// ​ |Specify a regular expression that a value must not match.\\ E.g. ''​%%Error:​(.*?​)\.%%''​\\ If you mark the //Custom on fail// checkbox, it is possible to specify custom error handling options: either to discard the value, set a specified value or set a specified error message. ​ | 
 +|:::​|//​Check for error in JSON// ​ |Check for an application-level error message located at JSONpath. Stop processing if succeeded and message is not empty; otherwise continue processing with the value that was before this preprocessing step. Note that these external service errors are reported to user as is, without adding preprocessing step information.\\ E.g. ''​$.errors''​. If a JSON like ''​%%{"​errors":"​e1"​}%%''​ is received, the next preprocessing step will not be executed. ​ | 
 +^Throttling ​ ^^^ 
 +|   ​|//​Discard unchanged with heartbeat// ​ |Discard a value if it has not changed within the defined time period (in seconds).\\ Positive integer values are supported to specify the seconds (minimum - 1 second). Time suffixes can be used in this field (e.g. 30s, 1m, 2h, 1d). User macros and low-level discovery macros can be used in this field.\\ Only one throttling option can be specified for a discovery item.\\ E.g. ''​1m''​. If identical text is passed into this rule twice within 60 seconds, it will be discarded.\\ //Note//: Changing item prototypes does not reset throttling. Throttling is reset only when preprocessing steps are changed. ​ | 
 +^Prometheus ​ ^^^ 
 +|   ​|//​Prometheus to JSON// ​ |Convert required Prometheus metrics to JSON.\\ See [[:​manual/​config/​items/​itemtypes/​prometheus|Prometheus checks]] for more details. ​ |
  
-Once a rule is created, go to the items for that rule and press "​Create prototype" ​to create an item prototype. Note how macro {#FSNAME} is used where a file system name is required. When the discovery rule is processed, this macro will be substituted with the discovered file system.+Note that if the discovery ​rule has been applied ​to the host via template then the content of this tab is read-only.
  
-{{manual:​discovery:​low_level_discovery:​item_prototype_fs.png|}} 
  
-<​note>​If an item prototype is created with a //​Disabled//​ status, it will be added to a discovered entity, but in a disabled state.</​note>​+== Custom macros ==
  
-We can create several item prototypes for each file system metric we are interested ​in:+The **LLD macros** tab allows specify low-level discovery macro mappings with a custom JSONPath, allowing to extract discovery macro values to use in discovered items, triggers, etc. Note that this action will be applied to the result of discovery and the preprocessing applied so far (see above).
  
-{{manual:​discovery:​low_level_discovery:​item_prototypes_fs.png|}}+{{manual:​discovery:​low_level_discovery:​lld_rule_fs0c.png|}}
  
-Thenwe create trigger prototypes in similar way:+^Parameter^Description^ 
 +|//LLD macro// ​ |Name of the low-level discovery macrousing the following syntax: {#​MACRO}. ​ | 
 +|//​JSONPath// ​ |Path that is used to extract LLD macro value from LLD row, using JSONPath syntax.\\ For example, ''​$.foo''​ will extract %%"​%%bar%%"​%% and %%"​%%baz%%"​%% from this JSON''​[{%%"​%%foo%%"​%%:​%%"​%%bar%%"​%%},​ {%%"​%%foo%%"​%%:​%%"​%%baz%%"​%%}]''​\\ The values extracted from the defined JSON path are used to replace the LLD macros in item, trigger, etc. prototype fields.\\ JSONPath can be specified using the dot notation or the bracket notation. Bracket notation should be used in case of any special characters and Unicode, like ''​$['​unicode + special chars #​1'​]['​unicode + special chars #​2'​]''​. ​ |
  
-{{manual:​discovery:​low_level_discovery:​trigger_prototype_fs.png|}}+== Filter ==
  
-{{manual:discovery:low_level_discovery:​trigger_prototypes_fs.png|}}+The **Filters** tab contains ​discovery ​rule filter definitions allowing to filter discovery values
  
-And graph prototypes too:+{{manual:discovery:​low_level_discovery:​lld_rule_fs0d.png?​600|}}
  
-{{manual:discovery:low_level_discovery:graph_prototype_fs.png|}}+^Parameter^Description^ 
 +|//Type of calculation// ​ |The following options for calculating filters are available:​\\ **And** - all filters must be passed;\\ **Or** - enough if one filter is passed;\\ **And/Or** - uses //And// with different macro names and //Or// with the same macro name;\\ **Custom expression** - offers the possibility to define a custom calculation of filters. The formula must include all filters in the list. Limited to 255 symbols. ​ | 
 +|//​Filters// ​ |A filter can be used to generate real items, triggers, and graphs only for certain file systems. It expects a [[https://​en.wikipedia.org/​wiki/​Perl_Compatible_Regular_Expressions|Perl Compatible Regular Expression]] (PCRE). For instance, if you are only interested in C:, D:, and E: file systems, you could put {#FSNAME} into "​Macro"​ and <​nowiki>"​^C|^D|^E"</​nowiki>​ regular expression into "​Regular expression"​ text fields. Filtering is also possible by file system types using {#FSTYPE} macro (e.g. <​nowiki>"​^ext|^reiserfs"</​nowiki>​) and by drive types (supported only by Windows agent) using {#​FSDRIVETYPE} macro (e.g., <​nowiki>"​fixed"</​nowiki>​).\\ You can enter a regular expression or reference a global [[manual:regular_expressions|regular expression]] in "​Regular expression"​ field.\\ In order to test a regular expression you can use "grep -E", for example<code bash>for f in ext2 nfs reiserfs smbfs; do echo $f | grep -E '​^ext|^reiserfs'​ || echo "SKIP$f"; done</​code>​{#​FSDRIVETYPE} macro on Windows is supported since Zabbix **3.0.0**.\\ Defining several filters is supported since Zabbix **2.4.0**.\\ Note that if some macro from the filter is missing in the response, the found entity will be ignored.\\ Filter drop-down offers two values to specify whether a macro matches a regular expression or does not match. |
  
-{{manual:discovery:​low_level_discovery:​graph_prototypes_fs.png|}}+<note warning>​A mistake or typo in the regular expression used in LLD rule may cause deleting thousands of configuration elements, historical values and events for many hosts. For example, an incorrect %%"​%%File systems for discovery%%"%% regular expression may cause deleting thousands of items, triggers, historical values and events.</​note>​
  
-Finally, we have created ​a discovery rule that looks like shown below. It has five item prototypes, two trigger prototypes, and one graph prototype.+<note important>​Zabbix database in MySQL must be created ​as case-sensitive if file system names that differ only by case are to be discovered correctly.</​note>​
  
-{{manual:​discovery:​low_level_discovery:​lld_rules_fs.png|}}+== Form buttons ==
  
-//Note//: For configuring host prototypes, see the section about [[:​manual/​vm_monitoring#​host_prototypes|host prototype]] configuration in virtual machine monitoring.+Buttons at the bottom of the form allow to perform several operations.
  
-The screenshots below illustrate how discovered ​items, ​triggersand graphs look like in the host'​s ​configuration. ​Discovered entities are prefixed with a golden link to a discovery rule they come from.+|{{manual:​config:​button_add.png|}} ​ |Add a discovery rule. This button is only available for new discovery rules. ​ | 
 +|{{manual:​config:​button_update.png|}} ​ |Update the properties of a discovery rule. This button is only available for existing discovery rules. ​ | 
 +|{{manual:​config:​button_clone.png|}} ​ |Create another discovery rule based on the properties of the current discovery rule.  | 
 +|{{manual:​config:​button_check_now.png|}} ​ |Perform discovery based on the discovery rule immediately. ​The discovery rule must already exist. See [[:​manual/​config/​items/​check_now|more details]].\\ //Note// that when performing discovery immediatelyconfiguration cache is not updatedthus the result will not reflect very recent changes to discovery rule configuration. ​ | 
 +|{{manual:​config:​button_delete.png|}} ​ |Delete the discovery rule.  | 
 +|{{manual:​config:​button_cancel.png|}} ​ |Cancel the editing of discovery rule properties |
  
-{{manual:​discovery:​low_level_discovery:​discovered_items.png|}}+=== Item prototypes ===
  
-Note that discovered entities will not be created ​in case there are already existing entities with the same uniqueness criteria, ​for example, ​an item with the same key or graph with the same name.+Once a rule is created, go to the items for that rule and press "​Create prototype"​ to create ​an item prototype. Note how macro {#FSNAME} is used where a file system name is required. When the discovery rule is processed, this macro will be substituted ​with the discovered file system.
  
-Items (similarly, triggers and graphs) created by a low-level ​discovery ​rule cannot be manually deleted. However, they will be deleted automatically if a discovered entity (file system, interface, etc) stops being discovered (or does not pass the filter anymore). In this case the items, triggers and graphs will be deleted after the days defined in the //Keep lost resources period// field pass.+{{manual:discovery:​low_level_discovery:​item_prototype_fs1.png|}}
  
-When discovered entities become 'Not discovered anymore',​ an orange lifetime indicator is displayed ​in the items listMove your mouse pointer over it and message will be displayed indicating how many days are left until the item will be deleted.+Low-level discovery [[:​manual/​config/​macros/​lld_macros|macros]] and user [[:​manual/​appendix/​macros/​supported_by_location_user|macros]] may be used in item prototype configuration and item value preprocessing [[:​manual/​config/​items/​item#​item_value_preprocessing|parameters]]Note that when used in update intervals, ​single macro has to fill the whole field. Multiple macros in one field or macros mixed with text are not supported.
  
-{{1.9.9_lifetime_indicator.png|}}+<​note>​Context-specific escaping of low-level discovery macros is performed for safe use in regular expression and XPath preprocessing parameters.</​note>​
  
-If entities were marked ​for deletion, but were not deleted at the expected time (disabled discovery rule or item host), they will be deleted the next time the discovery rule is processed.+Attributes that are specific ​for item prototypes:
  
-{{manual:​discovery:​low_level_discovery:​discovered_triggers.png|}}+^Parameter^Description^ 
 +|//New application prototype// ​ |You may define a new application prototype.\\ In application prototypes you can use low-level discovery macros that, after discovery, will be substituted with real values to create applications that are specific for the discovered entity. See also [[manual:​discovery:​low_level_discovery:​notes|application discovery notes]] for more specific information. ​ | 
 +|//​Application prototypes// ​ |Select from the existing application prototypes. ​ | 
 +|//Create enabled// ​ |If checked the item will be added in an enabled state.\\ If unchecked, the item will be added to a discovered entity, but in a disabled state |
  
-{{manual:discovery:​low_level_discovery:​discovered_graphs.png|}}+We can create several item prototypes for each file system metric we are interested in:
  
-=== - Discovery of network interfaces ===+{{manual:​discovery:​low_level_discovery:​item_prototypes_fs.png|}}
  
-Discovery of network interfaces ​is done in exactly the same way as discovery of file systems, except that you use the discovery rule key "net.if.discovery"​ instead ​of "​vfs.fs.discovery"​ and use macro {#IFNAME} instead of {#FSNAME} in filter and item/​trigger/​graph ​prototypes.+//​[[:​manual/​config/​items/​itemupdate#​using_mass_update|Mass update]]// option ​is available ​if you want to update properties ​of several ​item prototypes ​at once.
  
-Examples of item prototypes ​that you might wish to create based on "​net.if.discovery":​ "​net.if.in[{#​IFNAME},​bytes]",​ "​net.if.out[{#​IFNAME},​bytes]"​.+=== Trigger ​prototypes ​===
  
-[[#​discovery_of_file_systems|See above]] for more information about the filter.+We create trigger prototypes in a similar way as item prototypes:
  
-=== - Discovery of CPUs and CPU cores ===+{{manual:​discovery:​low_level_discovery:​trigger_prototype_fs.png|}}
  
-Discovery of CPUs and CPU cores is done in a similar fashion as network interface discovery with the exception being that the discovery rule key is "​system.cpu.discovery"​. This discovery key returns two macros - {#​CPU.NUMBER} and {#​CPU.STATUS} identifying the CPU order number and status respectively. To note, a clear distinction cannot be made between actual, physical processors, cores and hyperthreads. {#​CPU.STATUS} on Linux, UNIX and BSD systems returns the status of the processor, which can be either "​online"​ or "​offline"​. On Windows systems, this same macro may represent a third value - "​unknown"​ - which indicates that a processor has been detected, but no information has been collected ​for it yet.+Attributes ​that are specific ​for trigger prototypes:
  
-CPU discovery relies on the agent'​s collector process to remain consistent with the data provided by the collector and save resources on obtaining the dataThis has the effect of this item key not working with the test (-t) command line flag of the agent binary, which will return ​NOT_SUPPORTED status and an accompanying message indicating that the collector process has not been started.+^Parameter^Description^ 
 +|//Create enabled// ​ |If checked ​the trigger will be added in an enabled state.\\ If unchecked, ​the trigger ​will be added to discovered entity, but in a disabled state |
  
-Item prototypes ​that can be created based on CPU discovery include "​system.cpu.load[{#​CPU.NUMBER},​ <​mode>​]",​ "​system.cpu.util[{#CPU.NUMBER},​ <​type>,​ <​mode>​]" et al.+When real triggers are created from the prototypes, there may be a need to be flexible as to what constant ('​20'​ in our example) is used for comparison in the expressionSee how [[:​manual/​discovery/​low_level_discovery#using_lld_macros_in_user_macro_contexts|user macros with context]] can be useful to accomplish such flexibility.
  
-=== Discovery of SNMP OIDs ===+You can define [[:​manual/​config/​triggers/​dependencies|dependencies]] between trigger prototypes as well (supported since Zabbix 3.0). To do that, go to the //​Dependencies//​ tab. A trigger prototype may depend on another trigger prototype from the same low-level discovery (LLD) rule or on a regular trigger. A trigger prototype may not depend on a trigger prototype from a different LLD rule or on a trigger created from trigger prototype. Host trigger prototype cannot depend on a trigger from a template.
  
-In this example, we will perform SNMP discovery ​on a switch. First, go to "​Configuration"​ -> "​Templates"​.+{{manual:discovery:​low_level_discovery:​trigger_prototypes_fs.png|}}
  
-{{manual:​discovery:​low_level_discovery:​templates_snmp.png|}}+=== Graph prototypes ===
  
-To edit discovery rules for a templateclick on the link in the "​Discovery"​ column.+We can create graph prototypestoo:
  
-Then, press "​Create rule" and fill the form with the details in the screenshot below.+{{manual:​discovery:​low_level_discovery:​graph_prototype_fs.png|}}
  
-Unlike file system and network interface ​discovery, the item does not necessarily have to have "​snmp.discovery"​ key - item type of SNMP agent is sufficient.+{{manual:discovery:​low_level_discovery:​graph_prototypes_fs.png|}}
  
-The OIDs to discover are defined in SNMP OID field in the following format: ''​discovery[{#MACRO1}, oid1, {#MACRO2}, oid2,]''​+Finally, we have created a discovery ​rule that looks like shown below. It has five item prototypestwo trigger prototypesand one graph prototype.
  
-where //{#MACRO1}//, //{#MACRO2}// …  are valid lld macro names and //oid1//, //oid2//... are OIDs capable of generating meaningful values for these macros. A built-in macro //​{#​SNMPINDEX}// containing index of the discovered OID is applied to discovered entities. The discovered entities are grouped by //​{#​SNMPINDEX}// macro value.+{{manual:​discovery:​low_level_discovery:​lld_rules_fs.png|}}
  
-To understand what we meanlet us perform few snmpwalks on our switch +//Note//: For configuring host prototypessee the section about [[:manual/​vm_monitoring#​host_prototypes|host prototype]] configuration in virtual machine monitoring.
-  $ snmpwalk -v 2c -c public 192.168.1.1 IF-MIB::​ifDescr +
-  IF-MIB::​ifDescr.1 = STRING: WAN +
-  IF-MIB::​ifDescr.2 = STRING: LAN1 +
-  IF-MIB::​ifDescr.3 = STRING: LAN2 +
-   +
-  $ snmpwalk -v 2c -c public 192.168.1.1 IF-MIB::​ifPhysAddress +
-  IF-MIB::​ifPhysAddress.1 = STRING: 8:​0:​27:​90:​7a:​75 +
-  IF-MIB::​ifPhysAddress.2 = STRING: 8:​0:​27:​90:​7a:​76 +
-  IF-MIB::​ifPhysAddress.3 = STRING: 8:​0:​27:​2b:​af:​9e+
  
-And set SNMP OID to: ''​discovery[{#​IFDESCR},​ ifDescr, {#​IFPHYSADDRESS},​ ifPhysAddress]''​+=== Discovered entities ===
  
-Now this rule will discover entities with {#IFDESCR} macros set to **WAN****LAN1** and **LAN2**, {#​IFPHYSADDRESS} macros set to **8:​0:​27:​90:​7a:​75**,​ **8:​0:​27:​90:​7a:​76**, and **8:​0:​27:​2b:​af:​9e**,​ {#​SNMPINDEX} macros set to the discovered OIDs indexes **1**, **2** and **3**: +The screenshots below illustrate how discovered itemstriggers, and graphs look like in the host's configuration. Discovered entities are prefixed with an orange link to a discovery rule they come from.
-  { +
-      "​data":​ [ +
-          { +
-              "​{#​SNMPINDEX}":​ "​1",​ +
-              "​{#​IFDESCR}":​ "​lo",​ +
-              "​{#​IFPHYSADDRESS}":​ "​8:​0:​27:​90:​7a:​75"​ +
-          }, +
-          { +
-              "​{#​SNMPINDEX}":​ "​2",​ +
-              "​{#​IFDESCR}":​ "​eth1",​ +
-              "​{#​IFPHYSADDRESS}":​ "​8:​0:​27:​90:​7a:​75"​ +
-          }, +
-          { +
-              "​{#​SNMPINDEX}":​ "​3",​ +
-              "​{#​IFDESCR}":​ "​eth0",​ +
-              "​{#​IFPHYSADDRESS}":​ "​8:​0:​27:​2b:​af:​9e"​ +
-          } +
-      ] +
-  }+
  
-If an entity does not have the specified OID, then the corresponding macro will be omitted for this entity. For example if we have the following data: +{{manual:discovery:low_level_discovery:discovered_items1.png|}}
-  ifDescr.1 "​Interface #1" +
-  ifDescr.2 "​Interface #2" +
-  ifDescr.4 "​Interface #4" +
-   +
-  ifAlias.1 "​eth0"​ +
-  ifAlias.2 "​eth1"​ +
-  ifAlias.3 "​eth2"​ +
-  ifAlias.5 "​eth4"​ +
-  +
-Then in this case SNMP discovery ''​discovery[{#IFDESCR}, ifDescr, ​{#IFALIAS}, ifAlias]''​ will return the following structure +
-  { +
-      "​data"​+
-          { +
-              "​{#​SNMPINDEX}"​1, +
-              "​{#​IFDESCR}":​ "​Interface #1", +
-              "​{#​IFALIAS}":​ "​eth0"​ +
-          }, +
-          { +
-              "​{#​SNMPINDEX}":​ 2, +
-              "​{#​IFDESCR}":​ "​Interface #2", +
-              "​{#​IFALIAS}":​ "​eth1"​ +
-          }, +
-          { +
-              "​{#​SNMPINDEX}":​ 3, +
-              "​{#​IFALIAS}":​ "​eth2"​ +
-          }, +
-          { +
-              "​{#​SNMPINDEX}":​ 4, +
-              "​{#​IFDESCR}":​ "​Interface #4" +
-          }, +
-          { +
-              "​{#​SNMPINDEX}":​ 5, +
-              "​{#​IFALIAS}":​ "​eth4"​ +
-          ​} +
-      ] +
-  ​ +
  
-{{manual:discovery:​low_level_discovery:​lld_rule_snmp.png|}}+Note that discovered entities will not be created in case there are already existing entities with the same uniqueness criteria, for example, an item with the same key or graph with the same name. An error message is displayed in this case that the low-level ​discovery ​rule could not create certain entities.
  
-The following screenshot illustrates how we can use these macros ​in item prototypes:+Items (similarly, triggers and graphs) created by a low-level discovery rule will be deleted automatically if a discovered entity (file system, interface, etc) stops being discovered (or does not pass the filter anymore). In this case the items, triggers and graphs will be deleted after the days defined ​in the //Keep lost resources period// field pass.
  
-{{manual:​discovery:​low_level_discovery:​item_prototype_snmp.png|}}+When discovered entities become 'Not discovered anymore',​ a lifetime indicator is displayed in the item list. Move your mouse pointer over it and a message will be displayed indicating how many days are left until the item is deleted.
  
-Again, creating as many item prototypes as needed:+{{:manual:​discovery:​low_level_discovery:​not_discovered_message.png|}}
  
-{{manual:discovery:​low_level_discovery:​item_prototypes_snmp.png|}}+If entities were marked for deletion, but were not deleted at the expected time (disabled ​discovery ​rule or item host), they will be deleted the next time the discovery rule is processed.
  
-As well as trigger prototypes:+Entities containing other entities, which are marked for deletion, will not update if changed on the discovery rule level. For example, LLD-based triggers will not update if they contain items that are marked for deletion.
  
-{{manual:​discovery:​low_level_discovery:​trigger_prototype_snmp.png|}}+{{manual:​discovery:​low_level_discovery:​discovered_triggers1.png|}}
  
-{{manual:​discovery:​low_level_discovery:​trigger_prototypes_snmp.png|}}+{{manual:​discovery:​low_level_discovery:​discovered_graphs1.png|}}
  
-And graph prototypes:+=== Other types of discovery ===
  
-{{manual:discovery:low_level_discovery:​graph_prototype_snmp.png|}}+More detail and how-tos on other types of out-of-the-box ​discovery ​is available in the following sections:
  
-{{manual:​discovery:​low_level_discovery:​graph_prototypes_snmp.png|}}+  * discovery of [[:manual/​discovery/​low_level_discovery/​network_interfaces|network interfaces]];​ 
 +  * discovery of [[:manual/discovery/​low_level_discovery/​cpu|CPUs and CPU cores]]; 
 +  * discovery of [[:manual/​discovery/​low_level_discovery/​snmp_oids|SNMP OIDs]]; 
 +  * discovery of [[:manual/​discovery/​low_level_discovery/​jmx|JMX objects]];​ 
 +  * discovery using [[:​manual/​discovery/​low_level_discovery/​sql_queries|ODBC SQL queries]];​ 
 +  * discovery of [[:​manual/​discovery/​low_level_discovery/​windows_services|Windows services]];​ 
 +  * discovery of [[:​manual/​discovery/​low_level_discovery/​host_interfaces|host interfaces]] in Zabbix.
  
-A summary of our discovery ​rule:+For more detail on the JSON format for discovery ​items and an example of how to implement your own file system discoverer as a Perl script, see [[#​creating_custom_lld_rules|creating custom LLD rules]].
  
-{{manual:​discovery:​low_level_discovery:​lld_rules_snmp.png?​600|}}+=== Data limits for return values ===
  
-When server ​runsit will create real items, triggers, and graphs, based on the values ​"snmp.discovery"​ returns. In host'​s ​configuration they will be prefixed with a golden link to a discovery rule they come from.+There is no limit for low-level discovery rule JSON data if it is received directly by Zabbix ​server, ​because return ​values ​are processed without being stored in a databaseThere'​s ​also no limit for custom low-level discovery rules, however, if it is intended ​to acquire custom LLD data using user parameter, then user parameter return value limit applies (512 KB).
  
-{{manual:​discovery:​low_level_discovery:​discovered_items_snmp.png|}}+If data has to go through Zabbix proxy it has to store this data in database so [[:manual/​config/​items/​item#​text_data_limits|database limits]] apply, for example, 2048 bytes on a Zabbix proxy run with IBM DB2 database.
  
-{{manual:​discovery:​low_level_discovery:​discovered_triggers_snmp.png|}}+=== Multiple LLD rules for same item ===
  
-{{manual:discovery:​low_level_discovery:​discovered_graphs_snmp.png|}}+Since Zabbix agent version 3.2 it is possible to define several low-level discovery rules with the same discovery ​item
  
-=== - Discovery using ODBC SQL queries ​===+To do that you need to define the Alias agent [[manual/​appendix/​config/​zabbix_agentd|parameter]],​ allowing to use altered discovery item keys in different discovery rules, for example ''​vfs.fs.discovery[foo]'',​ ''​vfs.fs.discovery[bar]'',​ etc. 
 +=== Creating custom LLD rules ===
  
-This type of discovery ​is done using SQL queries, whose results get automatically transformed into a JSON object suitable for low-level discovery. SQL queries are performed using items of type "​Database monitor"​. Therefore, most of the instructions on [[:​manual/​config/​items/​itemtypes/​odbc_checks|ODBC monitoring]] page apply in order to get working "​Database monitor"​ discovery ​rule, the only difference being that "​db.odbc.discovery[<​description>,<​dsn>​]"​ key should be used instead ​of "​db.odbc.select[<​description>​,<​dsn>​]"​.+It is also possible ​to create ​completely custom LLD rule, discovering any type of entities - for exampledatabases on a database server.
  
-As practical example to illustrate how the SQL query is transformed into JSON, let us consider low-level discovery ​of Zabbix proxies by performing an ODBC query on Zabbix databaseThis is useful ​for automatic creation of "​zabbix[proxy,<​name>​,lastaccess]"​ [[:​manual/​config/​items/​itemtypes/​internal|internal items]] ​to monitor which proxies are alive.+To do so, custom item should be created that returns ​JSON, specifying found objects and optionally ​some properties ​of themThe amount of macros per entity ​is not limited - while the built-in discovery rules return either one or two macros (for exampletwo for filesystem discovery)it is possible ​to return more.
  
-Let us start with discovery ​rule configuration:+The required JSON format is best illustrated ​with an example. Suppose we are running an old Zabbix 1.8 agent (one that does not support "​vfs.fs.discovery"), but we still need to discover file systems. Here is a simple Perl script for Linux that discovers mounted file systems and outputs JSON, which includes both file system name and type. One way to use it would be as a UserParameter with key "​vfs.fs.discovery_perl"​:
  
-{{manual:​discovery:​low_level_discovery:​discovery_rule_odbc.png}}+<code perl> 
 +#​!/​usr/​bin/​perl
  
-Here, the following direct query on Zabbix database is used to select all Zabbix proxies, together with the number of hosts they are monitoring. The number of hosts can be used, for instance, to filter out empty proxies:+$first = 1;
  
-<​code>​ +print "​[\n"​;
-mysql> SELECT h1.host, COUNT(h2.host) AS count FROM hosts h1 LEFT JOIN hosts h2 ON h1.hostid = h2.proxy_hostid WHERE h1.status IN (5, 6) GROUP BY h1.host; +
-+---------+-------+ +
-| host    | count | +
-+---------+-------+ +
-| Japan 1 |     5 | +
-| Japan 2 |    12 | +
-| Latvia ​ |     3 | +
-+---------+-------+ +
-3 rows in set (0.01 sec) +
-</​code>​+
  
-By the internal workings of "​db.odbc.discovery[]"​ item, the result of this query gets automatically transformed into the following JSON: +for (`cat /​proc/​mounts`)
- +
-<code js>+
 { {
-    "​data":​ [ + ($fsname$fstype) = m/\S+ (\S+) (\S+)/;
-        { +
-            "​{#​HOST}":​ "Japan 1", +
-            "​{#​COUNT}":​ "​5"​ +
-        }, +
-        { +
-            "​{#​HOST}":​ "Japan 2", +
-            "​{#​COUNT}":​ "​12"​ +
-        }, +
-        { +
-            "​{#​HOST}":​ "​Latvia",​ +
-            "​{#​COUNT}":​ "​3"​ +
-        } +
-    ] +
-+
-</code>+
  
-It can be seen that column names become macro names and selected rows become the values of these macros.+ print "​\t,​\n"​ if not $first; 
 + $first = 0;
  
-<​note>​ + print "​\t{\n";​ 
-If it is not obvious how a column name would be transformed into a macro nameit is suggested to use column aliases like "COUNT(h2.host) AS count" ​in the example above. + print "​\t\t\"​{#​FSNAME}\":​\"​$fsname\"​,\n"
- + print ​"\t\t\"​{#​FSTYPE}\":​\"​$fstype\"​\n";​ 
-In case a column name cannot be converted into a valid macro name, the discovery rule becomes not supported, with the error message detailing the offending column number. If additional help is desired, the obtained column names are provided under DebugLevel=4 in Zabbix server log file:+ print "​\t}\n";​ 
 +}
  
-<​code>​ +print "​]\n"​;
-$ grep db.odbc.discovery /​tmp/​zabbix_server.log +
- ... +
- ​23876:​20150114:​153410.856 In db_odbc_discovery() query:'​SELECT h1.host, COUNT(h2.host) FROM hosts h1 LEFT JOIN hosts h2 ON h1.hostid = h2.proxy_hostid WHERE h1.status IN (5, 6) GROUP BY h1.host;+
- ​23876:​20150114:​153410.860 db_odbc_discovery() column[1]:'​host'​ +
- ​23876:​20150114:​153410.860 db_odbc_discovery() column[2]:'​COUNT(h2.host)'​ +
- ​23876:​20150114:​153410.860 End of db_odbc_discovery():​NOTSUPPORTED +
- ​23876:​20150114:​153410.860 Item [Zabbix server:​db.odbc.discovery[proxies,​{$DSN}]] error: Cannot convert column #2 name to macro.+
 </​code>​ </​code>​
-</​note>​ 
  
-Now that we understand how a SQL query is transformed into a JSON object, we can use {#​HOST} ​macro in item prototypes:+<note important>​Allowed symbols for LLD macro names are **0-9** , **A-Z** , **_** , **.** \\ \\  Lowercase letters are not supported ​in the names.</​note>​
  
-{{manual:discovery:low_level_discovery:item_prototype_odbc.png}}+An example of its output (reformatted for clarity) is shown below. JSON for custom discovery checks has to follow the same format. 
 +<code javascript>​ 
 +
 + "{#​FSNAME}"​:"/", ​                          "​{#​FSTYPE}"​:"​rootfs" ​  }, 
 + { "​{#​FSNAME}"​:"/​sys", ​                       "​{#​FSTYPE}":"​sysfs" ​   }, 
 + { "​{#​FSNAME}":"/​proc", ​                      "​{#​FSTYPE}":"​proc" ​    }, 
 + { "​{#​FSNAME}":"/​dev", ​                       "​{#​FSTYPE}":"​devtmpfs"​ }, 
 + { "​{#​FSNAME}":"/​dev/​pts", ​                   "​{#​FSTYPE}":"​devpts" ​  }, 
 + { "​{#​FSNAME}":"/​lib/​init/​rw", ​               "​{#​FSTYPE}":"​tmpfs" ​   }, 
 + { "​{#​FSNAME}":"/​dev/​shm", ​                   "​{#​FSTYPE}":"​tmpfs" ​   }, 
 + { "​{#​FSNAME}":"/​home", ​                      "​{#​FSTYPE}":"​ext3" ​    }, 
 + { "​{#​FSNAME}":"/​tmp", ​                       "​{#​FSTYPE}":"​ext3" ​    }, 
 + { "​{#​FSNAME}":"/​usr", ​                       "​{#​FSTYPE}":"​ext3" ​    }, 
 + { "​{#​FSNAME}":"/​var", ​                       "​{#​FSTYPE}":"​ext3" ​    }, 
 + { "​{#​FSNAME}":"/​sys/​fs/​fuse/​connections", ​   "​{#​FSTYPE}":"​fusectl"  ​} 
 +
 +</​code>​
  
-Once discovery ​is performedan item will be created for each proxy:+In previous example it is required that the keys match the LLD macro names used in prototypesthe alternative is to extract LLD macro values using JSONPath ''​{#​FSNAME}''​ -> ''​$.fsname''​ and ''​{#​FSTYPE}''​ -> ''​$.fstype'',​ thus making such script possible:
  
-{{manual:discovery:low_level_discovery:​discovered_items_odbc.png}}+<code perl> 
 +#​!/​usr/​bin/​perl 
 +  
 +$first = 1; 
 +  
 +print "​[\n";​ 
 +  
 +for (`cat /​proc/​mounts`) 
 +{ 
 + ($fsname, $fstype) = m/\S+ (\S+) (\S+)/; 
 +  
 + print "​\t,​\n"​ if not $first; 
 + $first = 0; 
 +  
 + print "\t{\n"; 
 + print "​\t\t\"​fsname\"​:\"​$fsname\",​\n";​ 
 + print "​\t\t\"​fstype\"​:\"​$fstype\"​\n";​ 
 + print "\t}\n"; 
 +} 
 +  
 +print "​]\n";​ 
 +</​code>​
  
-=== - Discovery of Windows services === 
  
-Discovery ​of Windows services ​is done in exactly ​the same way as discovery of file systemsThe discovery rule key is "service.discovery" ​and the following macros are supported in filter and item/trigger/graph prototypes+An example ​of its output (reformatted for clarity) ​is shown below. JSON for custom discovery checks has to follow ​the same format. 
- +<code javascript>​ 
-<​code>​ +
-#​SERVICE.NAME +"fsname":​"/", ​                          "​fstype":"​rootfs" ​  }, 
-#​SERVICE.DISPLAYNAME + { "​fsname":"​/sys", ​                       "​fstype"​:"​sysfs" ​   }, 
-#​SERVICE.DESCRIPTION + { "​fsname":"/​proc", ​                      "​fstype":"​proc" ​    }, 
-#​SERVICE.STATE + { "​fsname":"/​dev", ​                       "​fstype":"​devtmpfs"​ }, 
-#​SERVICE.STATENAME + { "​fsname":"/​dev/​pts", ​                   "​fstype":"​devpts" ​  }, 
-#​SERVICE.PATH + { "​fsname":"/​lib/​init/​rw", ​               "​fstype":"​tmpfs" ​   }, 
-#​SERVICE.USER + { "​fsname":"/​dev/​shm", ​                   "​fstype":"​tmpfs" ​   }, 
-#​SERVICE.STARTUP + { "​fsname":"/​home", ​                      "​fstype":"​ext3" ​    }, 
-#​SERVICE.STARTUPNAME+ { "​fsname":"/​tmp", ​                       "​fstype":"​ext3" ​    }, 
 + { "​fsname":"/​usr", ​                       "​fstype":"​ext3" ​    }, 
 + { "​fsname":"/​var", ​                       "​fstype":"​ext3" ​    }, 
 + { "​fsname":"/​sys/​fs/​fuse/​connections", ​   "​fstype":"​fusectl" ​ } 
 +]
 </​code>​ </​code>​
  
-THe item prototype that can be created based on Windows service ​discovery ​is "service.info[{#​SERVICE.NAME},​ <​param>​]", where //​param// ​field can accept the following: //​displayname//,​ //state//, //path//, //user//, //startup// or //​description//​. For exampleitem "service.info[{#SERVICE.NAME}, displayname]" ​should be used to acquire the display name of service. The default parameter is //state// if //param// value is not specified ("service.info[{#​SERVICE.NAME}]")+Then, in the discovery ​rule'​s ​"Filter" field, ​we could specify ​"{#FSTYPE}" ​as macro and "rootfs|ext3" ​as a regular expression.
  
-Macro #​SERVICE.STATE and macro #SERVICE.STATENAME return the same values - #​SERVICE.STATE returns numerical value (0-7), but #​SERVICE.STATENAME returns text (running, paused, start pending, pause pending, continue pending, stop pending, stopped ​or unknown). The same applies to #​SERVICE.STARTUP (0-4) and #​SERVICE.STARTUPNAME (automatic, automatic delayed, manual, disabled, unknown).+<​note>​You don't have to use macro names FSNAME/​FSTYPE with custom LLD rules, you are free to use whatever names you likeIn case JSONPath is used then LLD row will be an array element that can be an object, but it can be also another array or a value</​note>​
  
-[[#discovery_of_file_systems|See above]] for more information about the filter. +Note that, if using a user parameter, the return value is limited to 512 KB. For more details, see [[:​manual/​discovery/​low_level_discovery#data_limits_for_return_values|data limits for LLD return values]].
-=== - Creating custom LLD rules ===+
  
-It is also possible to create a completely custom ​LLD rule, discovering any type of entities - for example, databases on a database server.+=== Using LLD macros in user macro contexts ===
  
-To do so, a custom item should ​be created that returns JSON, specifying found objects ​and optionally - some properties of themThe amount of macros ​per entity is not limited ​while the built-in ​discovery ​rules return either one or two macros ​(for example, two for filesystem discovery), it is possible ​to return more.+User macros [[:​manual/​config/​macros/​usermacros#​user_macro_context|with context]] can be used to accomplish more flexible thresholds in trigger expressions. Different thresholds may be defined on user macro level and then used in trigger constants depending on the discovered contextDiscovered context appears when the [[:​manual/​config/​macros/​lld_macros|low-level discovery macros]] used in the macros are resolved ​to real values.
  
-The required JSON format is best illustrated with an example. Suppose we are running an old Zabbix 1.8 agent (one that does not support "​vfs.fs.discovery"​),​ but we still need to discover ​file systems. Here is a simple Perl script for Linux that discovers mounted file systems and outputs JSONwhich includes both file system name and typeOne way to use it would be as a UserParameter with key "​vfs.fs.discovery_perl":​+To illustrate we can use data from the example ​above and assume ​that the following ​file systems ​will be discovered: ''/'',​ ''/​home''​''/​tmp'',​ ''/​usr'', ​ ''/​var''​.
  
-<code perl> +We may define a free-disk-space trigger prototype for a host, where the threshold is expressed by a user macro with context:
-#​!/​usr/​bin/​perl+
  
-$first = 1;+''​{host:​vfs.fs.size[{#​FSNAME},​pfree].last()}<​**{$LOW_SPACE_LIMIT:<​nowiki>"</​nowiki>​{#​FSNAME}<​nowiki>"</​nowiki>​}**''​
  
-print "{\n"; +Then add user macros: 
-print "​\t\"​data\"​:[\n\n";​+  * ''​{$LOW_SPACE_LIMIT}''​ **10** 
 +  * ''​{$LOW_SPACE_LIMIT:/​home}''​ **20** 
 +  * ''​{$LOW_SPACE_LIMIT:/​tmp}''​ **50**
  
-for (`cat /proc/​mounts`) +Now, once the file systems are discovered, events will be generated if ''​/''​''​/usr''​ and ''​/var''​ filesystems have less than **10**% of free disk space, the ''​/home''​ filesystem - less than **20**% of free disk space or the ''​/tmp''​ filesystem - less than **50**% of free disk space.
-+
-    ($fsname$fstype) = m/\S+ (\S+) (\S+)/+
-    $fsname =~ s!/!\\/!g;+
  
-    print "​\t,​\n"​ if not $first; 
-    $first = 0; 
- 
-    print "​\t{\n";​ 
-    print "​\t\t\"​{#​FSNAME}\":​\"​$fsname\",​\n";​ 
-    print "​\t\t\"​{#​FSTYPE}\":​\"​$fstype\"​\n";​ 
-    print "​\t}\n";​ 
-} 
- 
-print "​\n\t]\n";​ 
-print "​}\n";​ 
-</​code>​ 
- 
-<note important>​Allowed symbols for LLD macro names are **0-9** , **A-Z** , **_** , **.** \\ \\  Lowercase letters are not supported in the names.</​note>​ 
- 
-An example of its output (reformatted for clarity) is shown below. JSON for custom discovery checks has to follow the same format. 
- 
-  { 
-    "​data":​[ 
-    ​ 
-    { "​{#​FSNAME}":"​\/", ​                          "​{#​FSTYPE}":"​rootfs" ​  }, 
-    { "​{#​FSNAME}":"​\/​sys", ​                       "​{#​FSTYPE}":"​sysfs" ​   }, 
-    { "​{#​FSNAME}":"​\/​proc", ​                      "​{#​FSTYPE}":"​proc" ​    }, 
-    { "​{#​FSNAME}":"​\/​dev", ​                       "​{#​FSTYPE}":"​devtmpfs"​ }, 
-    { "​{#​FSNAME}":"​\/​dev\/​pts", ​                  "​{#​FSTYPE}":"​devpts" ​  }, 
-    { "​{#​FSNAME}":"​\/", ​                          "​{#​FSTYPE}":"​ext3" ​    }, 
-    { "​{#​FSNAME}":"​\/​lib\/​init\/​rw", ​             "​{#​FSTYPE}":"​tmpfs" ​   }, 
-    { "​{#​FSNAME}":"​\/​dev\/​shm", ​                  "​{#​FSTYPE}":"​tmpfs" ​   }, 
-    { "​{#​FSNAME}":"​\/​home", ​                      "​{#​FSTYPE}":"​ext3" ​    }, 
-    { "​{#​FSNAME}":"​\/​tmp", ​                       "​{#​FSTYPE}":"​ext3" ​    }, 
-    { "​{#​FSNAME}":"​\/​usr", ​                       "​{#​FSTYPE}":"​ext3" ​    }, 
-    { "​{#​FSNAME}":"​\/​var", ​                       "​{#​FSTYPE}":"​ext3" ​    }, 
-    { "​{#​FSNAME}":"​\/​sys\/​fs\/​fuse\/​connections",​ "​{#​FSTYPE}":"​fusectl" ​ } 
-    ​ 
-    ] 
-  } 
- 
-Then, in the discovery rule's "​Filter"​ field, we could specify "​{#​FSTYPE}"​ as a macro and "​rootfs|ext3"​ as a regular expression. 
  
-<​note>​You don't have to use macro names FSNAME/​FSTYPE with custom LLD rules, you are free to use whatever names you like.</​note>​