howto batch populate/insert/add of numerous hosts

[LEM]

-zabbix 1.0 - not tested (yet) on 1.1-

I needed to quickly add numerous hosts to my zabbix database (150 switches). I proceeded this way:
. creation of a template for this elements (both items and triggers/actions)
. creation of a text file containing hosts (hostnames;ip-address)
. used the add_host function in a php script

Here's the text file sample (this file is named list.csv):

hostname-1;192.168.0.1
hostname-2;192.168.0.2
hostname-3;192.168.0.3
...


Here's the PHP code (named add_hosts.php):

<?

include "include/config.inc.php";
$page["title"] = "Hosts";
$page["file"] = "hosts.php";
show_header($page["title"],0,0);
insert_confirm_javascript();

// hard variables
$hard_port = "10050"; //cause I want to use ZABBIX agentd on this port
$hard_status = "0"; //cause I want the status to be monitored
$hard_useip = "1"; //cause I want to use IP address
$hard_newgroup = ""; //I did not used it
$hard_host_templateid = "10055"; //This is the hostid of the host template
$hard_groups = array(4,6,3); //This is the groupid of groups I want my new hosts to belong to...

$listing = file("list.csv", "r");
foreach ($listing as $element_number => $element)
{
list($host, $ip) = explode (";", $element);

$result=add_host($host,$hard_port,$hard_status,$ha rd_useip,$ip,$hard_host_templateid,$hard_newgroup, $hard_groups);
show_messages ($result,"Host added","Cannot add host");
}
?>


When I fire a browser to add_hosts.php, It will inject all hosts found in 'list.csv' to the zabbix database, with all regular checks the add_host function provide.


Off course, a nice php page where you can define you file format, upload you file and run a 'test' before adding should be better... but afterall, this kind of batch population if not a regular action.

Hope this will hope some to quickly populate zabbix with switch list, printer list... the kind of list who are quickly build in text format...

[fab]

Nice tip.

I went a step further and implemented it within the php frontend of zabbix:

(hosts.php -> append to if(isset($HTTP_GET_VARS["register"])) Block

if($HTTP_GET_VARS["register"]=="add multiple hosts")
{
$listing=file("/usr/local/apache2/htdocs/toy3/chroot/".$HTTP_GET_VARS['mfile']);

foreach($listing as $element_number => $element)
{
list($mhost, $mip, $mport)=explode (",", $element);

$result=add_host($mhost,$mport,$HTTP_GET_VARS["mstatus"],$HTTP_GET_VARS["museip"],$mip,$HTTP_GET_VARS["host_templateid"],$HTTP_GET_VARS["mnewgroup"],$HTTP_GET_VARS["mgroups"]);
show_messages($result,"Host added","Cannot add host");
unset($HTTP_GET_VARS["hostid"]);
}
}


Then create a new form:

echo "<br>";
echo "<a name=\"mform\"></a>";
show_table2_header_begin();
echo "Multiple Hosts";

show_table2_v_delimiter();
echo "<form method=\"get\" action=\"hosts.php#mform\">";

echo "CSV file";
show_table2_h_delimiter();
echo "<input class=\"biginput\" name=\"mfile\" value=\"$mfile\" size=20>";

show_table2_v_delimiter();
echo "Groups";
show_table2_h_delimiter();
echo "<select multiple class=\"biginput\" name=\"mgroups[]\" size=\"5\">";
$result=DBselect("select distinct groupid,name from groups order by name");
while($row=DBfetch($result))
{
if(isset($HTTP_GET_VARS["hostid"]))
{
$sql="select count(*) as count from hosts_groups where hostid=".$HTTP_GET_VARS["hostid"]." and groupid=".$row["groupid"];
$result2=DBselect($sql);
$row2=DBfetch($result2);
if($row2["count"]==0)
{
echo "<option value=\"".$row["groupid"]."\">".$row["name"];
}
else
{
echo "<option value=\"".$row["groupid"]."\" selected>".$row["name"];
}
}
else
{
echo "<option value=\"".$row["groupid"]."\">".$row["name"];
}
}
echo "</select>";

show_table2_v_delimiter();
echo nbsp("New group");
show_table2_h_delimiter();
echo "<input class=\"biginput\" name=\"mnewgroup\" size=20 value=\"$mnewgroup\">";

show_table2_v_delimiter();
echo nbsp("Use IP address");
show_table2_h_delimiter();
echo "<INPUT TYPE=\"CHECKBOX\" class=\"biginput\" NAME=\"museip\" $museip>";

show_table2_v_delimiter();
echo "Status";
show_table2_h_delimiter();
echo "<select class=\"biginput\" name=\"mstatus\" size=\"1\">";
if($mstatus==0)
{
echo "<option value=\"0\" selected>Monitored";
echo "<option value=\"1\">Not monitored";
}
else
{
echo "<option value=\"0\">Monitored";
echo "<option value=\"1\" selected>Not monitored";
}
echo "</select>";

show_table2_v_delimiter2();
echo "<input class=\"button\" type=\"submit\" name=\"register\" value=\"add multiple hosts\">";

show_table2_header_end();


As you can see, the csv file now has the following format:

host,[ip],port

If the 'Use IP' Box is checked, ip is mandatory for all hosts in the csv file.

I hope this is useful,
Frank.

[James Wells]

Greetings,

I know I have pointed a few people at this post recently so I thought I would drop an update to the status.

I am currently in the process of taking both Lem's and Fab's modified PHP code and mainlining it. I hope to have it complete in time to get it into 1.1beta3, but I can't promise anything at present.

My goal for this is to create a bulk Loader for hosts, items, and users. Longer term goal will be to get bulk loaders for IT services, triggers, and actions built up as well.

[James Wells]

Greetings,

Okay, the Host and User portion is currently checked into CVS.

The host format format is;

HOST,<Hostname>,<Host IP>,<Host Port>,<Host Status>,<Template Host>,<Zabbix Server>,<Host Group(s)>

HOST: This is the command to tell the bulk loader that this entry is a host.
Hostname: This is the hostname you wish to use in the Zabbix UI.
Host IP: If Host IP contains anything, it will be used as the IP address for this host, otherwise the host will use Hostname.
Host Port: If HostPort contains anything it will be used as the port for the Zabbix agent, otherwise it will default to 10050.
Host Status: This field can be Template, Monitored, or Not Monitored. Any thing else will be considered Not Monitored.
Template Host: This field contains the full name of the Template host, otherwise the host will not use a template.
Zabbix Server: This field contains the full name of the Zabbix server. If it is blank or the server does not exist, it is assumed that this host in on the default Zabbix Server.
Host Group(s): This field contains the name of the group or groups, comma seperated, that this host belongs to. If this field is empty then it it assumed that this host belongs to no groups. If this contains one or more groups that are not in the Zabbix Server, the groups will be created and this host added to those groups.

USER,<User First Name>,<User Surname>,<Login Name>,<Password>,<URL>,<Auto Logout Time>,<Language>,<Screen Refresh Time>,<Host Group(s)>

USER: This is the command to tell the bulk loader that this entry is a User
User First Name: This is the users first name
User Surname: This is the users last name
Login Name: This is the name the user will login with. NOTE: User will not be created if this is blank.
Password: This is the password that user will login with. If blank, this will default to the login name.
URL: This is the URL the user is redirected to upon login.
Auto Logout Time: This is the number of seconds before an idle user will be logged off. If blank, this will default to 900 seconds.
Language: This is the the language that that Zabbix will display to the user. If blank, this will default to en_gb. Valid Choices are;

en_gb: English
fr_fr: French
de_de: German
it_it: Itallian
ja_ja: Japanese
lv_lv: Latvian
ru_ru: Russian
sp_sp: Spanish

Screen Refresh Time: This is the number of seconds before monitoring pages will refresh. If blank, this will default to 30 seconds.
User Group(s): This is a comma separated list of user groups that the user belongs to. If this contains one or more groups that are not in the Zabbix Server, the groups will be created and user added to those groups.



The sample file I have created while testing the code as I add it is;

HOST,host01.test.com,,10050,template,__TEMPLATE_Li nux_SVR,zabbix-server01.test.com,dank,dunk,donk
HOST,host02.test.com,192.168.0.2,10051,monitored,_ _TEMPLATE_Linux_SVR,zabbix-server02.test.com,dink,dunk,donk
HOST,host03.test.com,192.168.0.3,,not monitored,__TEMPLATE_Linux_SVR,zabbix-server03.test.com,dink,dank,donk
HOST,host04.test.com,,10050,Template,__TEMPLATE_Li nux_SVR,zabbix-server01.test.com,dink,dank,dunk
HOST,host05.test.com,,10050,Monitored,__TEMPLATE_L inux_SVR,zabbix-server02.test.com,dank,dunk,donk,doink
USER,Test,User1,tuser1,tuser1,,900,en_gb,30,dink,d ank,dunk,donk
USER,Test,User2,tuser2,,,900,de_de,30,dink,dank,du nk,donk
HOST,host11.test.com,,10050,Not Monitored,__TEMPLATE_Win32_SVR,zabbix-server03.test.com,dink,dunk,donk
USER,Test,User2,tuser2,,,900,de_de,30,dink,dank,du nk,donk
HOST,host12.test.com,192.168.1.2,10051,0,__TEMPLAT E_Win32_SVR,zabbix-server01.test.com,dink,dank,donk
HOST,host13.test.com,192.168.0.3,,0,__TEMPLATE_Win 32_SVR,zabbix-server02.test.com,dink,dank,dunk
USER,Test,User3,tuser3,tuser3,,,lv_lv,30,dink,dank ,dunk,donk
HOST,host14.test.com,,10050,0,__TEMPLATE_Win32_SVR ,zabbix-server03.test.com,dink,dank,dunk,donk,doink
HOST,host15.test.com,,10050,0,__TEMPLATE_Win32_SVR ,zabbix-server01.test.com,dink,dank,dunk,donk
USER,Test,User4,tuser4,tuser4,,900,,30,dink,dank,d unk,donk
USER,Test,User5,tuser5,tuser5,,900,en_gb,,dink,dan k,dunk,donk


EDIT: Forgot to mention. The way I am building the tool allows you to mix and match your CSV entries, so you can bulk load hosts and users in the same file, not that you would really want to, but it is there if need it. ;)