The ZABBIX Agent service terminated unexpectedly

[modest]

Hi,

I'm having issue with Zabbix agent 1.8 running on Windows Server 2008 (Domain Controller), when it stops working unexpectedly.

I tried to run it in debug mode, but didn't get anything that could point out reason why it stops. Log shows that it stops when processing requests, for example:
last entries in log before crash:

12268:20091228:155218.245 Processing request.
 12268:20091228:155218.245 Requested [perf_counter[\PhysicalDisk(_Total)\Avg. Disk Write Queue Length]]

and

12904:20091228:160408.872 Processing request.
 12904:20091228:160408.872 Requested [perf_counter[\System\File Write Bytes/sec]]

For Windows System Log:

Log Name:      System
Source:        Service Control Manager
Date:          28.12.2009 16:38:41
Event ID:      7034
Task Category: None
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      server.domain.com
Description:
The ZABBIX Agent service terminated unexpectedly.  It has done this 10 time(s).
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Service Control Manager" Guid="{555908D1-A6D7-4695-8E1E-26931D2012F4}" EventSourceName="Service Control Manager" />
    <EventID Qualifiers="49152">7034</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2009-12-28T14:38:41.000000000Z" />
    <EventRecordID>1611673</EventRecordID>
    <Correlation />
    <Execution ProcessID="0" ThreadID="0" />
    <Channel>System</Channel>
    <Computer>server.domain.com</Computer>
    <Security />
  </System>
  <EventData>
    <Data Name="param1">ZABBIX Agent</Data>
    <Data Name="param2">10</Data>
  </EventData>
</Event>

From MS Application Logs:

Log Name:      Application
Source:        Application Error
Date:          28.12.2009 15:47:18
Event ID:      1000
Task Category: (100)
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      server.domain.com
Description:
Faulting application zabbix_agentd.exe, version 0.0.0.0, time stamp 0x4b30f5b3, faulting module ntdll.dll, version 6.0.6002.18005, time stamp 0x49e0421d, exception code 0xc0000374, fault offset 0x00000000000aef37, process id 0x3224, application start time 0x01ca87c42a4b4656.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Application Error" />
    <EventID Qualifiers="0">1000</EventID>
    <Level>2</Level>
    <Task>100</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2009-12-28T13:47:18.000000000Z" />
    <EventRecordID>32056</EventRecordID>
    <Channel>Application</Channel>
    <Computer>server.domain.com</Computer>
    <Security />
  </System>
  <EventData>
    <Data>zabbix_agentd.exe</Data>
    <Data>0.0.0.0</Data>
    <Data>4b30f5b3</Data>
    <Data>ntdll.dll</Data>
    <Data>6.0.6002.18005</Data>
    <Data>49e0421d</Data>
    <Data>c0000374</Data>
    <Data>00000000000aef37</Data>
    <Data>3224</Data>
    <Data>01ca87c42a4b4656</Data>
  </EventData>
</Event>

I had similar issue with server running Windows 2003, but after several Zabbix agent restarts it works fine. Also Server running Windows 2008 R2 has no problems at this moment.

[simonuk1]

Does it only do it with one particular performance counter ?

Might be worth checking the syntax, has done that to me before if the perf counter has wrong spaces etc

[modest]

It just crashes it doesn't depend on performance counter as you see - every time the counter on which it stops is different.

By the way - nothing haven't been changed - it's default settings (counters)

[modest]

Today I tried to use Zabbix Agent 1.6.8 instead of 1.8 and everything works great. Agent isn't crashing any more.

Isn't that strange

[modest]

Found same problem also on other servers running Windows 2003, seems that Agent 1.8 is having some kind a bug.

With 1.6.8 everything works just fine.

Event Type:	Error
Event Source:	Application Error
Event Category:	(100)
Event ID:	1000
Date:		29.12.2009
Time:		15:15:25
User:		N/A
Computer:	COMPUTER
Description:
Faulting application zabbix_agentd.exe, version 0.0.0.0, faulting module ntdll.dll, version 5.2.3790.4455, fault address 0x0002b58b.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 70 70 6c 69 63 61 74   Applicat
0008: 69 6f 6e 20 46 61 69 6c   ion Fail
0010: 75 72 65 20 20 7a 61 62   ure  zab
0018: 62 69 78 5f 61 67 65 6e   bix_agen
0020: 74 64 2e 65 78 65 20 30   td.exe 0
0028: 2e 30 2e 30 2e 30 20 69   .0.0.0 i
0030: 6e 20 6e 74 64 6c 6c 2e   n ntdll.
0038: 64 6c 6c 20 35 2e 32 2e   dll 5.2.
0040: 33 37 39 30 2e 34 34 35   3790.445
0048: 35 20 61 74 20 6f 66 66   5 at off
0050: 73 65 74 20 30 30 30 32   set 0002
0058: 62 35 38 62               b58b

and

Event Type:	Error
Event Source:	Application Error
Event Category:	(100)
Event ID:	1000
Date:		29.12.2009
Time:		14:42:30
User:		N/A
Computer:	COMPUTER
Description:
Faulting application zabbix_agentd.exe, version 0.0.0.0, faulting module zabbix_agentd.exe, version 0.0.0.0, fault address 0x00012e26.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 70 70 6c 69 63 61 74   Applicat
0008: 69 6f 6e 20 46 61 69 6c   ion Fail
0010: 75 72 65 20 20 7a 61 62   ure  zab
0018: 62 69 78 5f 61 67 65 6e   bix_agen
0020: 74 64 2e 65 78 65 20 30   td.exe 0
0028: 2e 30 2e 30 2e 30 20 69   .0.0.0 i
0030: 6e 20 7a 61 62 62 69 78   n zabbix
0038: 5f 61 67 65 6e 74 64 2e   _agentd.
0040: 65 78 65 20 30 2e 30 2e   exe 0.0.
0048: 30 2e 30 20 61 74 20 6f   0.0 at o
0050: 66 66 73 65 74 20 30 30   ffset 00
0058: 30 31 32 65 32 36         012e26

[modest]

Update again, and I hope my spam helps to developers to solve this problem...

We didn't get any of Zabbix 1.8 agents running on Windows 2008 SP2 + Active Directory.

When using Zabbix agent 1.6.8 everything works fine!

[brian]

We have 2 monitored Windows 2008 SP2 servers, both of the are Domain Controllers. Agent is working fine, I have only one issue with them: we are not receiving CPU data. With 1.6.8 it works well.

(I reported this in another topic: http://www.zabbix.com/forum/showthread.php?t=13132)

[modest]

By default Zabbix monitors Processor load, which in my opinion is kind of Linux like value. And I never understand how the Zabbix agent gets it, and how it's calculated. We had several issues when it's reporting high load on server, but there is no such a thing in reality.

To get correct data of CPU load (Windows systems), I added counter perf_counter[\Processor(_Total)\% Processor Time]. It works perfect on Windows 2003/2008/2008 R2 and reports correct information about CPU Load.

[brian]

@modest
I took your advice and changed the item. This way it is much more "human-readable".

thanks