Get age of latest file in folder

[harvey]

Hi,

I am in the process of migrating from Nagios. One feature that I used on Nagios was to check if a backup had happened on a server.

I did this by checking the age of the newest file in a specific folder. If the file was older than 25 hours it would alert me.

How can I achieve this in Zabbix?

Regards

Harvey

[eskytthe]

I think you need to make a user parmeter for this. Look here:
http://www.zabbix.com/documentation/...ser_parameters
/Erik

[frater]

This code will give the age of the newest file in that folder in seconds.

#!/bin/bash
export PATH=${PATH}:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin

[ -z "${1}" ] && exit

FOLDER="`echo "$1" | sed 's/\/$//'`"

if [ -d "${1}" ] ; then
  FILE="`ls -1tr "${FOLDER}/" | tail -n1`"
  STAMP=`stat -c%Z "${FOLDER}/${FILE}" | awk -F\. '{print $1}'`
  echo "`date +%s` ${STAMP}" | awk '{print $1 - $2}'
fi

[harvey]

Hi All,

Thanks for your feedback.

User parameters looks like just what I am after.

I already had the script:-

#!/bin/bash
FILE=`ls -t1 /Backup/backup | head -n1`
NOW=`date +%s`
OLD=`stat -c %Z /Backup/backup/$FILE`
AGE=$(((NOW-OLD)/3600))
echo $AGE

Which works ok as a normal script. I added the following two lines to /etc/zabbix/zabbix_agent.conf:-

UserParameter=backupDate,backupDate
UserParameter=backupAge,FILE=`ls -t1 /Backup/backup | head -n1`;NOW=`date +%s`;OLD=`stat -c %Z /Backups/backup/$FILE`;AGE=$(((NOW-OLD)/3600));echo $AGE

Then restarted zabbix agent with /etc/init.d/zabbix-agent restart on the remote server. Also created a script called backupDate in /usr/local/bin and gave it execute permissions on the remote server.

Then on the zabbix server added two items to the host with the key backupAge and backupDate

Both of which when from Active to Not Supported.

Any ideas where I am going wrong?

[eskytthe]

Try run it as the zabbix user.
Try full path to your scirpt/commands.
BR
Erik

[frater]

I would make it more generic instead of hardcoding the directory into the script.

I used "ls -1rt | tail -n1", but of course "ls -1t | head -n1" is "better".
(I'm used to type "ls -alrt" on the prompt)

There are a few things wrong with both your and my script.
It doesn't exclude directories and it doesn't handle an empty directory that well.

Here's my revised script

cat /usr/local/sbin/latestfile

#!/bin/bash
export PATH=${PATH}:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin

[ -z "${1}" ] && exit

FOLDER="`echo "$1" | sed 's/\/$//'`"

if [ -d "${1}" ] ; then
  FILE="`ls -1pt "${FOLDER}/" | grep -v '/$' | head -n1`"
  STAMP=0
  if [ ! -z "${FILE}" ] ; then
    STAMP=`stat -c%Z "${FOLDER}/${FILE}" | awk -F\. '{print $1}'`
  fi
  echo "`date +%s` ${STAMP}" | awk '{print $1 - $2}'
fi

It was quite funny to see your script was almost the same as mine.


You can test the script with "sudo -u zabbix latestfile /usr/local/sbin"

UserParameter=vfs.file.latest[*], latestfile "$1"

Item:

vfs.file.latest[ /usr/local/sbin ]

[frater]

I added some code, so it can take a 2nd (optional) parameter.
A regular expression that filters the files.

I did some tests putting the 2nd parameter in the 'ls' command, but this gave strange results....
I then decided to use 'grep "${REGEXP}" which is more powerful anyway.

From CLI:

latestfile <dir> [ <regexp> ]

From Zabbix:

vfs.file.latest[ \usr\local\sbin, "za.*" ]
vfs.file.latest[ \usr\local\sbin ]

#!/bin/bash
export PATH=${PATH}:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin

[ -z "${1}" ] && exit
FOLDER="`echo "$1" | sed 's/\/$//'`"

REGEXP='.*'
[ -z "${2}" ] || REGEXP="${2}"

if [ -d "${1}" ] ; then
  FILE="`ls -1tp "${FOLDER}/" 2>/dev/null | grep -v '/$' | egrep "${REGEXP}" | head -n1`"
  STAMP=0
  if [ ! -z "${FILE}" ] ; then
    STAMP=`stat -c%Z "${FILE}" | awk -F\. '{print $1}'`
  fi
  echo "`date +%s` ${STAMP}" | awk '{print $1 - $2}'
fi

[harvey]

Thanks so much for all your help. Still having problems though.

Ok so on the remote machine I have:-

/usr/local/bin/backupDate

I can run:-

sudo -u zabbix backupDate /Backup/backup

and it works perfectly!

The code is:-

#!/bin/bash
export PATH=${PATH}:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin
FILE=`/bin/ls -t1 $1 | /usr/bin/head -n1`
NOW=`/bin/date +%s`
OLD=`/usr/bin/stat -c %Y $1/$FILE`
AGE=$(((NOW-OLD)/3600))
echo $AGE

I have also tried your code too!

Then on the remote machine /etc/zabbix/zabbix_agent.conf has:-

UserParameter=vfs.backup.date[*], backupDate "$1"
UserParameter=vfs.backup.latest[*], latestBackup "$1"

I restart the agent on the remote machine

/etc/init.d/zabbix-agent restart

Then on the zabbix server have:-

vfs.backup.latest[ /Backup/backup ]
vfs.backup.date[ /Backup/backup ]

Neither work and both say not supported.

Once again thanks for all your help. Don't think I'm far away. Ideas?

[frater]

Once again thanks for all your help. Don't think I'm far away. Ideas?

Yes, check both agent and server logs...
Maybe you need to raise debug level....

You are making them active in the webif before you test, do you?

BTW... I don't understand why you are dividing by 3600 (1h)?
It doesn't even round the result (59 minutes + 59 seconds will be 0)

I recommend to use 'awk' and divide by 60 if you want to divide at all.
But in zabbix most is done in seconds, so why not leave it that way?

This shows you what will go wrong.....

~# echo $((3599 / 3600))
0
~# echo "3599 3600" | awk '{print $1 / $2}'
0.999722

That's why you should do:

~# echo "3599 3600" | awk '{printf"%.0f\n", $1 / $2}'
1
~# echo "1800 3600" | awk '{printf"%.0f\n", $1 / $2}'
0
~# echo "1801 3600" | awk '{printf"%.0f\n", $1 / $2}'
1

[ghoz]

UserParameter=vfs.backup.date[*], backupDate "$1"
UserParameter=vfs.backup.latest[*], latestBackup "$1"

put debuglevel=4 in your agent config , it's verbose but genereally helpful.

I think the problem may be that the agent doesn't find the right path. you can try putting the full path of scripts in your config ...

[frater]

Today I needed this function for myself and found out my script wasn't adequate. I want to check a file that's placed in one of the subdirectories in the directory I want to check.

It needs to check if an FTP-backup is placed there...

I wrote 2 functions.
1 is called latestfilename and the other is latestfileage.
Both functions take the same parameters...
latestfileage returns the age of the newest file in seconds.
latestfilename returns the name of the newest file with the path relative to <folder>

depth and regexp are optional.
If depth is not given it will not look into the subdirectories

latestfilename folder depth regexp
latestfileage folder depth regexp

/etc/sudoers

zabbix ALL =(ALL) NOPASSWD: /usr/local/sbin/latestfiledate
zabbix ALL =(ALL) NOPASSWD: /usr/local/sbin/latestfilename

/usr/local/sbin/latestfilename

#!/bin/bash
export PATH=${PATH}:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin

FOLDER="`echo "$1" | sed 's/\/$//'`"
[ -z "${1}" ] && exit

MAXDEPTH=`echo "${2}" | tr -cd '0-9'`
[ -z "${2}" ] && MAXDEPTH=1

REGEXP='.*'
[ -z "${3}" ] || REGEXP="${3}"

if [ -d "${1}" ] ; then
  FILE="`find "${FOLDER}" -maxdepth ${MAXDEPTH} -type f | egrep "${REGEXP}" | xargs -I{} stat -c '%Z %n' {} | sort -rn | head -n1 | awk '{print $2}'`"
  FILE="${FILE##${FOLDER}/}"
  echo ${FILE}
fi

/usr/local/sbin/latestfileage

#!/bin/bash
export PATH=${PATH}:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin

FOLDER="`echo "$1" | sed 's/\/$//'`"
[ -z "${1}" ] && exit

MAXDEPTH=`echo "${2}" | tr -cd '0-9'`
[ -z "${2}" ] && MAXDEPTH=1

REGEXP='.*'
[ -z "${3}" ] || REGEXP="${3}"

if [ -d "${1}" ] ; then
  STAMP="`find "${FOLDER}" -maxdepth ${MAXDEPTH} -type f | egrep "${REGEXP}" | xargs -I{} stat -c '%Z %n' {} | sort -rn | head -n1 | awk '{print $1}'`"
  [ -z "${STAMP}" ] && STAMP=0
  echo "`date +%s` ${STAMP}" | awk '{print $1 - $2}'
fi

PS in the examples I still have 'latestfiledate'. Later I changed it into 'latestfileage' (a better name for it)

[cindywang]

cindywang

I think the problem may be that the agent doesn't find the right path.

[frater]

I expanded the function with a 4th optional parameter.
You can now add a size in KB, so only files with a minimum size of a certain value are taken in consideration....

If you want to check if a backupfile has landed on your system that at least has to be 1 MB in size you can add 1000 as the 4th parameter

I also changed the behaviour when no file is found.
latestfilename will return a dot (.) to prevent the item from becoming unsupported
latestfileage will return -1 (minus 1)

You could make a 2nd trigger that will check for that.

With these 2 items you can use diff to indicate an FTP-backup has succeeded...
I will follow with some examples later on.....


/usr/local/sbin/latestfileage

#!/bin/bash
export PATH=${PATH}:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin

FOLDER="`echo "$1" | sed 's/\/$//'`"
[ -z "${1}" ] && exit

MAXDEPTH=`echo "${2}" | tr -cd '0-9'`
[ -z "${2}" ] && MAXDEPTH=1

REGEXP='.*'
[ -z "${3}" ] || REGEXP="${3}"

MINSIZE=`echo "$4" | tr -cd '[0-9]'`
FINDOPT=''
[ -z "${MINSIZE}" ] || FINDOPT="-size +${MINSIZE}"

if [ -d "${1}" ] ; then
  STAMP="`find "${FOLDER}" -maxdepth ${MAXDEPTH} -type f ${FINDOPT} | egrep "${REGEXP}" | xargs -I{} stat -c '%Z %n' {} | sort -rn | head -n1 | awk '{print $1}'`"

  if [ -z "${STAMP}" ] ; then
    echo '-1'
  else
    echo "`date +%s` ${STAMP}" | awk '{print $1 - $2}'
  fi
fi

/usr/local/sbin/latestfilename

#!/bin/bash
export PATH=${PATH}:/usr/local/sbin:/sbin:/usr/sbin:/bin:/usr/bin

FOLDER="`echo "$1" | sed 's/\/$//'`"
[ -z "${1}" ] && exit

MAXDEPTH=`echo "${2}" | tr -cd '0-9'`
[ -z "${2}" ] && MAXDEPTH=1

REGEXP='.*'
[ -z "${3}" ] || REGEXP="${3}"

MINSIZE=`echo "$4" | tr -cd '[0-9]'`
FINDOPT=''
[ -z "${MINSIZE}" ] || FINDOPT="-size +${MINSIZE}"

if [ -d "${1}" ] ; then
  FILE="`find "${FOLDER}" -maxdepth ${MAXDEPTH} -type f ${FINDOPT} | egrep "${REGEXP}" | xargs -I{} stat -c '%Z %n' {} | sort -rn | head -n1 | awk '{print $2}'`"
  FILE="${FILE##${FOLDER}/}"

  RETVAL=.
  [ -z "${FILE}" ] || RETVAL="${FILE}"
  echo "${RETVAL}"
fi

[Zabbixtrainee]

Oldest file in a folder

Again I am posting to necropost...but anyway does this also work if I need to find the oldest file modify date from a specific folder?

We have a process that transfers receipts from stores to a folder. Their names are "receipts_storenumber_date" and then in the next day another program retrieves these files and therefore in this folder /home/siirto/nav/out/archive
there should not be a file with modify date older than 25h.

If I have understood correctly vfs.file.time[fuzzytime] wont work on this situation because it doesn't check if one file has been there ages. And daily there comes about 200 new files.

I really really would appreciate any kind of help.

[bacteria666]

Hi 4 all,

Well, I was looking for a solution which show me the newest file age in a folder just to monitor if my backup was replicated to my storage.

I used to use nagios3 to do that kind of monitoring with a script => https://exchange.nagios.org/director...le_age/details


PS.: my English is a little bit rusty