rsync of repo.zabbix.com

[GeneBean]

Does anyone know if there is a way to get rsync access to the official repositories located at repo.zabbix.com?

[James5mith]

I just joined the forums to search for this very question.

My current desire is to have a local mirror of zabbix that I then use to deploy to all of my machines locally. This saves bandwidth on the zabbix servers, and allows me to tighten up my network.

With CentOS/puppet/etc. I can rsync the repo to a local mirror server, and update from that, thus only having one hole in my firewall rather than 100's.

Any advice?

[LenR]

I think you can use wget, something like this for each repo area


cd /var/www/html/pub/zabbixzone/6Server/x86_64/
wget -q -r -t1 --no-parent -nd --mirror http://repo.zabbixzone.com/centos/6Server/x86_64/
createrepo --update /var/www/html/pub/zabbixzone/6Server/x86_64/

[James5mith]

I think you can use wget, something like this for each repo area


cd /var/www/html/pub/zabbixzone/6Server/x86_64/
wget -q -r -t1 --no-parent -nd --mirror http://repo.zabbixzone.com/centos/6Server/x86_64/
createrepo --update /var/www/html/pub/zabbixzone/6Server/x86_64/

Why would I use zabbixzone and not the main repo.zabbix.com ?

[LenR]

That was just an example of the technique, I didn't check the URL

[James5mith]

That was just an example of the technique, I didn't check the URL

Fair enough.

I went ahead and gave it a try. Seems to work. Don't know if they will appreciate a nightly crawl through the server with wget, but I still think it's better than 100's of servers hitting it each time we try and update.

[cru5ader]

looks like someone else need rsync:

looks like someone else need rsync: https://support.zabbix.com/browse/ZBXI-3

[rcsheets]

looks like someone else need rsync: https://support.zabbix.com/browse/ZBXI-3

This is now being tracked as ZBXNEXT-3050, though it doesn't seem like it's getting any attention.

[rcsheets]

Hooray!

Looks like rsync access has finally been enabled!

The feature request has been closed and rsync access appears to be working.

rcsheets@stinger:~$ rsync repo.zabbix.com::
mirror
rcsheets@stinger:~$ rsync repo.zabbix.com::mirror
drwxr-x---          4,096 2016/08/16 06:25:25 .
lrwxrwxrwx             27 2016/08/16 06:25:25 RPM-GPG-KEY-ZABBIX
-rw-r-x---          1,333 2016/08/16 03:46:03 RPM-GPG-KEY-ZABBIX-79EA5ED4
-rw-r-x---          1,719 2016/08/16 03:46:04 RPM-GPG-KEY-ZABBIX-A14FE591
-rw-r-x---          2,938 2016/08/12 22:23:06 zabbix-official-repo.key
drwxr-x---          4,096 2012/01/23 07:37:31 non-supported
drwxr-x---          4,096 2016/08/11 09:45:13 zabbix

The total size is currently just under 10 GiB.

[kloczek]

To create mirror of any yum repos all what you need is to install yum-utils package.
In this package you can find reposync program

# reposync --help

Yum-utils package has been deprecated, use dnf instead.
See 'man yum2dnf' for more information.


Usage: 
  Reposync is used to synchronize a remote yum repository to a local 
  directory using yum to retrieve the packages.
  
  /bin/reposync [options]
  

Options:
  -h, --help show this help message and exit
  -c CONFIG, --config=CONFIG
  config file to use (defaults to /etc/yum.conf)
  -a ARCH, --arch=ARCH act as if running the specified arch (default: current
  arch, note: does not override $releasever. x86_64 is a
  superset for i*86.)
  --source operate on source packages
  -r REPOID, --repoid=REPOID
  specify repo ids to query, can be specified multiple
  times (default is all enabled)
  -e CACHEDIR, --cachedir=CACHEDIR
  directory in which to store metadata
  -t, --tempcache Use a temp dir for storing/accessing yum-cache
  -d, --delete delete local packages no longer present in repository
  -p DESTDIR, --download_path=DESTDIR
  Path to download packages to: defaults to current dir
  --norepopath Don't add the reponame to the download path. Can only
  be used when syncing a single repository (default is
  to add the reponame)
  -g, --gpgcheck Remove packages that fail GPG signature checking after
  downloading
  -u, --urls Just list urls of what would be downloaded, don't
  download
  -n, --newest-only Download only newest packages per-repo
  -q, --quiet Output as little as possible
  -l, --plugins enable yum plugin support
  -m, --downloadcomps also download comps.xml
  --download-metadata download all the non-default metadata

In other words .. access to the repo over rsync is not needed.

[rcsheets]

To create mirror of any yum repos all what you need is to install yum-utils package.

That's great if all you need is the yum repo. If you need to mirror the entire repository, or if you simply wish to manage your mirror using rsync (which you are already using extensively if you have other local software mirrors), then rsync access is essential.

I for one will benefit substantially from the new rsync access. Those with different needs will be best served by different solutions. There's no call for categorically stating that rsync access is not needed.

[kloczek]

That's great if all you need is the yum repo. If you need to mirror the entire repository, or if you simply wish to manage your mirror using rsync (which you are already using extensively if you have other local software mirrors), then rsync access is essential.

I for one will benefit substantially from the new rsync access. Those with different needs will be best served by different solutions. There's no call for categorically stating that rsync access is not needed.

If on base URL there are multiple repos all what you need is run this command few times.
Additionally this command copies less because it regenerates repo index after update copy. It uses remote repo indexes to figure out what needs to be copied. It checks signs of the new packages as well.
Additionally on doing mirror you can make decision about delete older packages or not which may be not available on doing next mirroring cycles (on source repo side) and which may be needed on doing rollback using your mirror.
You can make decision about copying src.rpm packages as well (if those packages are stored in repo tree and registered in repo indexes).

In other words this command does not only what rsync is able to do and it doing it over the port over which repo is normally accessible so none of the other services needs to be opened on source repo side.
Less software running on server side -> lower risk of penetration of the repo server.

[dekapri]

Does reposync let you download other $releasever repo--e.g. rhel/8 while I run rhel/7? Or another OS repo--SLES when I run RHEL?

[RcaB5ayazp7RzFt33LNJ]

Looks like rsync is disabled again, I can't find anything on their JIRA about this. Can anybody find it noted somewhere?