Ad Widget

Collapse

Timescaledb conversion taking a long time for Zabbix Postgresql database

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • sysadm101
    Junior Member
    • Jun 2023
    • 4

    #1

    Timescaledb conversion taking a long time for Zabbix Postgresql database


    Hi,
    I am trying to migrate Zabbix postgresql database to Timescaledb (5 TimescaleDB setup 1) but the whole process has been super slow and it takes days to complete.

    I have tried using the timescaledb-tune utility or even manually adding aggressive values but the timescaledb conversion still wont utilize the resources on the host. The CPU never gets above 5% and allocating more memory to PostgreSQL does not help.

    shared_buffers=50% of the system memory
    max_worker_processes= 200
    ma_parallel_workers_per_gather = 4
    max_parallel_workers= 8

    About 700 GB data and took about 5 days to complete the Timescaledb conversion.

    Am I missing something? Any help with how I can make Timescaledb use most of the system resources for the Timescaledb conversion is appreciated!
  • eldar2116
    Junior Member
    • Jun 2018
    • 16

    #2
    I got same during upgrading from 5 to 6. They use COPY in the scripts, COPY works on 1 CPU core. On my SSD I got 3 days of conversion for 1TB database.
    Try wal log minimal, separate WAL and DATA directories on different block devices. And I found this https://github.com/timescale/timescaledb-parallel-copy but didn't check it myself, be careful.

    Comment

    Working...