Our great sponsors
-
sanoid
These are policy-driven snapshot management and replication tools which use OpenZFS for underlying next-gen storage. (Btrfs support plans are shelved unless and until btrfs becomes reliable.)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Try installing sanoid and have it replicate the dataset for you. It is a wrapper for zfs send that has lots of optimizations.
Native rsync is terrible for lots of small file as it copies each file one by one sequentially. If you have lots of cores to work with, use the fpsync utility that comes with the fpart command to run parallel rsync's. You can easily saturate a 10Gb link with multiple rsync processes in parallel