zstd VS mydumper

Of course, you may get different results with another dataset.

gzip (zlib -6) [ratio=32%] [compr=35Mo/s] [dec=407Mo/s]

zstd (zstd -2) [ratio=32%] [compr=356Mo/s] [dec=1067Mo/s]

NB1: The default for zstd is -3, but the table only had -2. The difference is probably small. The range is 1-22 for zstd and 1-9 for gzip.

NB2: The default program for gzip (at least with Debian) is the executable from zlib. With my workflows, libdeflate-gzip iscompatible and noticably faster.

NB3: This benchmark is 2 years old. The latest releases of zstd are much better, see https://github.com/facebook/zstd/releases

For a high compression, according to this benchmark xz can do slightly better, if you're willing to pay a 10× penalty on decompression.

xz -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]

zstd -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]

Compression, synchronization and backup systems often use rolling hash to implement "content-defined chunking", an effective form of deduplication.

In optimized implementations, Rabin-Karp is likely to be the bottleneck. See for instance https://github.com/facebook/zstd/pull/2483 which replaces a Rabin-Karp variant by a >2x faster Gear-Hashing.

Get the data https://publicdistst.blob.core.windows.net/data/root.tar.zst magnet:?xt=urn:btih:84931cd80409ba6331f2fcfbe64ba64d4381aec5&dn=root.tar.zst How to extract https://github.com/facebook/zstd Linux (debian): `sudo apt install zstd` ``` tar -I 'zstd -d -T0' -xvf root.tar.zst ```

I've done that experiment with zstd before.

https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md...

Not sure about brotli though.

If you want this functionality in zstd itself, check this out: https://github.com/facebook/zstd/pull/2349

The story says : "ZSTD 1.5.5 is released with a corruption fix found at Google"

You can create a MySQL backup for Railway.app using the MyDumper tool or by using a backup service like SimpleBackups. Both methods involve generating a dump file that can be used for restoration.

Maybe check mydumper which uses a multi-threaded multi-file approach so that backups are faster, but more important, restores are faster. Plus it makes it way easier to do a partial recovery (for instance, a specific table).

It looks like MyDumper added support to invoke PGP, etc. https://github.com/mydumper/mydumper/issues/101

Either xtrabackup, or mydumper.

Use mydumper. It has lightweight locking that can produce consistent backups without stopping replication. It also has multi-threaded sharding so it can backup much faster.

Agree with you, just let me suggest mydumper as a parallel and compressed version of mysqldump, much faster way to export and import tables- unless your tables are tiny.

Take a snapshot backup of the 5.1 server with mydumper.

Either way, use mydumper. It is much faster than basic mysqldump.

Check out mydumper it's multithreaded and way faster than mysqldump.

Better than basic mysqldump is mydumper. It has a bunch of nice features like multi-threaded parallel dumping. It can cut the dump/restore time by a huge amount.