mydumper
zstd
Our great sponsors
| mydumper | zstd | |
|---|---|---|
| 25 | 106 | |
| 2,352 | 22,407 | |
| 2.9% | 2.4% | |
| 9.6 | 9.7 | |
| 4 days ago | 4 days ago | |
| C | C | |
| GNU General Public License v3.0 only | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
mydumper
-
Restore or Migrate your MySQL Database to Railway
You can create a MySQL backup for Railway.app using the MyDumper tool or by using a backup service like SimpleBackups. Both methods involve generating a dump file that can be used for restoration.
-
Has anyone ever restored a 15GB db from mysqldump?
Maybe check mydumper which uses a multi-threaded multi-file approach so that backups are faster, but more important, restores are faster. Plus it makes it way easier to do a partial recovery (for instance, a specific table).
-
Recommend me an Azure-native solution for archival and encryption
It looks like MyDumper added support to invoke PGP, etc. https://github.com/mydumper/mydumper/issues/101
-
mysqldump for innodb + myisam mixed db
Either xtrabackup, or mydumper.
-
Is --single-transaction enough to protect mysqldump while backing up a replication replica?
Use mydumper. It has lightweight locking that can produce consistent backups without stopping replication. It also has multi-threaded sharding so it can backup much faster.
-
MariaBackup Restore to Second 'test' database
Agree with you, just let me suggest mydumper as a parallel and compressed version of mysqldump, much faster way to export and import tables- unless your tables are tiny.
-
Is replication between v5.1 and v8 possible?
Take a snapshot backup of the 5.1 server with mydumper.
-
How to transfer data across MySQL 5.7 servers, from Windows to Ubuntu?
Either way, use mydumper. It is much faster than basic mysqldump.
-
Fastest method of making a copy of a database on another server?
Check out mydumper it's multithreaded and way faster than mysqldump.
-
Migrating a database to a new server?
Better than basic mysqldump is mydumper. It has a bunch of nice features like multi-threaded parallel dumping. It can cut the dump/restore time by a huge amount.
zstd
-
SQLite VFS for ZSTD seekable format
This VFS will read a sqlite file after it has been compressed using [zstd seekable format](https://github.com/facebook/zstd/blob/dev/contrib/seekable_f...). Built to support read-only databases for full-text search. Benchmarks are provided in README.
-
Chrome Feature: ZSTD Content-Encoding
Of course, you may get different results with another dataset.
gzip (zlib -6) [ratio=32%] [compr=35Mo/s] [dec=407Mo/s]
zstd (zstd -2) [ratio=32%] [compr=356Mo/s] [dec=1067Mo/s]
NB1: The default for zstd is -3, but the table only had -2. The difference is probably small. The range is 1-22 for zstd and 1-9 for gzip.
NB2: The default program for gzip (at least with Debian) is the executable from zlib. With my workflows, libdeflate-gzip iscompatible and noticably faster.
NB3: This benchmark is 2 years old. The latest releases of zstd are much better, see https://github.com/facebook/zstd/releases
For a high compression, according to this benchmark xz can do slightly better, if you're willing to pay a 10× penalty on decompression.
xz -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
zstd -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
- Zstandard v1.5.6 – Chrome Edition
-
Optimizating Rabin-Karp Hashing
Compression, synchronization and backup systems often use rolling hash to implement "content-defined chunking", an effective form of deduplication.
In optimized implementations, Rabin-Karp is likely to be the bottleneck. See for instance https://github.com/facebook/zstd/pull/2483 which replaces a Rabin-Karp variant by a >2x faster Gear-Hashing.
- Show HN: macOS-cross-compiler – Compile binaries for macOS on Linux
-
Cyberpunk 2077 dev release
Get the data https://publicdistst.blob.core.windows.net/data/root.tar.zst magnet:?xt=urn:btih:84931cd80409ba6331f2fcfbe64ba64d4381aec5&dn=root.tar.zst How to extract https://github.com/facebook/zstd Linux (debian): `sudo apt install zstd` ``` tar -I 'zstd -d -T0' -xvf root.tar.zst ```
-
Honey, I shrunk the NPM package · Jamie Magee
I've done that experiment with zstd before.
https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md...
Not sure about brotli though.
-
How in the world should we unpack archive.org zst files on Windows?
If you want this functionality in zstd itself, check this out: https://github.com/facebook/zstd/pull/2349
- Release Zstandard v1.5.5 · facebook/zstd
- ZSTD 1.5.5 is released with a corruption fix found at Google
What are some alternatives?
rsync - An open source utility that provides fast incremental file transfer. It also has useful features for backup and restore operations among many other use cases.
LZ4 - Extremely Fast Compression algorithm
volumerize - Docker Volume Backups Multiple Backends
Snappy - A fast compressor/decompressor
symmetric-ds - SymmetricDS is database replication and file synchronization software that is platform independent, web enabled, and database agnostic. It is designed to make bi-directional data replication fast, easy, and resilient. It scales to a large number of nodes and works in near real-time across WAN and LAN networks.
LZMA - (Unofficial) Git mirror of LZMA SDK releases
RMySQL - Legacy DBI interface for MySQL
7-Zip-zstd - 7-Zip with support for Brotli, Fast-LZMA2, Lizard, LZ4, LZ5 and Zstandard
mydumper - Official MyDumper project [Moved to: https://github.com/mydumper/mydumper]
ZLib - A massively spiffy yet delicately unobtrusive compression library.
mysqld_exporter - Exporter for MySQL server metrics
brotli - Brotli compression format