Our great sponsors
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
rsync
An open source utility that provides fast incremental file transfer. It also has useful features for backup and restore operations among many other use cases.
It compresses the exported data using zstd, a very fast compression algorithm by Facebook.
We will talk about all this below. A very nice intro to the performance of the shell dumps / imports is also in this series of posts by Kenny Gryp. The benchmarks show that MySQL shell outperforms all other solutions (specifically mysqldump, mysqlpump and mydumper) in most situations.
Before we can import the data into a local MySQL instance, we need to download the relevant part of the dump to the local machine. Our import script uses rsync with a set of pre-configured --include and --exclude options under the hood to do that. Overall, we can typically download only about 2 GB of data out of the 23 GB dump.