zstd VS Skia

This VFS will read a sqlite file after it has been compressed using [zstd seekable format](https://github.com/facebook/zstd/blob/dev/contrib/seekable_f...). Built to support read-only databases for full-text search. Benchmarks are provided in README.

Of course, you may get different results with another dataset.

gzip (zlib -6) [ratio=32%] [compr=35Mo/s] [dec=407Mo/s]

zstd (zstd -2) [ratio=32%] [compr=356Mo/s] [dec=1067Mo/s]

NB1: The default for zstd is -3, but the table only had -2. The difference is probably small. The range is 1-22 for zstd and 1-9 for gzip.

NB2: The default program for gzip (at least with Debian) is the executable from zlib. With my workflows, libdeflate-gzip iscompatible and noticably faster.

NB3: This benchmark is 2 years old. The latest releases of zstd are much better, see https://github.com/facebook/zstd/releases

For a high compression, according to this benchmark xz can do slightly better, if you're willing to pay a 10× penalty on decompression.

xz -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]

zstd -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]

Compression, synchronization and backup systems often use rolling hash to implement "content-defined chunking", an effective form of deduplication.

In optimized implementations, Rabin-Karp is likely to be the bottleneck. See for instance https://github.com/facebook/zstd/pull/2483 which replaces a Rabin-Karp variant by a >2x faster Gear-Hashing.

Get the data https://publicdistst.blob.core.windows.net/data/root.tar.zst magnet:?xt=urn:btih:84931cd80409ba6331f2fcfbe64ba64d4381aec5&dn=root.tar.zst How to extract https://github.com/facebook/zstd Linux (debian): `sudo apt install zstd` ``` tar -I 'zstd -d -T0' -xvf root.tar.zst ```

I've done that experiment with zstd before.

https://github.com/facebook/zstd/blob/dev/programs/zstd.1.md...

Not sure about brotli though.

If you want this functionality in zstd itself, check this out: https://github.com/facebook/zstd/pull/2349

Actually, that's not entirely accurate. The lottie-web library itself doesn't support rendering to WebGL. However, there is a package called canvaskit-wasm that wraps Skia (a graphics engine) with WebAssembly (wasm). This package includes a module called skottie which supports rendering animations into a WebGL surface. However, there is a drawback with this approach: using wasm requires loading a relatively large package, and it's uncertain whether all features are supported correctly, as the official compatibility table that tracks lottie support on different platforms does not include skottie.

Skia is pretty great if you can get it running.

The imgs are mainly used by Tencent QQ and Baidu Tieba users. QQ and Tieba compress imgs by default, so the popular imgs (in China we call them "屌图" or "表情包") would be compressed million of times during the spreading, causing the super low quality. These APPs on Android are using Skia for image processing, which suffering from a legendary bug: result would be more green. The bug was fixed in 2016: https://github.com/google/skia/commit/c7d01d3e1d3621907c27b283fb7f8b6e177c629d

There’s also Skia by Google. Used by Android and Google Chrome.