zstd

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • zstd

    Zstandard - Fast real-time compression algorithm

    They have a nice table on that page: https://github.com/facebook/zstd#benchmarks

    Looking at that table, I think LZ4 is a winner. The compression ratio is not too far, compression speed is slightly faster, decompression speed is significantly faster, the code is much simpler so the compiled binary is smaller, and the project is unrelated to Facebook.

  • zstd-proxy

    High-performance real-time TCP compression

    I needed to compress a TCP stream [0] for a block-storage project and I've been very impressed with zstd. Combined with io_uring, I'm getting less CPU usage and higher throughput to send and compress data than using socat to send uncompressed data.

    [0]: https://github.com/fathyb/zstd-proxy

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  • sqlite-zstd

    Transparent dictionary-based row-level compression for SQLite

  • LZ4

    Extremely Fast Compression algorithm

    > The downside of lz4 is that it can’t be configured to run at higher & slower compression ratios.

    lz4 has some level of configurability? https://github.com/lz4/lz4/blob/v1.9.4/lib/lz4frame.h#L194

    There's also LZ4_HC.

  • compress

    Optimized Go Compression Packages

    There is a reasonably feature complete implementation of Zstd for Go: https://github.com/klauspost/compress/tree/master/zstd

    It may not offer the same API 1:1, but it has no interoperability issues that I've encountered. So, I just think no one has bothered to implement it in Rust because most use cases don't mind the added bloat you're talking about. Plus, other comments I've seen suggest that you can actually tune the size of the zstd library, although I'm not sure if the Rust bindings expose that.

  • PeaZip

    Free Zip / Unzip software and Rar file extractor. Cross-platform file and archive manager. Features volume spanning, compression, authenticated encryption. Supports 7Z, 7-Zip sfx, ACE, ARJ, Brotli, BZ2, CAB, CHM, CPIO, DEB, GZ, ISO, JAR, LHA/LZH, NSIS, OOo, PAQ/LPAQ, PEA, QUAD, RAR, RPM, split, TAR, Z, ZIP, ZIPX, Zstandard.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts