zstd

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • ONLYOFFICE ONLYOFFICE Docs — document collaboration in your environment
  • SonarQube - Static code analysis for 29 languages.
  • CodiumAI - TestGPT | Generating meaningful tests for busy devs
  • InfluxDB - Access the most powerful time series database as a service
  • zstd

    Zstandard - Fast real-time compression algorithm

    They have a nice table on that page: https://github.com/facebook/zstd#benchmarks

    Looking at that table, I think LZ4 is a winner. The compression ratio is not too far, compression speed is slightly faster, decompression speed is significantly faster, the code is much simpler so the compiled binary is smaller, and the project is unrelated to Facebook.

  • zstd-proxy

    High-performance real-time TCP compression

    I needed to compress a TCP stream [0] for a block-storage project and I've been very impressed with zstd. Combined with io_uring, I'm getting less CPU usage and higher throughput to send and compress data than using socat to send uncompressed data.

    [0]: https://github.com/fathyb/zstd-proxy

  • ONLYOFFICE

    ONLYOFFICE Docs — document collaboration in your environment. Powerful document editing and collaboration in your app or environment. Ultimate security, API and 30+ ready connectors, SaaS or on-premises

  • sqlite-zstd

    Transparent dictionary-based row-level compression for SQLite

  • LZ4

    Extremely Fast Compression algorithm

    > The downside of lz4 is that it can’t be configured to run at higher & slower compression ratios.

    lz4 has some level of configurability? https://github.com/lz4/lz4/blob/v1.9.4/lib/lz4frame.h#L194

    There's also LZ4_HC.

  • compress

    Optimized Go Compression Packages

    There is a reasonably feature complete implementation of Zstd for Go: https://github.com/klauspost/compress/tree/master/zstd

    It may not offer the same API 1:1, but it has no interoperability issues that I've encountered. So, I just think no one has bothered to implement it in Rust because most use cases don't mind the added bloat you're talking about. Plus, other comments I've seen suggest that you can actually tune the size of the zstd library, although I'm not sure if the Rust bindings expose that.

  • PeaZip

    Free Zip / Unzip software and Rar file extractor. Cross-platform file and archive manager. Features volume spanning, compression, authenticated encryption. Supports 7Z, 7-Zip sfx, ACE, ARJ, Brotli, BZ2, CAB, CHM, CPIO, DEB, GZ, ISO, JAR, LHA/LZH, NSIS, OOo, PAQ/LPAQ, PEA, QUAD, RAR, RPM, split, TAR, Z, ZIP, ZIPX, Zstandard.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts