Computing Adler32 Checksums at 41 GB/s

• fpng

  10 870 3.7 C

  Super fast C++ .PNG writer/reader

  This was actually considered, and other libraries do ignore checksums, or at least have options to:

  https://github.com/richgel999/fpng/issues/9

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• oss-nvjpg

  1 15 0.0 C++

  Hardware-accelerated JPEG decoding on the Nvidia Tegra X1

• wuffs

  88 4,162 9.4 C

  Wrangling Untrusted File Formats Safely

  

  https://github.com/google/wuffs/blob/main/std/adler32/common...

  Like the fpng implementation, it's SSE (128-bit registers), but the inner loop eats 32 bytes at a time, not 16.

  "Wuffs’ Adler-32 implementation is around 6.4x faster (11.3GB/s vs 1.76GB/s) than the one from zlib-the-library", which IIUC is roughly comparable to the article's defer32.

• libdeflate

  7 1,004 8.4 C

  Heavily optimized library for DEFLATE/zlib/gzip compression and decompression

  Note that libdeflate has used essentially the same method since 2016 (https://github.com/ebiggers/libdeflate/blob/v0.4/lib/adler32...), though I recently switched it to use a slightly different method (https://github.com/ebiggers/libdeflate/blob/v1.12/lib/x86/ad...) that performs more consistently across different families of x86 CPUs.

• zlib-ng

  14 1,556 9.1 C

  zlib replacement with optimizations for "next generation" systems.

  

  zlib-ng also has adler32 implementations optimized for various architectures: https://github.com/zlib-ng/zlib-ng

  Might be interesting to benchmark their implementation too to see how it compares.

Suggest a related project