C Compression

Open-source C projects categorized as Compression

Top 20 C Compression Projects

  • zstd

    Zstandard - Fast real-time compression algorithm

    Latest mention: Linus Torvalds on the importance of ECC in main memory (RAM) | news.ycombinator.com | 2021-01-03

    Yes. I work at Facebook on data compression.

    The libraries we maintain (1) are responsible for a non-trivial part of Facebook's overall compute footprint, (2) should basically never fail of their own accord, and (3) have pretty good error monitoring. So my team is operating what is effectively (among other things) a very sensitive detector for hardware failure.

    And indeed we see examples all the time of blobs that fail to decompress, and usually when we dig in we find that the blob is only a single bit-flip away from a blob that decompresses successfully into a syntactically correct message. I can't share numbers, but, off the top of my head, I think it's the largest source of failures we see. It happens frequently enough that I wrote a tool to automate checking [0].

    So yes. It happens. Pretty frequently, in the sense that if you're doing xillions of operations a day, a one-in-a-xillion failure happens all the time.

    [0] https://github.com/facebook/zstd/tree/dev/contrib/diagnose_c...

  • brotli

    Brotli compression format

    Latest mention: Serving compressed static assets with HTTP in Go 1.16 | dev.to | 2021-01-17

    First, it checks if request accepts compressed encoding of response. If yes, it checks if script.js.gz (or script.js.br for brotli) is available in embedded file system and serves contents of script.js.gz.

  • borg

    Deduplicating archiver with compression and authenticated encryption.

    Latest mention: Encrypted Backup Shootout | news.ycombinator.com | 2021-01-02

    borg (c) - https://github.com/borgbackup/borg

  • lz4

    Extremely Fast Compression algorithm

    Latest mention: lz4_flex 0.7.2 reaches parity with cpp reference implementation on speed and ratio | reddit.com/r/rust | 2021-01-18

    It's hard to tell, how much since the latest lz4 1.9 has been improved, because the (benchmarks)[https://github.com/lz4/lz4/commit/31763c59fca9a38bc697cd37f591d6ba385f7614#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5] are executed with different processors

  • zlib

    A massively spiffy yet delicately unobtrusive compression library.

  • lzfse

    LZFSE compression library and command line tool

    Latest mention: Why is Zip preferred on Mac while RAR seems like the favorite on Windows? | reddit.com/r/MacOS | 2021-01-21

    I'm not entirely correct, though. Because Unarchiver likely uses this compression algorithm by calling directly into the API. And there is an open-source implementation for it: lzfse/lzfse.

  • opus

    Modern audio compression for the internet.

    Latest mention: Using Opus in Unity3d. | reddit.com/r/dissonance_voip | 2020-12-31

    If you don't own Dissonance it'll be a little more difficult. You'll need to compile opus into a dll (source code here) and then write a C# wrapper to call the opus methods from C# (using DllImport).

  • FiniteStateEntropy

    New generation entropy codecs : Finite State Entropy and Huff0

  • smaz

    Small strings compression library

  • minizip

    Fork of the popular zip manipulation library found in the zlib distribution.

  • zlib-ng

    zlib replacement with optimizations for "next generation" systems.

  • flate2-rs

    DEFLATE, gzip, and zlib bindings for Rust

  • python-blosc

    A Python wrapper for the extremely fast Blosc compression library

  • precomp-cpp

    Precomp, C++ version - further compress already compressed files

    Latest mention: Help me out with some compression tips | reddit.com/r/compression | 2021-01-06
  • TinyJpeg

    Single header lib for JPEG encoding. Public domain. C99. stb style.

  • snebu

    Simple Network Encrypting Backup Utility

    Latest mention: Encrypted Backup Shootout | news.ycombinator.com | 2021-01-02

    snebu (c) - https://github.com/derekp7/snebu

  • apultra

    Free open-source compressor for apLib with 5-7% better ratios

    Latest mention: Free open source aPLib compressor gains serious use in retrocomputing | news.ycombinator.com | 2021-01-12
  • bzip2-rs

    libbz2 (bzip2 compression) bindings for Rust

  • MV-Tractus

    A simple tool to extract motion vectors from h264 encoded videos.

    Latest mention: Motion detection from video compression | reddit.com/r/computervision | 2021-01-05

    look for libraries to extract motion vector from video streams. e.g. https://github.com/vadimkantorov/mpegflow or https://github.com/jishnujayakumar/MV-Tractus

  • lzmat

    git mirror of LZMAT (http://www.matcode.com/lzmat.htm)

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-01-21.

Index

What are some of the best open-source Compression projects in C? This list will help you:

Project Stars
1 zstd 13,801
2 brotli 9,610
3 borg 6,891
4 lz4 5,647
5 zlib 2,738
6 lzfse 1,578
7 opus 1,127
8 FiniteStateEntropy 934
9 smaz 875
10 minizip 735
11 zlib-ng 712
12 flate2-rs 378
13 python-blosc 261
14 precomp-cpp 239
15 TinyJpeg 138
16 snebu 88
17 apultra 52
18 bzip2-rs 50
19 MV-Tractus 46
20 lzmat 2