C Compression

Open-source C projects categorized as Compression

Top 23 C Compression Projects

  • GitHub repo zstd

    Zstandard - Fast real-time compression algorithm

    Project mention: news.ycombinator.com | 2021-02-21

    and speed the table building up by about ~2x [0]. This is insignificant for large decompressions, but if you're decompressing only a few KB, the table building time can dominate the actual decompression.

    It sort of goes to show that while Huffman codes have been around for ages, implementations can still improve, especially as the hardware we use changes.

    [0] https://github.com/facebook/zstd/pull/2271

  • GitHub repo brotli

    Brotli compression format

    Project mention: C Deep | dev.to | 2021-02-27

    Brotli - General-purpose lossless compression algorithm library. Has speeds comparable to DEFLATE, but much higher compression ratios. MIT.

  • Scout

    Get performance insights in less than 4 minutes. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.

  • GitHub repo BorgBackup

    Deduplicating archiver with compression and authenticated encryption.

    Project mention: Backing up a locally hosted Nextcloud instance | reddit.com/r/NextCloud | 2021-03-01

    rsync.net (link to their borg specific backup solution which is cheaper than regular rsync.net backups) with borg backup is what I use. All data is encrypted before it ever leaves my machine with a key and passphrase that are stored locally. The data on rsync.net would be totally useless to anyone that got their hands on it.

  • GitHub repo LZ4

    Extremely Fast Compression algorithm

    Project mention: Activision warns a standard 500GB PS4 may no longer fit Call of Duty: Warzone, Black Ops Cold War and Modern Warfare | reddit.com/r/Games | 2021-02-25

    Oodle Kraken and lz4 are a thing.

  • GitHub repo ZLib

    A massively spiffy yet delicately unobtrusive compression library.

    Project mention: C Deep | dev.to | 2021-02-27

    Zlib - Massively spiffy yet delicately unobtrusive compression library. BSD-3-Clause

  • GitHub repo LZFSE

    LZFSE compression library and command line tool

    Project mention: Why is Zip preferred on Mac while RAR seems like the favorite on Windows? | reddit.com/r/MacOS | 2021-01-21

    I'm not entirely correct, though. Because Unarchiver likely uses this compression algorithm by calling directly into the API. And there is an open-source implementation for it: lzfse/lzfse.

  • GitHub repo opus

    Modern audio compression for the internet.

    Project mention: Google Duo: More reliable voice calls for more people on Android | reddit.com/r/Android | 2021-03-02

    Currently, the royalty-free open-source codec Opus, is the most widely used codec for WebRTC-based VOIP applications.

  • GitHub repo FiniteStateEntropy

    New generation entropy codecs : Finite State Entropy and Huff0

    Project mention: C Deep | dev.to | 2021-02-27

    FiniteStateEntropy - Two highly efficient compression codecs optimized for modern CPUs. BSD-2-Clause

  • GitHub repo smaz

    Small strings compression library

    Project mention: C Deep | dev.to | 2021-02-27

    smaz - Efficient string compression library. BSD-3-Clause

  • GitHub repo Minizip

    Fork of the popular zip manipulation library found in the zlib distribution.

  • GitHub repo zlib-ng

    zlib replacement with optimizations for "next generation" systems.

  • GitHub repo zip

    A portable, simple zip library written in C (by kuba--)

    Project mention: C Deep | dev.to | 2021-02-27

    zip - Really really small zip archive processing library. Unlicense

  • GitHub repo TurboPFor

    Fastest Integer Compression

    Project mention: C Deep | dev.to | 2021-02-27

    TurboPFor - Fastest integer compression. GPL-2.0-or-later

  • GitHub repo lizard

    Lizard (formerly LZ5) is an efficient compressor with very fast decompression. It achieves compression ratio that is comparable to zip/zlib and zstd/brotli (at low and medium compression levels) at decompression speed of 1000 MB/s and faster.

    Project mention: C Deep | dev.to | 2021-02-27

    Lizard - Formerly LZ5; an efficient compressor with fast decompression. Achieves compression ratios comparable with zip and zlib at decompression speeds of 1000MB/s and faster. BSD-2-Clause

  • GitHub repo flate2-rs

    DEFLATE, gzip, and zlib bindings for Rust

  • GitHub repo pythonlibs

    A Python wrapper for the extremely fast Blosc compression library

  • GitHub repo precomp-cpp

    Precomp, C++ version - further compress already compressed files

    Project mention: Help me out with some compression tips | reddit.com/r/compression | 2021-01-06
  • GitHub repo libmspack

    A library for some loosely related Microsoft compression formats, CAB, CHM, HLP, LIT, KWAJ and SZDD.

    Project mention: Protontricks, can't install Wmp11(Permission Denied). | reddit.com/r/wine_gaming | 2021-01-26

    Reminds me I should get back and finally implement the symlink "fix" in cabextract I promised to do ._.

  • GitHub repo Snebu

    Simple Network Encrypting Backup Utility

    Project mention: Pet Project Thread February 26 2021 | reddit.com/r/sysadmin | 2021-02-27

    Would a mention of my open source backup system, Snebu (or https://github.com/derekp7/snebu) fit in this thread? Elevator pitch -- GPLv3 C code, snapshot-based, compresses, encrypts, deduplicates, can back up clients without installing an agent (just need ssh, bash, tar, and find commands on client for "pull" backups), push backups can have restricted permissions (i.e., give a client permission to push backups only, but not delete backups, or give a user restore-only permissions). Uses tar to collect the data, stores metadata in an SQLite DB on the server, files are stored in LZO format (can be read directly with lzop) (unless client-side encryption is used, but the data can still be decrypted with openssl then decompressed with lzop). Encryption is public-key based instead of needing to keep a shared symmetric key or passphrase laying around on your backup server.

  • GitHub repo apultra

    Free open-source compressor for apLib with 5-7% better ratios

    Project mention: Free open source aPLib compressor gains serious use in retrocomputing | news.ycombinator.com | 2021-01-12
  • GitHub repo bzip2-rs

    libbz2 (bzip2 compression) bindings for Rust

  • GitHub repo MV-Tractus

    A simple tool to extract motion vectors from h264 encoded videos.

    Project mention: Motion detection from video compression | reddit.com/r/computervision | 2021-01-05

    look for libraries to extract motion vector from video streams. e.g. https://github.com/vadimkantorov/mpegflow or https://github.com/jishnujayakumar/MV-Tractus

  • GitHub repo FPC

    FPC - Fast Prefix Coder

    Project mention: news.ycombinator.com | 2021-02-22
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-03-02.

Index

What are some of the best open-source Compression projects in C? This list will help you:

Project Stars
1 zstd 13,987
2 brotli 9,762
3 BorgBackup 6,994
4 LZ4 5,883
5 ZLib 2,769
6 LZFSE 1,585
7 opus 1,157
8 FiniteStateEntropy 943
9 smaz 882
10 Minizip 743
11 zlib-ng 725
12 zip 584
13 TurboPFor 519
14 lizard 401
15 flate2-rs 387
16 pythonlibs 264
17 precomp-cpp 247
18 libmspack 103
19 Snebu 92
20 apultra 57
21 bzip2-rs 53
22 MV-Tractus 47
23 FPC 20