SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 C Compression Projects
-
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
cute_headers
Collection of cross-platform one-file C/C++ libraries with no dependencies, primarily used for games
-
-
-
cstore_fdw
Columnar storage extension for Postgres built as a foreign data wrapper. Check out https://github.com/citusdata/citus for a modernized columnar storage implementation built as a table access method.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
astc-encoder
The Arm ASTC Encoder, a compressor for the Adaptive Scalable Texture Compression data format.
-
-
p7zip
A new p7zip fork with additional codecs and improvements (forked from https://sourceforge.net/projects/sevenzip/ AND https://sourceforge.net/projects/p7zip/).
-
lizard
Lizard (formerly LZ5) is an efficient compressor with very fast decompression. It achieves compression ratio that is comparable to zip/zlib and zstd/brotli (at low and medium compression levels) at decompression speed of 1000 MB/s and faster. (by inikep)
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Of course, you may get different results with another dataset.
gzip (zlib -6) [ratio=32%] [compr=35Mo/s] [dec=407Mo/s]
zstd (zstd -2) [ratio=32%] [compr=356Mo/s] [dec=1067Mo/s]
NB1: The default for zstd is -3, but the table only had -2. The difference is probably small. The range is 1-22 for zstd and 1-9 for gzip.
NB2: The default program for gzip (at least with Debian) is the executable from zlib. With my workflows, libdeflate-gzip iscompatible and noticably faster.
NB3: This benchmark is 2 years old. The latest releases of zstd are much better, see https://github.com/facebook/zstd/releases
For a high compression, according to this benchmark xz can do slightly better, if you're willing to pay a 10× penalty on decompression.
xz -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
zstd -9 [ratio=23%] [compr=2.6Mo/s] [dec=88Mo/s]
LZ4 is a bit more complicated, but seems faster: https://github.com/lz4/lz4/blob/dev/doc/lz4_Block_format.md
Opus doesn't support 44.1 kHz because compatibility and effort/benefit ratio:
https://github.com/xiph/opus/issues/43
The browser audio limitation is presumably a workaround to some bug or performance limitation that was relevant at some point in history (the site was created in 2014).
Project mention: Moving a Billion Postgres Rows on a $100 Budget | news.ycombinator.com | 2024-02-21Columnar store PostgreSQL extension exists, here are two but I think I’m missing at least another one:
https://github.com/citusdata/cstore_fdw
https://github.com/hydradatabase/hydra
You can also connect other stores using the foreign data wrappers, like parquet files stored on an object store, duckdb, clickhouse… though the joins aren’t optimised as PostgreSQL would do full scan on the external table when joining.
Project mention: Show HN: Pzip- blazing fast concurrent zip archiver and extractor | news.ycombinator.com | 2023-09-24Please note that allowing for 2% bigger resulting file could mean huge speedup in these circumstances even with the same compression routines, seeing these benchmarks of zlib and zlib-ng for different compression levels:
https://github.com/zlib-ng/zlib-ng/discussions/871
IMO the fair comparison of the real speed improvement brought by a new program is only between the almost identical resulting compressed sizes.
Project mention: Intel QuickAssist Technology Zstandard Plugin for Zstandard | news.ycombinator.com | 2023-08-16It's obsolete. It's limited to 32KB LZ window with huffman coding. Zstd can use a much larger window (8MB recommended) and a much better entropy coder: https://github.com/Cyan4973/FiniteStateEntropy
For a benchmark on a standard set: https://github.com/inikep/lzbench/blob/master/lzbench18_sort...
Project mention: Show HN: Time Series Benchmark TurboPFor,TurboFloat,TurboFloat LzX,TurboGorilla | news.ycombinator.com | 2023-06-25
Kamila Szewczyk is working on a bzip3 to improve the state-of-the-art in the domain of compressors based on Burrows-Wheeler:
https://github.com/kspalaiologos/bzip3
I’m keeping fingers crossed for the project. Especially given that the author is 19 and her best work is yet to come.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
C Compression related posts
- Xz: Update maintainer and author info. The other maintainer suddenly disappeared
- Thanks Andres Freud
- The xz-utils backdoor has been removed
- Chrome Feature: ZSTD Content-Encoding
- Xz Repository Censored by GitHub
- Zstandard v1.5.6 – Chrome Edition
- Optimizating Rabin-Karp Hashing
-
A note from our sponsor - SaaSHub
www.saashub.com | 26 Apr 2024
Index
What are some of the best open-source Compression projects in C? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | zstd | 22,407 |
| 2 | LZ4 | 9,208 |
| 3 | ZLib | 5,278 |
| 4 | cute_headers | 4,105 |
| 5 | opus | 2,092 |
| 6 | LZFSE | 1,748 |
| 7 | cstore_fdw | 1,738 |
| 8 | zlib-ng | 1,440 |
| 9 | zip | 1,315 |
| 10 | FiniteStateEntropy | 1,263 |
| 11 | Minizip-ng | 1,160 |
| 12 | smaz | 1,131 |
| 13 | astc-encoder | 975 |
| 14 | c-blosc | 958 |
| 15 | lzbench | 841 |
| 16 | TurboPFor | 743 |
| 17 | p7zip | 735 |
| 18 | squashfs-tools | 708 |
| 19 | bzip3 | 643 |
| 20 | lizard | 633 |
| 21 | zson | 527 |
| 22 | simdcomp | 474 |
| 23 | gozstd | 412 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com