Civic Auth comes with multiple SSO options, optional embedded wallets, and user management โ all implemented with just a few lines of code. Start building today. Learn more โ
Top 23 Compression Open-Source Projects
-
DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Project mention: DeepSpeed-Domino: Communication-Free LLM Training Engine | news.ycombinator.com | 2024-11-26 -
Civic Auth
Auth in Less Than 5 Minutes. Civic Auth comes with multiple SSO options, optional embedded wallets, and user management โ all implemented with just a few lines of code. Start building today.
-
CyberChef
The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis
Project mention: Show HN: JSONXMLKit โ Format JSON/XML and Decode Base64 in the Browser | news.ycombinator.com | 2025-04-22I tend to use CyberChef for this sort of thing: https://gchq.github.io/CyberChef/
-
Project mention: Why do I find Rust inadequate for text compression codecs? | news.ycombinator.com | 2025-03-07
If zstd give you an error and you don't handle it, the next calls may cause UB, so it kinda does both things.
https://github.com/facebook/zstd/blob/b16d193512d3ded82fd584...
-
digital_video_introduction
A hands-on introduction to video technology: image, video, codec (av1, vp9, h265) and more (ffmpeg encoding). Translations: ๐บ๐ธ ๐จ๐ณ ๐ฏ๐ต ๐ฎ๐น ๐ฐ๐ท ๐ท๐บ ๐ง๐ท ๐ช๐ธ
-
Projects like Brotli aren't built to maximize personal profit; they're driven by passion and a genuine love for software engineering.
It's clear that the industry is shifting from being geeky and nerdy to being more business and management focused.
[0] https://github.com/google/brotli
-
-
Project mention: Chunking Attacks on File Backup Services Using Content-De๏ฌned Chunking [pdf] | news.ycombinator.com | 2025-03-22
-
CodeRabbit
CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
-
-
-
-
draco
Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics.
-
-
> Making a nanite mesh is complicated, with a lot of internal offsets for linking, and so far only Unreal Engine's editor does it.
meshoptimizer [1] is an OSS implementation of meshlet generation, which is what most people think of when they think of "Nanite's algorithm". Bevy, mentioned in a sibling reply, uses meshoptimizer as the generation tool.
(Strictly speaking, "Nanite" is a brand name that encompasses a large collection of techniques, including meshlets, software rasterization, streaming, etc. For clarity during technical discussions, I prefer to talk about individual techniques, since they're really separate, even though they complement one another. For example, software rasterization can be used without meshlets if your triangles are really small. Streaming can be useful even if you aren't using meshlets. And so on.)
[1]: https://github.com/zeux/meshoptimizer
-
Project mention: Which open-source projects are widely used but maintained by just a few people? | news.ycombinator.com | 2024-09-09
-
Looks like it depends on https://github.com/nodeca/pako for the zlib compression.
> Almost as fast in modern JS engines as C implementation (see benchmarks).
Impressive, although zlib itself is no longer the bar to beat for zlib, I think that goes to https://github.com/zlib-ng/zlib-ng these days
-
-
PeaZip
Free Zip / Unzip software and Rar file extractor. Cross-platform file and archive manager. Features volume spanning, compression, authenticated encryption. Supports 7Z, 7-Zip sfx, ACE, ARJ, Brotli, BZ2, CAB, CHM, CPIO, DEB, GZ, ISO, JAR, LHA/LZH, NSIS, OOo, PAQ/LPAQ, PEA, QUAD, RAR, RPM, split, TAR, Z, ZIP, ZIPX, Zstandard.
-
I didn't know Go reserved some registers for its own use. For libraries that don't use cgo but do use assembly [1], does this mean they have to generate that assembly with a compiler flag that avoids reserved registers?
[1] A common optimization technique, like in the superb https://github.com/klauspost/compress
-
cute_headers
Collection of cross-platform one-file C/C++ libraries with no dependencies, primarily used for games
Project mention: Collection of one-file C/C++ libraries with no dependencies, primarily for games | news.ycombinator.com | 2025-02-08Since you left this "optimize me" comment here:
https://github.com/RandyGaul/cute_headers/blob/755849fc2819d...
See an optimized quaternion multiplication implementation in SSE by me here:
https://stackoverflow.com/questions/18542894/how-to-multiply...
-
caesium-image-compressor
Caesium is an image compression software that helps you store, send and share digital pictures, supporting JPG, PNG, WebP and TIFF formats. You can quickly reduce the file size (and resolution, if you want) by preserving the overall quality of the image.
-
SharpZipLib
#ziplib is a Zip, GZip, Tar and BZip2 library written entirely in C# for the .NET platform.
Project mention: Preserve File Creation and Modification Date When Downloading in a ZIP File | dev.to | 2025-03-12You need a ZIP library that supports writing this NTFS extra field. I came across SharpZipLib. It hasnโt been updated since August 2023, but it works perfectly fine for a server-side packaging solution.
-
zip.js
JavaScript library to zip and unzip files supporting multi-core compression, compression streams, zip64, split files and encryption.
-
StegCloak
Hide secrets with invisible characters in plain text securely using passwords ๐ง๐ปโโ๏ธโญ
Project mention: Ategcloak: Hide secrets with invisible Unicode characters with passwords | news.ycombinator.com | 2025-02-12 -
InfluxDB
InfluxDB high-performance time series database. Collect, organize, and act on massive volumes of high-resolution data to power real-time intelligent systems.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Compression discussion
Compression related posts
-
Snappy: Fast Compressor/Decompressor
-
The Road Ahead for JSZip
-
Chunking Attacks on File Backup Services Using Content-De๏ฌned Chunking [pdf]
-
LZAV 4.9 โ Fast Data Compression Algorithm Release
-
zlib-ng: zlib replacement with optimizations for "next generation" systems
-
Preserve File Creation and Modification Date When Downloading in a ZIP File
-
Datei-Erstellungs- und -Bearbeitungsdatum beim Download in einer ZIP-Datei beibehalten
-
A note from our sponsor - Civic Auth
www.civic.com | 28 Apr 2025
Index
What are some of the best open-source Compression projects? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | DeepSpeed | 38,124 |
| 2 | CyberChef | 30,996 |
| 3 | zstd | 24,771 |
| 4 | digital_video_introduction | 15,752 |
| 5 | brotli | 13,987 |
| 6 | PaddleNLP | 12,539 |
| 7 | BorgBackup | 11,748 |
| 8 | LZ4 | 10,862 |
| 9 | JSZip | 9,994 |
| 10 | Compressor | 7,115 |
| 11 | draco | 6,735 |
| 12 | Snappy | 6,312 |
| 13 | meshoptimizer | 6,193 |
| 14 | ZLib | 6,122 |
| 15 | pako | 5,794 |
| 16 | CompactGUI | 5,506 |
| 17 | PeaZip | 5,087 |
| 18 | compress | 5,021 |
| 19 | cute_headers | 4,618 |
| 20 | caesium-image-compressor | 4,134 |
| 21 | SharpZipLib | 3,825 |
| 22 | zip.js | 3,607 |
| 23 | StegCloak | 3,506 |
Sponsored
Auth in Less Than 5 Minutes
Civic Auth comes with multiple SSO options, optional embedded wallets, and user management โ all implemented with just a few lines of code. Start building today.
www.civic.com