Compression

Top 23 Compression Open-Source Projects

Compression
  1. DeepSpeed

    DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

    Project mention: DeepSpeed-Domino: Communication-Free LLM Training Engine | news.ycombinator.com | 2024-11-26
  2. Civic Auth

    Auth in Less Than 5 Minutes. Civic Auth comes with multiple SSO options, optional embedded wallets, and user management โ€” all implemented with just a few lines of code. Start building today.

    Civic Auth logo
  3. CyberChef

    The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis

    Project mention: Show HN: JSONXMLKit โ€“ Format JSON/XML and Decode Base64 in the Browser | news.ycombinator.com | 2025-04-22

    I tend to use CyberChef for this sort of thing: https://gchq.github.io/CyberChef/

  4. zstd

    Zstandard - Fast real-time compression algorithm

    Project mention: Why do I find Rust inadequate for text compression codecs? | news.ycombinator.com | 2025-03-07

    If zstd give you an error and you don't handle it, the next calls may cause UB, so it kinda does both things.

    https://github.com/facebook/zstd/blob/b16d193512d3ded82fd584...

  5. digital_video_introduction

    A hands-on introduction to video technology: image, video, codec (av1, vp9, h265) and more (ffmpeg encoding). Translations: ๐Ÿ‡บ๐Ÿ‡ธ ๐Ÿ‡จ๐Ÿ‡ณ ๐Ÿ‡ฏ๐Ÿ‡ต ๐Ÿ‡ฎ๐Ÿ‡น ๐Ÿ‡ฐ๐Ÿ‡ท ๐Ÿ‡ท๐Ÿ‡บ ๐Ÿ‡ง๐Ÿ‡ท ๐Ÿ‡ช๐Ÿ‡ธ

  6. brotli

    Brotli compression format

    Project mention: A Career Ending Mistake | news.ycombinator.com | 2024-11-24

    Projects like Brotli aren't built to maximize personal profit; they're driven by passion and a genuine love for software engineering.

    It's clear that the industry is shifting from being geeky and nerdy to being more business and management focused.

    [0] https://github.com/google/brotli

  7. PaddleNLP

    Easy-to-use and powerful LLM and SLM library with awesome model zoo.

  8. BorgBackup

    Deduplicating archiver with compression and authenticated encryption.

    Project mention: Chunking Attacks on File Backup Services Using Content-De๏ฌned Chunking [pdf] | news.ycombinator.com | 2025-03-22
  9. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  10. LZ4

    Extremely Fast Compression algorithm

    Project mention: LZ4 v1.10.0 โ€“ Multicores Edition | news.ycombinator.com | 2024-07-27
  11. JSZip

    Create, read and edit .zip files with Javascript

    Project mention: The Road Ahead for JSZip | news.ycombinator.com | 2025-03-27
  12. Compressor

    An android image compression library.

  13. draco

    Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics.

  14. Snappy

    A fast compressor/decompressor

    Project mention: Snappy: Fast Compressor/Decompressor | news.ycombinator.com | 2025-04-21
  15. meshoptimizer

    Mesh optimization library that makes meshes smaller and faster to render

    Project mention: UE5 Nanite in WebGPU | news.ycombinator.com | 2024-09-05

    > Making a nanite mesh is complicated, with a lot of internal offsets for linking, and so far only Unreal Engine's editor does it.

    meshoptimizer [1] is an OSS implementation of meshlet generation, which is what most people think of when they think of "Nanite's algorithm". Bevy, mentioned in a sibling reply, uses meshoptimizer as the generation tool.

    (Strictly speaking, "Nanite" is a brand name that encompasses a large collection of techniques, including meshlets, software rasterization, streaming, etc. For clarity during technical discussions, I prefer to talk about individual techniques, since they're really separate, even though they complement one another. For example, software rasterization can be used without meshlets if your triangles are really small. Streaming can be useful even if you aren't using meshlets. And so on.)

    [1]: https://github.com/zeux/meshoptimizer

  16. ZLib

    A massively spiffy yet delicately unobtrusive compression library.

    Project mention: Which open-source projects are widely used but maintained by just a few people? | news.ycombinator.com | 2024-09-09
  17. pako

    high speed zlib port to javascript, works in browser & node.js

    Project mention: Fast-PNG: PNG image decoder and encoder | news.ycombinator.com | 2025-03-11

    Looks like it depends on https://github.com/nodeca/pako for the zlib compression.

    > Almost as fast in modern JS engines as C implementation (see benchmarks).

    Impressive, although zlib itself is no longer the bar to beat for zlib, I think that goes to https://github.com/zlib-ng/zlib-ng these days

  18. CompactGUI

    Transparently compress active games and programs using Windows 10/11 APIs

  19. PeaZip

    Free Zip / Unzip software and Rar file extractor. Cross-platform file and archive manager. Features volume spanning, compression, authenticated encryption. Supports 7Z, 7-Zip sfx, ACE, ARJ, Brotli, BZ2, CAB, CHM, CPIO, DEB, GZ, ISO, JAR, LHA/LZH, NSIS, OOo, PAQ/LPAQ, PEA, QUAD, RAR, RPM, split, TAR, Z, ZIP, ZIPX, Zstandard.

    Project mention: PeaZip 10.0.0 Released | news.ycombinator.com | 2024-10-26
  20. compress

    Optimized Go Compression Packages

    Project mention: Runtime code generation and execution in Go | news.ycombinator.com | 2024-05-29

    I didn't know Go reserved some registers for its own use. For libraries that don't use cgo but do use assembly [1], does this mean they have to generate that assembly with a compiler flag that avoids reserved registers?

    [1] A common optimization technique, like in the superb https://github.com/klauspost/compress

  21. cute_headers

    Collection of cross-platform one-file C/C++ libraries with no dependencies, primarily used for games

    Project mention: Collection of one-file C/C++ libraries with no dependencies, primarily for games | news.ycombinator.com | 2025-02-08

    Since you left this "optimize me" comment here:

    https://github.com/RandyGaul/cute_headers/blob/755849fc2819d...

    See an optimized quaternion multiplication implementation in SSE by me here:

    https://stackoverflow.com/questions/18542894/how-to-multiply...

  22. caesium-image-compressor

    Caesium is an image compression software that helps you store, send and share digital pictures, supporting JPG, PNG, WebP and TIFF formats. You can quickly reduce the file size (and resolution, if you want) by preserving the overall quality of the image.

  23. SharpZipLib

    #ziplib is a Zip, GZip, Tar and BZip2 library written entirely in C# for the .NET platform.

    Project mention: Preserve File Creation and Modification Date When Downloading in a ZIP File | dev.to | 2025-03-12

    You need a ZIP library that supports writing this NTFS extra field. I came across SharpZipLib. It hasnโ€™t been updated since August 2023, but it works perfectly fine for a server-side packaging solution.

  24. zip.js

    JavaScript library to zip and unzip files supporting multi-core compression, compression streams, zip64, split files and encryption.

  25. StegCloak

    Hide secrets with invisible characters in plain text securely using passwords ๐Ÿง™๐Ÿปโ€โ™‚๏ธโญ

    Project mention: Ategcloak: Hide secrets with invisible Unicode characters with passwords | news.ycombinator.com | 2025-02-12
  26. InfluxDB

    InfluxDB high-performance time series database. Collect, organize, and act on massive volumes of high-resolution data to power real-time intelligent systems.

    InfluxDB logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Compression discussion

Log in or Post with

Compression related posts

  • Snappy: Fast Compressor/Decompressor

    1 project | news.ycombinator.com | 21 Apr 2025
  • The Road Ahead for JSZip

    1 project | news.ycombinator.com | 27 Mar 2025
  • Chunking Attacks on File Backup Services Using Content-De๏ฌned Chunking [pdf]

    1 project | news.ycombinator.com | 22 Mar 2025
  • LZAV 4.9 โ€“ Fast Data Compression Algorithm Release

    1 project | news.ycombinator.com | 20 Mar 2025
  • zlib-ng: zlib replacement with optimizations for "next generation" systems

    2 projects | news.ycombinator.com | 16 Mar 2025
  • Preserve File Creation and Modification Date When Downloading in a ZIP File

    1 project | dev.to | 12 Mar 2025
  • Datei-Erstellungs- und -Bearbeitungsdatum beim Download in einer ZIP-Datei beibehalten

    1 project | dev.to | 12 Mar 2025
  • A note from our sponsor - Civic Auth
    www.civic.com | 28 Apr 2025
    Civic Auth comes with multiple SSO options, optional embedded wallets, and user management โ€” all implemented with just a few lines of code. Start building today. Learn more โ†’

Index

What are some of the best open-source Compression projects? This list will help you:

# Project Stars
1 DeepSpeed 38,124
2 CyberChef 30,996
3 zstd 24,771
4 digital_video_introduction 15,752
5 brotli 13,987
6 PaddleNLP 12,539
7 BorgBackup 11,748
8 LZ4 10,862
9 JSZip 9,994
10 Compressor 7,115
11 draco 6,735
12 Snappy 6,312
13 meshoptimizer 6,193
14 ZLib 6,122
15 pako 5,794
16 CompactGUI 5,506
17 PeaZip 5,087
18 compress 5,021
19 cute_headers 4,618
20 caesium-image-compressor 4,134
21 SharpZipLib 3,825
22 zip.js 3,607
23 StegCloak 3,506

Sponsored
Auth in Less Than 5 Minutes
Civic Auth comes with multiple SSO options, optional embedded wallets, and user management โ€” all implemented with just a few lines of code. Start building today.
www.civic.com

Did you know that JavaScript is
the 3rd most popular programming language
based on number of references?