Python Compression

Open-source Python projects categorized as Compression

Top 23 Python Compression Projects

  • DeepSpeed

    DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

    Project mention: DeepSpeed-FastGen: High-Throughput for LLMs via MII and DeepSpeed-Inference | | 2023-11-04
  • PaddleNLP

    👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.

    Project mention: Chatgpt 到底是不是开源的? | /r/China_irl | 2023-03-25
  • Onboard AI

    Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at

  • BorgBackup

    Deduplicating archiver with compression and authenticated encryption.

    Project mention: Borg CVE fix requires migration | | 2023-10-10
  • Crunch

    Insane(ly slow but wicked good) PNG image optimization (by chrissimpkins)

    Project mention: (Newbie) How can I replicate this UI? Would SpriteKit be the best option? | /r/iOSProgramming | 2023-03-08

    If you go the png/jpg route, don’t forget to crunch your assets to save disk space.

  • unblob

    Extract files from any kind of container formats

    Project mention: How are zlib, gzip and zip related? | | 2023-11-27

    If you are interested in implementation details, how to unpack/decompress them, check out these Python implementations:




  • aimet

    AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

  • borgmatic

    Simple, configuration-driven backup software for servers and workstations

    Project mention: Kopia: Open-Source, Fast and Secure Open-Source Backup Software | | 2023-09-15

    Not really dumb. I do use them too but with Borgbackup on the top (since they support it natively).

    I found Borgmatic ( ) to be the best way to run my backups. It takes care of everything from pruning to verifying the checksum etc... and it integrates with some monitoring (like cronitor).

    So Borgmatic + is the best combo

  • InfluxDB

    Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  • model-optimization

    A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

  • compression

    Data compression in TensorFlow (by tensorflow)

  • nncf

    Neural Network Compression Framework for enhanced OpenVINO™ inference

    Project mention: FLaNK Stack Weekly 06 Nov 2023 | | 2023-11-06
  • zipfly

    Python Zip Stream

    Project mention: ZipFly 6.0.5 | | 2023-01-24
  • refinery

    High Octane Triage Analysis (by binref)

    Project mention: refinery: High Octane Triage Analysis - The Binary Refinery™ is a collection of Python scripts that implement transformations of binary data such as compression and encryption | /r/blueteamsec | 2023-03-02
  • ratarmount

    Access large archives as a filesystem efficiently, e.g., TAR, RAR, ZIP, GZ, BZ2, XZ, ZSTD archives

    Project mention: Show HN: Rapidgzip – Parallel Gzip Decompressing with 10 GB/S | | 2023-09-04
  • swin2sr

    Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration. Advances in Image Manipulation (AIM) workshop ECCV 2022. Try it out! over 2.5 M runs

    Project mention: Braided Beauty in a Pretty City | /r/WaifuDiffusion | 2023-01-22

    Upscaling model can be found here, and instructions for use in Automatic1111 WebUI can be found here.

  • pythonlibs

    A Python wrapper for the extremely fast Blosc compression library

  • 3d-model-convert-to-gltf

    Convert 3d model (STL/IGES/STEP/OBJ/FBX) to gltf and compression

  • DictDataBase

    A python NoSQL dictionary database, with concurrent access and ACID compliance

    Project mention: LiliDB (JSON-based database) | /r/Python | 2023-04-15

    Nice! Quite similar to a database I am working on:

  • archive-pdf-tools

    Fast PDF generation and compression. Deals with millions of pages daily.

    Project mention: How to Entertain with Your Pocket Calculator (1975) | | 2023-05-14

    (author of the PDF generation sw here, thanks to jrochkind1 for pointing me in this direction) Would you mind filing a bug here with your findings, so that we can figure out what is going on?

    We ( tested the software extensively with the various readers, and we haven't found problems with the PDFs on Apple Preview.

  • nncodec

    Fraunhofer Neural Network Encoder/Decoder (NNCodec)

    Project mention: Fraunhofer Neural Network Encoder/Decoder (NNCodec) | | 2022-12-04
  • reCBZ

    Utility for converting and optimizing comics for e-readers & mobile devices

    Project mention: Looking for Epub files of full manga series/volumes | /r/mangapiracy | 2023-06-08
  • compressio

    Lossless in-memory compression of pandas DataFrames and Series powered by the visions type system. Up to 10x less RAM needed for the same data.

  • fidelity_controllable_compression

    Official PyTorch Implementation of Fidelity Controllable Extreme Image Compression with GAN

  • comepress

    📥 Comepress - Convert and Optimize your Web Project's Image Files to Next-Gen WebP format in just one click!

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-11-27.

Python Compression related posts


What are some of the best open-source Compression projects in Python? This list will help you:

Project Stars
1 DeepSpeed 29,742
2 PaddleNLP 10,568
3 BorgBackup 9,982
4 Crunch 3,282
5 unblob 1,903
6 aimet 1,730
7 borgmatic 1,528
8 model-optimization 1,440
9 compression 788
10 nncf 703
11 zipfly 514
12 refinery 504
13 ratarmount 485
14 swin2sr 432
15 pythonlibs 335
16 3d-model-convert-to-gltf 221
17 DictDataBase 199
18 archive-pdf-tools 71
19 nncodec 53
20 reCBZ 50
21 compressio 27
22 fidelity_controllable_compression 24
23 comepress 23
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives