Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge. Learn more →
Top 23 Python Compression Projects
-
DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Project mention: DeepSpeed-FastGen: High-Throughput for LLMs via MII and DeepSpeed-Inference | news.ycombinator.com | 2023-11-04 -
PaddleNLP
👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
-
Onboard AI
Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at www.getonboard.dev.
-
-
Project mention: (Newbie) How can I replicate this UI? Would SpriteKit be the best option? | /r/iOSProgramming | 2023-03-08
If you go the png/jpg route, don’t forget to crunch your assets to save disk space.
-
If you are interested in implementation details, how to unpack/decompress them, check out these Python implementations:
- https://github.com/onekey-sec/unblob/blob/main/unblob/handle...
- https://github.com/onekey-sec/unblob/blob/main/unblob/handle...
- https://github.com/onekey-sec/unblob/blob/main/unblob/handle...
-
aimet
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
-
Project mention: Kopia: Open-Source, Fast and Secure Open-Source Backup Software | news.ycombinator.com | 2023-09-15
Not really dumb. I do use them too but with Borgbackup on the top (since they support it natively).
I found Borgmatic ( https://torsion.org/borgmatic/ ) to be the best way to run my backups. It takes care of everything from pruning to verifying the checksum etc... and it integrates with some monitoring (like cronitor).
So Borgmatic + rsync.net is the best combo
-
InfluxDB
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
-
model-optimization
A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.
-
-
-
-
Project mention: refinery: High Octane Triage Analysis - The Binary Refinery™ is a collection of Python scripts that implement transformations of binary data such as compression and encryption | /r/blueteamsec | 2023-03-02
-
ratarmount
Access large archives as a filesystem efficiently, e.g., TAR, RAR, ZIP, GZ, BZ2, XZ, ZSTD archives
Project mention: Show HN: Rapidgzip – Parallel Gzip Decompressing with 10 GB/S | news.ycombinator.com | 2023-09-04 -
swin2sr
Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration. Advances in Image Manipulation (AIM) workshop ECCV 2022. Try it out! over 2.5 M runs https://replicate.com/mv-lab/swin2sr
Upscaling model can be found here, and instructions for use in Automatic1111 WebUI can be found here.
-
-
-
Nice! Quite similar to a database I am working on: https://github.com/mkrd/DictDataBase
-
Project mention: How to Entertain with Your Pocket Calculator (1975) | news.ycombinator.com | 2023-05-14
(author of the PDF generation sw here, thanks to jrochkind1 for pointing me in this direction) Would you mind filing a bug here with your findings, so that we can figure out what is going on? https://github.com/internetarchive/archive-pdf-tools
We (archive.org) tested the software extensively with the various readers, and we haven't found problems with the PDFs on Apple Preview.
-
Project mention: Fraunhofer Neural Network Encoder/Decoder (NNCodec) | news.ycombinator.com | 2022-12-04
-
-
compressio
Lossless in-memory compression of pandas DataFrames and Series powered by the visions type system. Up to 10x less RAM needed for the same data.
-
fidelity_controllable_compression
Official PyTorch Implementation of Fidelity Controllable Extreme Image Compression with GAN
-
comepress
📥 Comepress - Convert and Optimize your Web Project's Image Files to Next-Gen WebP format in just one click!
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
The latest post mention was on 2023-11-27.
Python Compression related posts
- Borg CVE fix requires migration
- disc space is not freeing
- I installed Arch today!
- Ask HN: What is the most cost effective way to do backups?
- How to best configure an HDD for cold storage?
- How to set up offsite backup for Unraid server?
- What is a good "just works" backup service for linux?
-
A note from our sponsor - InfluxDB
www.influxdata.com | 30 Nov 2023
Index
What are some of the best open-source Compression projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | DeepSpeed | 29,742 |
| 2 | PaddleNLP | 10,568 |
| 3 | BorgBackup | 9,982 |
| 4 | Crunch | 3,282 |
| 5 | unblob | 1,903 |
| 6 | aimet | 1,730 |
| 7 | borgmatic | 1,528 |
| 8 | model-optimization | 1,440 |
| 9 | compression | 788 |
| 10 | nncf | 703 |
| 11 | zipfly | 514 |
| 12 | refinery | 504 |
| 13 | ratarmount | 485 |
| 14 | swin2sr | 432 |
| 15 | pythonlibs | 335 |
| 16 | 3d-model-convert-to-gltf | 221 |
| 17 | DictDataBase | 199 |
| 18 | archive-pdf-tools | 71 |
| 19 | nncodec | 53 |
| 20 | reCBZ | 50 |
| 21 | compressio | 27 |
| 22 | fidelity_controllable_compression | 24 |
| 23 | comepress | 23 |
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com