The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning. Learn more →
Top 23 Python Compression Projects
-
DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
-
PaddleNLP
👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
aimet
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
model-optimization
A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.
-
-
-
ratarmount
Access large archives as a filesystem efficiently, e.g., TAR, RAR, ZIP, GZ, BZ2, XZ, ZSTD archives
-
-
swin2sr
Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration. Advances in Image Manipulation (AIM) workshop ECCV 2022. Try it out! over 3.3M runs https://replicate.com/mv-lab/swin2sr
-
-
-
SecretPixel
SecretPixel is a cutting-edge steganography tool designed to securely conceal sensitive information within images. It stands out in the realm of digital steganography by combining advanced encryption, compression, and a seeded Least Significant Bit (LSB) technique to provide a robust solution for embedding data undetectably.
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Can we discuss MLOps, Deployment, Optimizations, and Speed? | /r/LocalLLaMA | 2023-12-06DeepSpeed can handle parallelism concerns, and even offload data/model to RAM, or even NVMe (!?) . I'm surprised I don't see this project used more.
Project mention: Ask HN: Open-source Windows 11 backup solutions | news.ycombinator.com | 2024-04-04i use - and recommend - "borgbackup": for example with the "vorta" graphical frontend
* https://www.borgbackup.org/
* https://vorta.borgbase.com/install/windows/
just my 0.02€
- for important files, a separate box where I have borgmatic [1] in deduplication mode installed; this is updated once in a while
Just curious: Do you have any reason to believe that such a data corruption bug is likely in ZFS? It seems like saying that ext4 could have a bug and you should also store stuff on NTFS, just in case (which I think does not make sense..).
[1]: https://github.com/borgmatic-collective/borgmatic
Project mention: A Curated List for Efficient Large Language Models | news.ycombinator.com | 2023-07-15
Project mention: Ratarmount: Access large archives as a filesystem efficiently | news.ycombinator.com | 2024-04-10
Project mention: Show HN: SecretPixel – Advanced Image Steganography Tool | news.ycombinator.com | 2024-02-03I've tested Binwalk on all the example files, and the BMP and TGA samples didn't show any zlib compressed data (https://i.ibb.co/R4WWsV2/2024-02-04-16h22-31.png). Can you please confirm that you have used the files from the example folder (https://github.com/x011/SecretPixel/tree/main/examples)? I will try to work on the PNG and TIFF cases soon. Thanks for the paper :)
Project mention: Show HN: Georaptor – A Python Geohash Compression Tool v3.0.0 | news.ycombinator.com | 2024-04-03
Project mention: How to Entertain with Your Pocket Calculator (1975) | news.ycombinator.com | 2023-05-14(author of the PDF generation sw here, thanks to jrochkind1 for pointing me in this direction) Would you mind filing a bug here with your findings, so that we can figure out what is going on? https://github.com/internetarchive/archive-pdf-tools
We (archive.org) tested the software extensively with the various readers, and we haven't found problems with the PDFs on Apple Preview.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python Compression related posts
- Ratarmount: Access large archives as a filesystem efficiently
- TSAC: Low Bitrate Audio Compression
- Show HN: SecretPixel – Advanced Image Steganography Tool
- What do you use for VPS backup? Would improved borg setup - pull mode - be enough? Or, do you use something else?
- Borg CVE fix requires migration
- disc space is not freeing
- I installed Arch today!
-
A note from our sponsor - WorkOS
workos.com | 26 Apr 2024
Index
What are some of the best open-source Compression projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | DeepSpeed | 32,550 |
| 2 | PaddleNLP | 11,423 |
| 3 | BorgBackup | 10,526 |
| 4 | Crunch | 3,325 |
| 5 | unblob | 2,046 |
| 6 | aimet | 1,900 |
| 7 | borgmatic | 1,639 |
| 8 | model-optimization | 1,465 |
| 9 | compression | 823 |
| 10 | nncf | 777 |
| 11 | Awesome-Efficient-LLM | 775 |
| 12 | ratarmount | 628 |
| 13 | refinery | 588 |
| 14 | swin2sr | 526 |
| 15 | zipfly | 518 |
| 16 | pythonlibs | 341 |
| 17 | 3d-model-convert-to-gltf | 231 |
| 18 | DictDataBase | 220 |
| 19 | SecretPixel | 187 |
| 20 | georaptor | 187 |
| 21 | KVQuant | 183 |
| 22 | archive-pdf-tools | 80 |
| 23 | reCBZ | 60 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com