Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge. Learn more →
Top 23 Python Compression Projects
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.Project mention: DeepSpeed-FastGen: High-Throughput for LLMs via MII and DeepSpeed-Inference | news.ycombinator.com | 2023-11-04
👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.Project mention: Chatgpt 到底是不是开源的？ | /r/China_irl | 2023-03-25
Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at www.getonboard.dev.
Deduplicating archiver with compression and authenticated encryption.Project mention: Borg CVE fix requires migration | news.ycombinator.com | 2023-10-10
Insane(ly slow but wicked good) PNG image optimization (by chrissimpkins)Project mention: (Newbie) How can I replicate this UI? Would SpriteKit be the best option? | /r/iOSProgramming | 2023-03-08
If you go the png/jpg route, don’t forget to crunch your assets to save disk space.
Extract files from any kind of container formatsProject mention: How are zlib, gzip and zip related? | news.ycombinator.com | 2023-11-27
If you are interested in implementation details, how to unpack/decompress them, check out these Python implementations:
AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.
Simple, configuration-driven backup software for servers and workstationsProject mention: Kopia: Open-Source, Fast and Secure Open-Source Backup Software | news.ycombinator.com | 2023-09-15
Not really dumb. I do use them too but with Borgbackup on the top (since they support it natively).
I found Borgmatic ( https://torsion.org/borgmatic/ ) to be the best way to run my backups. It takes care of everything from pruning to verifying the checksum etc... and it integrates with some monitoring (like cronitor).
So Borgmatic + rsync.net is the best combo
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.
Data compression in TensorFlow (by tensorflow)
Neural Network Compression Framework for enhanced OpenVINO™ inferenceProject mention: FLaNK Stack Weekly 06 Nov 2023 | dev.to | 2023-11-06
Python Zip StreamProject mention: ZipFly 6.0.5 | news.ycombinator.com | 2023-01-24
High Octane Triage Analysis (by binref)Project mention: refinery: High Octane Triage Analysis - The Binary Refinery™ is a collection of Python scripts that implement transformations of binary data such as compression and encryption | /r/blueteamsec | 2023-03-02
Access large archives as a filesystem efficiently, e.g., TAR, RAR, ZIP, GZ, BZ2, XZ, ZSTD archivesProject mention: Show HN: Rapidgzip – Parallel Gzip Decompressing with 10 GB/S | news.ycombinator.com | 2023-09-04
Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration. Advances in Image Manipulation (AIM) workshop ECCV 2022. Try it out! over 2.5 M runs https://replicate.com/mv-lab/swin2srProject mention: Braided Beauty in a Pretty City | /r/WaifuDiffusion | 2023-01-22
Upscaling model can be found here, and instructions for use in Automatic1111 WebUI can be found here.
A Python wrapper for the extremely fast Blosc compression library
Convert 3d model (STL/IGES/STEP/OBJ/FBX) to gltf and compression
A python NoSQL dictionary database, with concurrent access and ACID complianceProject mention: LiliDB (JSON-based database) | /r/Python | 2023-04-15
Nice! Quite similar to a database I am working on: https://github.com/mkrd/DictDataBase
Fast PDF generation and compression. Deals with millions of pages daily.Project mention: How to Entertain with Your Pocket Calculator (1975) | news.ycombinator.com | 2023-05-14
(author of the PDF generation sw here, thanks to jrochkind1 for pointing me in this direction) Would you mind filing a bug here with your findings, so that we can figure out what is going on? https://github.com/internetarchive/archive-pdf-tools
We (archive.org) tested the software extensively with the various readers, and we haven't found problems with the PDFs on Apple Preview.
Fraunhofer Neural Network Encoder/Decoder (NNCodec)Project mention: Fraunhofer Neural Network Encoder/Decoder (NNCodec) | news.ycombinator.com | 2022-12-04
Utility for converting and optimizing comics for e-readers & mobile devicesProject mention: Looking for Epub files of full manga series/volumes | /r/mangapiracy | 2023-06-08
Lossless in-memory compression of pandas DataFrames and Series powered by the visions type system. Up to 10x less RAM needed for the same data.
Official PyTorch Implementation of Fidelity Controllable Extreme Image Compression with GAN
📥 Comepress - Convert and Optimize your Web Project's Image Files to Next-Gen WebP format in just one click!
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Python Compression related posts
Borg CVE fix requires migration
1 project | news.ycombinator.com | 10 Oct 2023
disc space is not freeing
1 project | /r/openSUSE | 25 Jun 2023
I installed Arch today!
1 project | /r/linux4noobs | 12 Jun 2023
Ask HN: What is the most cost effective way to do backups?
1 project | news.ycombinator.com | 8 Jun 2023
How to best configure an HDD for cold storage?
1 project | /r/DataHoarder | 7 Jun 2023
How to set up offsite backup for Unraid server?
1 project | /r/unRAID | 3 Jun 2023
What is a good "just works" backup service for linux?
1 project | /r/linux4noobs | 31 May 2023
A note from our sponsor - InfluxDB
www.influxdata.com | 30 Nov 2023
What are some of the best open-source Compression projects in Python? This list will help you: