Python Compression

Open-source Python projects categorized as Compression

Top 23 Python Compression Projects

  • BorgBackup

    Deduplicating archiver with compression and authenticated encryption.

    Project mention: Beware that BorgBackup 2.0 will soon come, breaking compatibility with current repositories and scripts. | reddit.com/r/linux | 2022-12-04

    Seriously, I'm having trouble seeing what the, presumably execution time, ram requirements and/or storage requirements for such a transfer are. Going through the documents: https://github.com/borgbackup/borg/blob/master/docs/usage/transfer.rst , I see pretty much the same description there. I assume that, at the very least, this requires crypto time and the storage space required to duplicate a backup, but is there more or less than I'm seeing? This doesn't seem to be an in-place conversion? Am I reading that correctly?

  • model-optimization

    A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

  • Scout APM

    Truly a developer’s best friend. Scout APM is great for developers who want to find and fix performance issues in their applications. With Scout, we'll take care of the bugs so you can focus on building great things 🚀.

  • borgmatic

    Simple, configuration-driven backup software for servers and workstations

    Project mention: Beware that BorgBackup 2.0 will soon come, breaking compatibility with current repositories and scripts. | reddit.com/r/linux | 2022-12-04

    Over at Vorta (feature branch) we mostly adjusted to the latest betas. So I expect to have support for both versions, once there is an RC. Borgmatic added support in Oct.

  • aimet

    AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

    Project mention: I was looking for some great quantization open-source libraries that could actually be applied in production (both edge or cloud CPU/GPU). Do you know if I am missing any good libraries? | reddit.com/r/learnmachinelearning | 2022-07-14

    Qualcomm AIMET | Advanced quantization and compression techniques for trained neural network models

  • unblob

    Extract files from any kind of container formats

    Project mention: Show HN: Unblob – accurate, fast, and easy-to-use extraction suite | news.ycombinator.com | 2022-11-09
  • compression

    Data compression in TensorFlow (by tensorflow)

    Project mention: [P] I turned Stable Diffusion into a lossy image compression codec and it performs great! | reddit.com/r/MachineLearning | 2022-09-19

    Eg : https://github.com/tensorflow/compression

  • nncf

    Neural Network Compression Framework for enhanced OpenVINO™ inference

    Project mention: NN mixed-precision quantization framework that supports TF? | reddit.com/r/tensorflow | 2022-11-15

    I am aware of NNCF (https://github.com/openvinotoolkit/nncf), but it doesn't support mixed precision quantization for TF. What other frameworks support that for TF? (implement HAWQ or AutoQ algorithms for example)

  • InfluxDB

    Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Data Platform where developers build real-time applications for analytics, IoT and cloud-native services in less time with less code.

  • zipfly

    Python Zip Stream

    Project mention: GitHub - sandes/zipfly: Writing large ZIP archives without memory inflation | reddit.com/r/Python | 2022-06-01
  • refinery

    High Octane Triage Analysis (by binref)

  • ratarmount

    Random Access Tar Mount

    Project mention: Looking for advice uploading data while at uni. I need to split the data i need to upload to carry it with me | reddit.com/r/DataHoarder | 2022-10-11

    As an added complication this would need to work under windows (i need onenote and that's win only :/ ) ; this alone makes the majority of solutions that i came up with impossible. One way could've been splitting the data onto various tar files and then mounting those with rartarmount but...linux only :( .

  • 3d-model-convert-to-gltf

    Convert 3d model (STL/IGES/STEP/OBJ/FBX) to gltf and compression

  • DictDataBase

    A python NoSQL database that uses dicts, and provides thread and process safety

    Project mention: Sunday Daily Thread: What's everyone working on this week? | reddit.com/r/Python | 2022-11-20

    So in the end, this is an attempt to but a truly document oriented equivalent to SQLite, and I think there is potential. I would love to do the same benchmark with the SQLite solution you proposed and compare the two, but my time is limiting me right now… Here is the github link if you are interested: https://github.com/mkrd/DictDataBase

  • swin2sr

    Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration at the Advances in Image Manipulation (AIM) workshop ECCV 2022, Tel Aviv

    Project mention: The DEFINITIVE Comparison to Upscalers | reddit.com/r/StableDiffusion | 2022-10-12
  • archive-pdf-tools

    Fast PDF generation and compression. Deals with millions of pages daily.

  • compressio

    Lossless in-memory compression of pandas DataFrames and Series powered by the visions type system. Up to 10x less RAM needed for the same data.

    Project mention: Visions – User defined data type systems | reddit.com/r/Python | 2022-02-04

    Visions is a python library for working with user defined data type systems. Out of the box, it provides type inference and automated data cleaning of sequence data with backend specific implementations for pandas, spark, python, and numpy. We often use it as a first pass cleaning step when working with tabular data and to simplify the backend logic of both pandas-profiling and our tabular data compression library compressio.

  • fidelity_controllable_compression

    Official PyTorch Implementation of Fidelity Controllable Extreme Image Compression with GAN

    Project mention: Convert | reddit.com/r/SMBCComics | 2022-03-28

    Ok. I tried compressing an image of waldo as far as I could (using this neural network) and searching for it in PI. Couldn't find it in the first 1 billion digits in hexadecimal (I actually couldn't find much much smaller sequences as well. 1 billion is simply not enough digits). I think I'm done for the night, if someone has an idea how to get a few trillions of digits of PI in binary/hexadecimal - I'm listening. For what it's worth, here's the smallest representation of waldo's face in binary I could create (at 728 bytes, I'm sure a smaller image can be created with some effort) -

  • drakify

    📻 The "slowed and reverbed" version of your favorite song is a click away.

    Project mention: Slowed and Reverbed with Python (+more effects) | reddit.com/r/audioengineering | 2022-04-28
  • reCBZ

    Utility for converting and optimizing manga & comics for e-readers & mobile devices

    Project mention: New tool to create Kobo EPUBs (KEPUB), specifically designed for manga. | reddit.com/r/kobo | 2022-12-03
  • Compresssio-GUI

    🖼 Streamline/Optimize your Images to save storage space using Tinify API...

  • nncodec

    Fraunhofer Neural Network Encoder/Decoder (NNCodec)

    Project mention: Fraunhofer Neural Network Encoder/Decoder (NNCodec) | news.ycombinator.com | 2022-12-04
  • xtarfile

    Wrapper around tarfile to add support for more compression formats

    Project mention: conda-forge VS xtarfile - a user suggested alternative | libhunt.com/r/conda-forge | 2022-10-23
  • Pixxia

    🖼 Pixxia uses lossy compression methods to reduce the document size of your JPG/PNG files.

  • lossytextcompressor

    Lossy text compressor

    Project mention: Silly Lossy Text Compression Idea | news.ycombinator.com | 2022-05-21

    Cool :) I made a little script that uses a thesaurus to try to shorten text https://github.com/anfractuosity/lossytextcompressor

  • Sonar

    Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-12-04.

Python Compression related posts

Index

What are some of the best open-source Compression projects in Python? This list will help you:

Project Stars
1 BorgBackup 8,838
2 model-optimization 1,337
3 borgmatic 1,266
4 aimet 1,253
5 unblob 821
6 compression 689
7 nncf 533
8 zipfly 504
9 refinery 332
10 ratarmount 309
11 3d-model-convert-to-gltf 162
12 DictDataBase 157
13 swin2sr 135
14 archive-pdf-tools 42
15 compressio 22
16 fidelity_controllable_compression 20
17 drakify 18
18 reCBZ 17
19 Compresssio-GUI 9
20 nncodec 8
21 xtarfile 7
22 Pixxia 6
23 lossytextcompressor 5
Workflow assistant built for devs & their teams
Automate the mundane part of your day, with live actionable messages for your GitHub & Jira tasks.
www.zigi.ai