Top 23 Python quantization Projects

quantization

• Add a project

1. LlamaFactory

   1 8 71,870 9.6 Python

   Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

   

   Project mention: Llama-Factory: Unified, Efficient Fine-Tuning for 100 Open LLMs | news.ycombinator.com | 2025-09-18

2. SaaSHub

   www.saashub.com featured

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

3. faster-whisper

   2 28 23,393 6.8 Python

   Faster Whisper transcription with CTranslate2

   

   Project mention: I built a free, local video transcription tool, because I didn't want to pay $10/hour or upload my files to a stranger's server | dev.to | 2026-05-09

   Transcribes it locally using faster-whisper

4. Chinese-LLaMA-Alpaca

   3 4 18,949 2.3 Python

   中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

   

5. bitsandbytes

   4 63 8,258 9.3 Python

   Accessible large language models via k-bit quantization for PyTorch.

   

6. nunchaku

   5 2 3,861 9.4 Python

   [ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

   

7. optimum

   6 8 3,409 8.1 Python

   🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

   

8. llm-compressor

   7 3 3,331 9.8 Python

   Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

   

9. Pretrained-Language-Model

   8 1 3,158 6.1 Python

   Pretrained language model and its related optimization techniques developed by Huawei Noah's Ark Lab.

   

10. ao

    9 4 2,843 9.9 Python

    PyTorch native quantization and sparsity for training and inference

    

    Project mention: Gemma 3 270M re-implemented in pure PyTorch for local tinkering | news.ycombinator.com | 2025-08-20

11. xTuring

    10 31 2,667 7.3 Python

    Build, personalize and control your own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6

    

12. neural-compressor

    11 3 2,651 9.4 Python

    SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

    

13. aimet

    12 2 2,634 9.9 Python

    AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

    

14. mixtral-offloading

    13 3 2,329 8.6 Python

    Run Mixtral-8x7B models in Colab or consumer desktops

    

15. mmrazor

    14 4 1,672 0.0 Python

    OpenMMLab Model Compression Toolbox and Benchmark.

    

16. model-optimization

    15 2 1,573 6.7 Python

    A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

    

    Project mention: 95% Accurate Wake Word Detection: Low-Power CNN + MFCC Guide | dev.to | 2025-10-19

    TensorFlow Model Optimization Toolkit

17. auto-round

    16 3 1,436 9.8 Python

    A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

    

    Project mention: Advanced Quantization Algorithm for LLMs | news.ycombinator.com | 2026-05-01

    hmm... at Q4_K_M, stock-style quantization is retaining ~99–99.8% of BF16 accuracy, AutoRound pushes that to ~99.4–100.n% (??) the gap is roughly 0.1–0.7 percentage points

    https://github.com/intel/auto-round/blob/main/docs/gguf_alg_...

18. nncf

    17 2 1,169 9.7 Python

    Neural Network Compression Framework for enhanced OpenVINO™ inference

    

19. z80ai

    18 1 1,092 6.3 Python

    Z80-μLM is a 2-bit quantized language model small enough to run on an 8-bit Z80 processor. Train conversational models in Python, export them as CP/M .COM binaries, and chat with your vintage computer.

    

    Project mention: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB | news.ycombinator.com | 2025-12-28

20. optimum-quanto

    19 1 1,042 7.3 Python

    A pytorch quantization backend for optimum

    

21. finn

    20 4 1,003 2.7 Python

    Dataflow compiler for QNN inference on FPGAs

    

22. hqq

    21 7 940 7.4 Python

    Official implementation of Half-Quadratic Quantization (HQQ)

    

23. OmniQuant

    22 4 899 2.7 Python

    [ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

    

24. SqueezeLLM

    23 5 718 5.1 Python

    [ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization

    

Python quantization related posts

• I built a free, local video transcription tool, because I didn't want to pay $10/hour or upload my files to a stranger's server

  2 projects | dev.to | 9 May 2026

• Advanced Quantization Algorithm for LLMs

  2 projects | news.ycombinator.com | 1 May 2026

• LLM compressor: compress models for efficient deployment

  1 project | news.ycombinator.com | 20 Aug 2024

• Creando Subtítulos Automáticos para Vídeos con Python, Faster-Whisper, FFmpeg, Streamlit, Pillow

  7 projects | dev.to | 29 Apr 2024

• Apple Explores Home Robotics as Potential 'Next Big Thing'

  3 projects | news.ycombinator.com | 4 Apr 2024

• Half-Quadratic Quantization of Large Machine Learning Models

  1 project | news.ycombinator.com | 14 Mar 2024

• New Mixtral HQQ Quantzied 4-bit/2-bit configuration

  1 project | news.ycombinator.com | 18 Dec 2023
• A note from our sponsor - SaaSHub
  www.saashub.com | 9 Jun 2026

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Python projects with our weekly report!