The Era of 1-bit LLMs: ternary parameters for cost-effective computing

• unilm

  44 18,915 9.0 Python

  Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

  

+1 On this, the real proof would have been testing both models side-by-side.

It seems that it may be published on GitHub [1] according to HuggingFace [2].

[1] https://github.com/microsoft/unilm/tree/master/bitnet

[2] https://huggingface.co/papers/2402.17764

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• llama.cpp

  792 59,810 10.0 C++

  LLM inference in C/C++

It does result in a significant degradation relative to unquantized model of the same size, but even with simple llama.cpp K-quantization, it's still worth it all the way down to 2-bit. The chart in this llama.cpp PR speaks for itself:

https://github.com/ggerganov/llama.cpp/pull/1684#issue-17396...

• quantized-nets

  1 23 - Python

  Contains code for Binary, Ternary, N-bit Quantized and Hybrid CNNs for low precision experiments.

People have been doing this 6 years ago.

    https://github.com/yashkant/quantized-nets

• trained-ternary-quantization

  1 109 - Jupyter Notebook

  Reducing the size of convolutional neural networks

https://github.com/TropComplique/trained-ternary-quantization

• Ternary-Weights-Network

  1 35 - Python

  Ternay-Weights-Network/Pytorch

https://github.com/buaabai/Ternary-Weights-Network

• StableLM

  43 15,859 5.0 Jupyter Notebook

  StableLM: Stability AI Language Models

  

https://github.com/Stability-AI/StableLM?tab=readme-ov-file#...

Suggest a related project