Any news on training LoRAs in 4-bit mode?

• text-generation-webui

  876 36,293 9.9 Python

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

Posted https://github.com/oobabooga/text-generation-webui/issues/813 earlier today. Still hasn't been merged with main repo but got the following fork suggestions:

• LLaMA-8bit-LoRA

  3 145 5.1 Python

  Repository for Chat LLaMA - training a LoRA for the LLaMA (1 or 2) models on HuggingFace with 8-bit or 4-bit quantization. Research only.

  

https://github.com/serp-ai/LLaMA-8bit-LoRA/blob/main/docs/merging_the_weights.md < merge models

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• alpaca_lora_4bit

  41 529 8.6 Python

https://github.com/johnsmith0031/alpaca_lora_4bit < train loras

• text-generation-webui-testing

  9 22 9.9 Python

  A fork of textgen that still supports V1 GPTQ, 4-bit lora and other GPTQ models besides llama.

https://github.com/Ph0rk0z/text-generation-webui-testing < 4bit lora use from the UI on old GPTQ

Suggest a related project