LLaMA-8bit-LoRA
Repository for Chat LLaMA - training a LoRA for the LLaMA (1 or 2) models on HuggingFace with 8-bit or 4-bit quantization. Research only. (by serp-ai)
text-generation-webui-testing
A fork of textgen that still supports V1 GPTQ, 4-bit lora and other GPTQ models besides llama. (by Ph0rk0z)
| LLaMA-8bit-LoRA | text-generation-webui-testing | |
|---|---|---|
| 3 | 9 | |
| 145 | 22 | |
| 0.7% | - | |
| 5.1 | 9.9 | |
| 8 months ago | 3 days ago | |
| Python | Python | |
| - | GNU Affero General Public License v3.0 |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
LLaMA-8bit-LoRA
Posts with mentions or reviews of LLaMA-8bit-LoRA.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-04-06.
-
Any news on training LoRAs in 4-bit mode?
https://github.com/serp-ai/LLaMA-8bit-LoRA/blob/main/docs/merging_the_weights.md < merge models
- [R] 🤖🌟 Unlock the Power of Personal AI: Introducing ChatLLaMA, Your Custom Personal Assistant! 🚀💬
text-generation-webui-testing
Posts with mentions or reviews of text-generation-webui-testing.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-06-28.
-
Slow inference on R720 w/P40 (or not)?
Also autograd from here: https://github.com/Ph0rk0z/text-generation-webui-testing/ and it's matching GPTQ: https://github.com/Ph0rk0z/GPTQ-Merged/tree/dual-model
-
Call me a fool, but I thought 24 GB of ram would get me 2048 context with 13B GPTQ
Some of the novel-ai ones work but the first one I tried to make it not spazz out was: https://github.com/Ph0rk0z/text-generation-webui-testing/blob/DualModel/presets/MOSS.yaml
-
Finetuning on multiple GPUs
i've never tried that particular one. everything else I threw at it trained through : https://github.com/Ph0rk0z/text-generation-webui-testing/ successfully.
-
Best current tutorial for training your own LoRA? Also I've got a 24GB 3090, so which models would you recommend fine tuning on?
as integrated in https://github.com/Ph0rk0z/text-generation-webui-testing/
-
Monkeypatch Issues
if you like the "monkeypatch" https://github.com/Ph0rk0z/text-generation-webui-testing/ is better. I think in the discussion someone got it running on windows.
- My Lora training locally experiments
-
Any news on training LoRAs in 4-bit mode?
https://github.com/Ph0rk0z/text-generation-webui-testing < 4bit lora use from the UI on old GPTQ
-
Keep your GPUs cool
well im running it with oobabooga/text-generation-webui and 8 bit now works after i did this fix Add 8bit threshold for my Pascal card. I use 1.5 or 1.0, otherwise NaN · Ph0rk0z/text-generation-webui-testing@ecad08f (github.com)
- 4bit LoRA Guide for Oobabooga!
What are some alternatives?
When comparing LLaMA-8bit-LoRA and text-generation-webui-testing you can also consider the following projects:
alpaca-lora - Instruct-tune LLaMA on consumer hardware
bitsandbytes - Accessible large language models via k-bit quantization for PyTorch.
sparsegpt-for-LLaMA - Code for the paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot" with LLaMA implementation.
alpaca_lora_4bit
trl - Train transformer language models with reinforcement learning.
alpaca_lora_4bit_readme - Just a simple HowTo for https://github.com/johnsmith0031/alpaca_lora_4bit
Sparsebit - A model compression and acceleration toolbox based on pytorch.
private-gpt - Interact with your documents using the power of GPT, 100% privately, no data leaks
axolotl - Go ahead and axolotl questions
GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
LLaMA-8bit-LoRA vs alpaca-lora
text-generation-webui-testing vs bitsandbytes
LLaMA-8bit-LoRA vs sparsegpt-for-LLaMA
text-generation-webui-testing vs alpaca_lora_4bit
LLaMA-8bit-LoRA vs trl
text-generation-webui-testing vs alpaca_lora_4bit_readme
LLaMA-8bit-LoRA vs Sparsebit
text-generation-webui-testing vs private-gpt
LLaMA-8bit-LoRA vs alpaca_lora_4bit
text-generation-webui-testing vs axolotl
text-generation-webui-testing vs GPTQ-for-LLaMa
text-generation-webui-testing vs text-generation-webui