LLaMA-8bit-LoRA
text-generation-webui-testing
LLaMA-8bit-LoRA | text-generation-webui-testing | |
---|---|---|
3 | 9 | |
145 | 22 | |
0.7% | - | |
5.1 | 9.9 | |
8 months ago | 3 days ago | |
Python | Python | |
- | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
LLaMA-8bit-LoRA
-
Any news on training LoRAs in 4-bit mode?
https://github.com/serp-ai/LLaMA-8bit-LoRA/blob/main/docs/merging_the_weights.md < merge models
- [R] ๐ค๐ Unlock the Power of Personal AI: Introducing ChatLLaMA, Your Custom Personal Assistant! ๐๐ฌ
text-generation-webui-testing
-
Slow inference on R720 w/P40 (or not)?
Also autograd from here: https://github.com/Ph0rk0z/text-generation-webui-testing/ and it's matching GPTQ: https://github.com/Ph0rk0z/GPTQ-Merged/tree/dual-model
-
Call me a fool, but I thought 24 GB of ram would get me 2048 context with 13B GPTQ
Some of the novel-ai ones work but the first one I tried to make it not spazz out was: https://github.com/Ph0rk0z/text-generation-webui-testing/blob/DualModel/presets/MOSS.yaml
-
Finetuning on multiple GPUs
i've never tried that particular one. everything else I threw at it trained through : https://github.com/Ph0rk0z/text-generation-webui-testing/ successfully.
-
Best current tutorial for training your own LoRA? Also I've got a 24GB 3090, so which models would you recommend fine tuning on?
as integrated in https://github.com/Ph0rk0z/text-generation-webui-testing/
-
Monkeypatch Issues
if you like the "monkeypatch" https://github.com/Ph0rk0z/text-generation-webui-testing/ is better. I think in the discussion someone got it running on windows.
- My Lora training locally experiments
-
Any news on training LoRAs in 4-bit mode?
https://github.com/Ph0rk0z/text-generation-webui-testing < 4bit lora use from the UI on old GPTQ
-
Keep your GPUs cool
well im running it with oobabooga/text-generation-webui and 8 bit now works after i did this fix Add 8bit threshold for my Pascal card. I use 1.5 or 1.0, otherwise NaN ยท Ph0rk0z/text-generation-webui-testing@ecad08f (github.com)
- 4bit LoRA Guide for Oobabooga!
What are some alternatives?
alpaca-lora - Instruct-tune LLaMA on consumer hardware
bitsandbytes - Accessible large language models via k-bit quantization for PyTorch.
sparsegpt-for-LLaMA - Code for the paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot" with LLaMA implementation.
alpaca_lora_4bit
trl - Train transformer language models with reinforcement learning.
alpaca_lora_4bit_readme - Just a simple HowTo for https://github.com/johnsmith0031/alpaca_lora_4bit
Sparsebit - A model compression and acceleration toolbox based on pytorch.
private-gpt - Interact with your documents using the power of GPT, 100% privately, no data leaks
axolotl - Go ahead and axolotl questions
GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.