xTuring
alpaca_lora_4bit
xTuring | alpaca_lora_4bit | |
---|---|---|
31 | 41 | |
2,525 | 529 | |
0.9% | - | |
8.4 | 8.6 | |
about 1 month ago | 5 months ago | |
Python | Python | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
xTuring
-
I'm developing an open-source AI tool called xTuring, enabling anyone to construct a Language Model with just 5 lines of code. I'd love to hear your thoughts!
Explore the project on GitHub here.
-
LLaMA 2 fine-tuning made easier and faster
If you're curious, I encourage you to: - Dive deeper with the LLaMA 2 tutorial here. - Explore the project on GitHub here. - Connect with our community on Discord here.
-
RAG vs. Fine-Tuning
If you want best performance, you need to do both RAG and fine-tuning very well. There are plenty of resources on doing fine-tuning thought. I'm one of the contributors to https://github.com/stochasticai/xturing project focused on fine-tuning LLMs. You can find help in the discord channel listed on the GitHub.
- Build, customize and control your own personal LLMs via xTuring OSS
-
Finetuning LLaMA 2 (the base models) ?
What tools do you use and achieved great results ? … For me i have tried xturing and SFTTrainer and they got me a semi okay results.
-
Finetuning using Google Colab (Free Tier)
Code: https://github.com/stochasticai/xTuring/blob/main/examples/llama/llama_lora_int8.py Colab: https://colab.research.google.com/drive/1SQUXq1AMZPSLD4mk3A3swUIc6Y2dclme?usp=sharing
-
I would like to try my hand at finetuning some models. What is the best way to start? I have some questions that I'd appreciate your help on.
We are a group of researchers out of Harvard working on open-source library called xTuring, focused on fine-tuning LLMs: https://github.com/stochasticai/xturing.
-
Fine tuning on my tweets
Fine tuning I was thinking about using this (low GPU memory footprint): https://github.com/stochasticai/xturing/blob/main/examples/int4_finetuning/README.md
-
Colab for finetuning llama models in 4-bit?
I can't speak for QLORA, as I haven't had a chance to get an implementation working, but I've had success with StochasticAI's Xturing. It's by far the most streamlined method of finetuning I've come across, and they offer int8 and int4 fintuning (but only for llama-7B).
- Just wanna say this.
alpaca_lora_4bit
-
Open Inference Engine Comparison | Features and Functionality of TGI, vLLM, llama.cpp, and TensorRT-LLM
For training there is also https://github.com/johnsmith0031/alpaca_lora_4bit
-
Quantized 8k Context Base Models for 4-bit Fine Tuning
I've been trying to fine tune an erotica model on some large context chat history (reverse proxy logs) and a literotica-instruct dataset I made, with a max context of 8k. The large context size eats a lot of VRAM so I've been trying to find the most efficient way to experiment considering I'd like to do multiple runs to test some ideas. So I'm going to try and use https://github.com/johnsmith0031/alpaca_lora_4bit, which is supposed to train faster and use less memory than qlora.
-
A simple repo for fine-tuning LLMs with both GPTQ and bitsandbytes quantization. Also supports ExLlama for inference for the best speed.
Follow up the popular work of u/tloen alpaca-lora, I wrapped the setup of alpaca_lora_4bit to add support for GPTQ training in form of installable pip packages. You can perform training and inference with multiple quantizations method to compare the results.
-
Does we still need monkey patch with exllama loader for lora?
" Using LoRAs with GPTQ-for-LLaMa This requires using a monkey patch that is supported by this web UI: https://github.com/johnsmith0031/alpaca_lora_4bit"
-
Why isn’t QLoRA being used more widely for fine tuning models?
4-bit GPTQ LoRA training was available since early April. I did not see any comparison to it in the QLoRA paper or even a mention, so it makes me think they were not aware it already existed.
- Fine-tuning with alpaca_lora_4bit on 8k context SuperHOT models
-
Any guide/intro to fine-tuning anywhere?
https://github.com/johnsmith0031/alpaca_lora_4bit is still the SOTA - Faster than qlora, trains on a GPTQ base.
-
"Samantha-33B-SuperHOT-8K-GPTQ" now that's a great name for a true model.
I would also like to know how one would finetune this in 4 bit? I think one could take the merged 8K PEFT with the LLaMA weights, and then quantize it to 4 bit, and then train with https://github.com/johnsmith0031/alpaca_lora_4bit ?
-
Help with QLoRA
I was under the impression that you just git clone this repo into text-generation-webui/repositories (so you would have GPTQ_for_Llama and alpaca_lora_4bit in the folder), and then just load with monkey patch. Is that not correct? I also tried just downloading alpaca_lora_4bit on its own, git cloning text-gen-webui within it, and installing requirements.txt for both and running with monkey patch. I was following the sections of alpaca_lora_4bit, "Text Generation Webui Monkey Patch" and "monkey patch inside webui"
-
Best uncensored model for an a6000
I dont have any familiarity with esxi, but I can say that there are quite a few posts about people doing it on proxmox. I've currently got a machine with 2x3090 passing through to VM's. When I'm training, I pass them both through to the same VM and can do lora 4-bit training on llama33 using https://github.com/johnsmith0031/alpaca_lora_4bit. Then, at inference time, I run a single card into a different VM, and have an extra card available for experimentation.
What are some alternatives?
quivr - Your GenAI Second Brain 🧠 A personal productivity assistant (RAG) ⚡️🤖 Chat with your docs (PDF, CSV, ...) & apps using Langchain, GPT 3.5 / 4 turbo, Private, Anthropic, VertexAI, Ollama, LLMs, Groq that you can share with users ! Local & Private alternative to OpenAI GPTs & ChatGPT powered by retrieval-augmented generation.
flash-attention - Fast and memory-efficient exact attention
axolotl - Go ahead and axolotl questions
qlora - QLoRA: Efficient Finetuning of Quantized LLMs
FinGPT - FinGPT: Open-Source Financial Large Language Models! Revolutionize 🔥 We release the trained model on HuggingFace.
StableLM - StableLM: Stability AI Language Models
awesome-totally-open-chatgpt - A list of totally open alternatives to ChatGPT
safetensors - Simple, safe way to store and distribute tensors
Meshtasticator - Discrete-event and interactive simulator for Meshtastic.
alpaca-lora - Instruct-tune LLaMA on consumer hardware
Zicklein - Finetuning instruct-LLaMA on german datasets.
transformers - 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.