h2o-llmstudio
alpaca_lora_4bit
h2o-llmstudio | alpaca_lora_4bit | |
---|---|---|
13 | 41 | |
3,614 | 529 | |
3.3% | - | |
9.3 | 8.6 | |
about 14 hours ago | 5 months ago | |
Python | Python | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
h2o-llmstudio
- Paid dev gig: develop a basic LLM PEFT finetuning utility
-
building LLM model to answer question
Vector databases are probably a good place to start, though you've already tried LlamaIndex. You might want to try https://github.com/h2oai/h2o-llmstudio and https://github.com/h2oai/h2ogpt.
- [P] Uptraining a pretrained model using company data?
-
Permissive LLaMA 7b chat/instruct model
Training framework: https://github.com/h2oai/h2o-llmstudio
-
Is what I need possible currently?
Check out LLM Studio for fine tuning LLMs. Open source: https://github.com/h2oai/h2o-llmstudio
- FLaNK Stack Weekly for 30 April 2023
- FLaNK Stack Weekly for 24April2023
- GitHub - h2oai/h2o-llmstudio: H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs
- New Open Source Framework and No-Code GUI for Fine-Tuning LLMs: H2O LLM Studio
- Can an average person learn how to build a LLM model?
alpaca_lora_4bit
-
Open Inference Engine Comparison | Features and Functionality of TGI, vLLM, llama.cpp, and TensorRT-LLM
For training there is also https://github.com/johnsmith0031/alpaca_lora_4bit
-
Quantized 8k Context Base Models for 4-bit Fine Tuning
I've been trying to fine tune an erotica model on some large context chat history (reverse proxy logs) and a literotica-instruct dataset I made, with a max context of 8k. The large context size eats a lot of VRAM so I've been trying to find the most efficient way to experiment considering I'd like to do multiple runs to test some ideas. So I'm going to try and use https://github.com/johnsmith0031/alpaca_lora_4bit, which is supposed to train faster and use less memory than qlora.
-
A simple repo for fine-tuning LLMs with both GPTQ and bitsandbytes quantization. Also supports ExLlama for inference for the best speed.
Follow up the popular work of u/tloen alpaca-lora, I wrapped the setup of alpaca_lora_4bit to add support for GPTQ training in form of installable pip packages. You can perform training and inference with multiple quantizations method to compare the results.
-
Does we still need monkey patch with exllama loader for lora?
" Using LoRAs with GPTQ-for-LLaMa This requires using a monkey patch that is supported by this web UI: https://github.com/johnsmith0031/alpaca_lora_4bit"
-
Why isnโt QLoRA being used more widely for fine tuning models?
4-bit GPTQ LoRA training was available since early April. I did not see any comparison to it in the QLoRA paper or even a mention, so it makes me think they were not aware it already existed.
- Fine-tuning with alpaca_lora_4bit on 8k context SuperHOT models
-
Any guide/intro to fine-tuning anywhere?
https://github.com/johnsmith0031/alpaca_lora_4bit is still the SOTA - Faster than qlora, trains on a GPTQ base.
-
"Samantha-33B-SuperHOT-8K-GPTQ" now that's a great name for a true model.
I would also like to know how one would finetune this in 4 bit? I think one could take the merged 8K PEFT with the LLaMA weights, and then quantize it to 4 bit, and then train with https://github.com/johnsmith0031/alpaca_lora_4bit ?
-
Help with QLoRA
I was under the impression that you just git clone this repo into text-generation-webui/repositories (so you would have GPTQ_for_Llama and alpaca_lora_4bit in the folder), and then just load with monkey patch. Is that not correct? I also tried just downloading alpaca_lora_4bit on its own, git cloning text-gen-webui within it, and installing requirements.txt for both and running with monkey patch. I was following the sections of alpaca_lora_4bit, "Text Generation Webui Monkey Patch" and "monkey patch inside webui"
-
Best uncensored model for an a6000
I dont have any familiarity with esxi, but I can say that there are quite a few posts about people doing it on proxmox. I've currently got a machine with 2x3090 passing through to VM's. When I'm training, I pass them both through to the same VM and can do lora 4-bit training on llama33 using https://github.com/johnsmith0031/alpaca_lora_4bit. Then, at inference time, I run a single card into a different VM, and have an extra card available for experimentation.
What are some alternatives?
h2ogpt - Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://codellama.h2o.ai/
flash-attention - Fast and memory-efficient exact attention
killport - A command-line tool to easily kill processes running on a specified port.
qlora - QLoRA: Efficient Finetuning of Quantized LLMs
HealthGPT - Query your Apple Health data with natural language ๐ฌ ๐ฉบ
StableLM - StableLM: Stability AI Language Models
bark - ๐ Text-Prompted Generative Audio Model
safetensors - Simple, safe way to store and distribute tensors
pandas-ai - Chat with your database (SQL, CSV, pandas, polars, mongodb, noSQL, etc). PandasAI makes data analysis conversational using LLMs (GPT 3.5 / 4, Anthropic, VertexAI) and RAG.
alpaca-lora - Instruct-tune LLaMA on consumer hardware
ue5-llama-lora - A proof-of-concept project that showcases the potential for using small, locally trainable LLMs to create next-generation documentation tools.
transformers - ๐ค Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.