GPTQ-Merged
axolotl
| GPTQ-Merged | axolotl | |
|---|---|---|
| 2 | 29 | |
| 2 | 6,105 | |
| - | 13.7% | |
| 8.1 | 9.8 | |
| 8 months ago | 5 days ago | |
| Python | Python | |
| - | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
GPTQ-Merged
-
Slow inference on R720 w/P40 (or not)?
Also autograd from here: https://github.com/Ph0rk0z/text-generation-webui-testing/ and it's matching GPTQ: https://github.com/Ph0rk0z/GPTQ-Merged/tree/dual-model
-
Finetuning on multiple GPUs
Probably need to add universal support to the native functions because it uses llama only. If you edit the load_llama functions in autograd py to use generic stuff like this: https://github.com/Ph0rk0z/GPTQ-Merged/blob/dual-model/src/alpaca_lora_4bit/autograd_4bit.py it has a good chance of working. Might need to also add trust_remote_code.
axolotl
-
Ask HN: Most efficient way to fine-tune an LLM in 2024?
The approach I see used is axolotl with QLoRA using cloud GPUs which can be quite cheap.
https://github.com/OpenAccess-AI-Collective/axolotl
- FLaNK AI - 01 April 2024
-
LoRA from Scratch implementation for LLM finetuning
https://github.com/OpenAccess-AI-Collective/axolotl
- Optimized Triton Kernels for full fine tunes
- Axolotl
-
Let’s Collaborate to Build a High-Quality, Open-Source Dataset for LLMs!
One option is to look at what Axolotl uses. They have a list of different dataset formats that they support. They're mostly in JSON with specific field names, so you could start putting a dataset together with a text editor or a JSON editor.
- Axolotl: Streamline fine-tuning of AI models
-
Dataset Creation Tools?
You can save that overall set into a json file and load it up as training data in whatever you're using. I'm using axolotl for it at the moment. Though a GUI based option is probably best for the first couple of tries until you get a feel for the options.
-
Progress on Reproducing Phi-1/1.5
Looking forward to the results! If it turns out the dataset is reproducible, then it might be a good candidate for ReLora training on axolotl!
What are some alternatives?
text-generation-webui-testing - A fork of textgen that still supports V1 GPTQ, 4-bit lora and other GPTQ models besides llama.
signal-cli - signal-cli provides an unofficial commandline, JSON-RPC and dbus interface for the Signal messenger.
gpt-llm-trainer
LoRA - Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
mlc-llm - Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
LMFlow - An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
koboldcpp - A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
OpenPipe - Turn expensive prompts into cheap fine-tuned models
xTuring - Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6
libsignal - Home to the Signal Protocol as well as other cryptographic primitives which make Signal possible.
org.signal.Signal
Signal-Desktop - A private messenger for Windows, macOS, and Linux.