axolotl
Our great sponsors
| axolotl | gpt-llm-trainer | |
|---|---|---|
| 29 | 4 | |
| 5,811 | 3,795 | |
| 25.9% | - | |
| 9.8 | 5.2 | |
| about 7 hours ago | about 1 month ago | |
| Python | Jupyter Notebook | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
axolotl
-
Ask HN: Most efficient way to fine-tune an LLM in 2024?
The approach I see used is axolotl with QLoRA using cloud GPUs which can be quite cheap.
https://github.com/OpenAccess-AI-Collective/axolotl
- FLaNK AI - 01 April 2024
-
LoRA from Scratch implementation for LLM finetuning
https://github.com/OpenAccess-AI-Collective/axolotl
- Optimized Triton Kernels for full fine tunes
- Axolotl
-
Let’s Collaborate to Build a High-Quality, Open-Source Dataset for LLMs!
One option is to look at what Axolotl uses. They have a list of different dataset formats that they support. They're mostly in JSON with specific field names, so you could start putting a dataset together with a text editor or a JSON editor.
- Axolotl: Streamline fine-tuning of AI models
-
Dataset Creation Tools?
You can save that overall set into a json file and load it up as training data in whatever you're using. I'm using axolotl for it at the moment. Though a GUI based option is probably best for the first couple of tries until you get a feel for the options.
-
Progress on Reproducing Phi-1/1.5
Looking forward to the results! If it turns out the dataset is reproducible, then it might be a good candidate for ReLora training on axolotl!
gpt-llm-trainer
- FLaNK Stack Weekly 06 Nov 2023
-
Show HN: Fine-tune your own Llama 2 to replace GPT-3.5/4
Very nice, thanks!
Check out what Matt Shumer put together as well: https://github.com/mshumer/gpt-llm-trainer.
I have used his trainer for auto distillation of GPT-4 into GPT3.5 fine tunes, but plan to do the same for Llama as well.
Cheers!
-
[D] Anyone tried gpt-llm-trainer?
Hey guys, so I stumbled upon this Linkedin post, this guy was showing a jupyter notebook on google colab and was explaining step by step how to train your own model to accomplish very specific tasks, and I believe the base model he was using Llama 2 7B Fine tuning version. This is the github link: https://github.com/mshumer/gpt-llm-trainer
- GPT-LLM-Trainer
What are some alternatives?
signal-cli - signal-cli provides an unofficial commandline, JSON-RPC and dbus interface for the Signal messenger.
OpenPipe - Turn expensive prompts into cheap fine-tuned models
LoRA - Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
Llama-2-Onnx
mlc-llm - Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
trieve - All-in-one infrastructure for building search, recommendations, and RAG. Trieve combines search language models with tools for tuning ranking and relevance.
LMFlow - An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
open_model_zoo - Pre-trained Deep Learning models and demos (high quality and extremely fast)
koboldcpp - A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
vllm - A high-throughput and memory-efficient inference and serving engine for LLMs
deepeval - The LLM Evaluation Framework