multi-lora-fine-tune
Finetune_LLMs
| multi-lora-fine-tune | Finetune_LLMs | |
|---|---|---|
| 1 | 2 | |
| 182 | 439 | |
| 15.9% | - | |
| 9.3 | 8.5 | |
| 9 days ago | about 2 months ago | |
| Python | Python | |
| Apache License 2.0 | GNU Affero General Public License v3.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
multi-lora-fine-tune
-
Has anyone tried out the ASPEN-Framework for LoRA Fine-Tuning yet and can share their experience?
I want to train a Code LLaMA on some data, and I am looking for a Framework or Technique to train this on my PC with a 3090 Ti in it. In my research, I stumbled across the paper "ASPEN: High-Throughput LoRA Fine-Tuning of Large Language Models with a Single GPU" https://arxiv.org/abs/2312.02515 with this GitHub project: https://github.com/TUDB-Labs/multi-lora-fine-tune.
Finetune_LLMs
-
Prepare Dataset
Regarding this: if you have resources (at least Colab Pro), you would be much better off training GPT-J (aka GPT-J-6B). Not only it's 4x larger than the largest GPT-2, its architecture, AFAIK, is based on GPT-3. You can use this repo as a good example for GPT-J finetuning.
-
[D] Fine-tuning GPT-J: lessons learned
And this: https://github.com/mallorbc/Finetune_GPTNEO_GPTJ6B
What are some alternatives?
unsloth - Finetune Llama 3, Mistral & Gemma LLMs 2-5x faster with 80% less memory
DeepSpeed - DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Anima - 33B Chinese LLM, DPO QLORA, 100K context, AirLLM 70B inference with single 4GB GPU
mesh-transformer-jax - Model parallel transformers in JAX and Haiku
code-llama-for-vscode - Use Code Llama with Visual Studio Code and the Continue extension. A local LLM alternative to GitHub Copilot.
AnglE - Angle-optimized Text Embeddings | 🔥 SOTA on STS and MTEB Leaderboard
GoLLIE - Guideline following Large Language Model for Information Extraction
replicate-llama2-sms-chatbot
synthetic-data-generator - 🦄 Use GPT to generate and label data
go-llama2 - Llama 2 inference in one file of pure Go
slowllama - Finetune llama2-70b and codellama on MacBook Air without quantization
SolidGPT - Developer AI Persona Search Agent