relora
AtomGPT
relora | AtomGPT | |
---|---|---|
2 | 1 | |
399 | 189 | |
- | - | |
8.3 | 10.0 | |
20 days ago | 9 months ago | |
Jupyter Notebook | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
relora
- ReLoRA: High-Rank Training Through Low-Rank Updates
-
Aurelian: 70B 32K story-writing (and more) [Alpha]
Similarly, the dominant components selected before training may change order as you train. ReLORA is basically a way to re-align and make sure you are always training something close to the current most important params.
AtomGPT
What are some alternatives?
LongLoRA - Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)
realtime-bakllava - llama.cpp with BakLLaVA model describes what does it see
adanet - Fast and flexible AutoML with learning guarantees.
vllm - A high-throughput and memory-efficient inference and serving engine for LLMs
chatgpt-extractive-shortener - Shortens a paragraph of text with ChatGPT, using successive rounds of word-level extractive summarization.
GoLLIE - Guideline following Large Language Model for Information Extraction
safe-rlhf - Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback
ray-llm - RayLLM - LLMs on Ray
pinferencia - Python + Inference - Model Deployment library in Python. Simplest model inference server ever.
Flipped-Learning - [ICLR 2023] Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners
Cornucopia-LLaMA-Fin-Chinese - 聚宝盆(Cornucopia): 中文金融系列开源可商用大模型,并提供一套高效轻量化的垂直领域LLM训练框架(Pretraining、SFT、RLHF、Quantize等)
ml-engineering - Machine Learning Engineering Open Book