Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
LongLoRA Alternatives
Similar projects and alternatives to LongLoRA
-
-
arxiv-sanity-lite
arxiv-sanity lite: tag arxiv papers of interest get recommendations of similar papers in a nice UI using SVMs over tfidf feature vectors based on paper abstracts.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
relora
Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates
-
discus
A data-centric AI package for ML/AI. Get the best high-quality data for the best results. Discord: https://discord.gg/t6ADqBKrdZ
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
-
xTuring
Build, customize and control you own LLMs. From data pre-processing to fine-tuning, xTuring provides an easy way to personalize open-source LLMs. Join our discord community: https://discord.gg/TgHXuSJEk6
-
llama-peft-tuner
Tune LLaMa-7B on Alpaca Dataset using PEFT / LORA Based on @zphang's https://github.com/zphang/minimal-llama scripts.
-
LLM-Finetuning-Hub
Discontinued Toolkit for fine-tuning, ablating and unit-testing open-source LLMs. [Moved to: https://github.com/georgian-io/LLM-Finetuning-Toolkit]
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a better LongLoRA alternative or higher similarity.
LongLoRA reviews and mentions
Posts with mentions or reviews of LongLoRA.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-12-15.
-
Ask HN: AI/ML papers to catch up with current state of AI?
LongAlpaca / One of many ways to extend context, and a useful dataset / https://arxiv.org/abs/2309.12307
-
Aurelian: 70B 32K story-writing (and more) [Alpha]
Finally, LongLORA is a method to reduce the number of computations over a large context, and also specifically train the embed and norm layers fully, that is, no quantization or LORA for those. They are small layers and easy to train without too much VRAM cost, but the LongLORA authors noticed they have a big impact on long context performance. I am not using their computation reduction methods, but I am using their suggestion to train embed/norm layers fully.
-
Why train on Yi 4K instead of 200K?
That used to be true, but things like LongLORA and LongQLoRA demonstrate that you can increase the context length of a foundation model.
-
Using Overfitting to Debug My LLM [P]
For reference, I am using the LongLoRA SFT implementation for fine-tuning a CodeLLaMA model on a code generation instruction. I have also attached my evaluation code below:
-
A note from our sponsor - InfluxDB
www.influxdata.com | 1 May 2024
Stats
Basic LongLoRA repo stats
4
2,445
9.1
3 months ago
dvlab-research/LongLoRA is an open source project licensed under Apache License 2.0 which is an OSI approved license.
The primary programming language of LongLoRA is Python.
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com