SaaSHub helps you find the best software and product alternatives Learn more →
RWKV-LM-LoRA Alternatives
Similar projects and alternatives to RWKV-LM-LoRA
-
text-generation-webui
A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
-
RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
SpikeGPT
Implementation of "SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks"
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
-
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a better RWKV-LM-LoRA alternative or higher similarity.
RWKV-LM-LoRA reviews and mentions
Posts with mentions or reviews of RWKV-LM-LoRA.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-12-09.
- People who've used RWKV, whats your wishlist for it?
-
Scaling Transformers to 1B Tokens
> RWKV
The current versions of RWKV slowly go insane when exposed to sequences that are too long, because the state slowly diverges over time as you increase past the context length of the training session. They are experimenting with ways to avoid this though: https://github.com/Blealtan/RWKV-LM-LoRA/tree/dev-infctx
-
[R] RWKV 14B ctx8192 is a zero-shot instruction-follower without finetuning, 23 token/s on 3090 after latest optimization (16G VRAM is enough, and you can stream layers to save more VRAM)
Someone in RWKV Discord tried it using LoRA (https://github.com/Blealtan/RWKV-LM-LoRA) and the result is quite nice. Join RWKV Discord for latest updates :)
- [P] RWKV 14B is a strong chatbot despite only trained on Pile (16G VRAM for 14B ctx4096 INT8, more optimizations incoming)
-
A note from our sponsor - SaaSHub
www.saashub.com | 14 May 2024
Stats
Basic RWKV-LM-LoRA repo stats
4
403
5.6
10 months ago
Blealtan/RWKV-LM-LoRA is an open source project licensed under Apache License 2.0 which is an OSI approved license.
The primary programming language of RWKV-LM-LoRA is Python.
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com