RWKV-CUDA Alternatives

RWKV-LM

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

ChatRWKV

ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.

RWKV-v2-RNN-Pile

RWKV-v2-RNN trained on the Pile. See https://github.com/BlinkDL/RWKV-LM for details.

rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

web-rwkv

Implementation of the RWKV language model in pure WebGPU/Rust.

AI-Writer

AI 写小说，生成玄幻和言情网文等等。中文预训练生成模型。采用我的 RWKV 模型，类似 GPT-2 。AI写作。RWKV for Chinese novel generation.

SmallInitEmb

LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence

RWKV-LM-LoRA

RWKV is a RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

token-shift-gpt

Implementation of Token Shift GPT - An autoregressive model that solely relies on shifting the sequence space for mixing

RWKV-infctx-trainer

RWKV infctx trainer, for training arbitary context sizes, to 10k and beyond!

blog_code

ai00_server

The all-in-one RWKV runtime box with embed, RAG, AI agents, and more.

faster-rwkv