-
RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
-
RWKV-v2-RNN-Pile
RWKV-v2-RNN trained on the Pile. See https://github.com/BlinkDL/RWKV-LM for details.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
And RWKV-3 is better. You are welcome to join the project (https://github.com/BlinkDL/RWKV-LM) to build upon it (I am an independent researcher).
See https://github.com/BlinkDL/RWKV-v2-RNN-Pile for the ppl vs ctxlen curve :)
Related posts
-
Ask HN: Freelancer? Seeking freelancer? (May 2024)
-
More Low-Bit LLMs
-
Kolmogorov-Arnold Network for Reinforcement Leaning, Initial Experiments
-
Create an AI prototyping environment using Jupyter Lab IDE with Typescript, LangChain.js and Ollama for rapid AI prototyping
-
Show HN: FileKitty – Combine and label text files for LLM prompt contexts