[R] RWKV-4 7B release: an attention-free RNN language model matching GPT-J performance (14B training in progress)

• RWKV-LM

  84 11,619 8.8 Python

  RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Github: https://github.com/BlinkDL/RWKV-LM

• transformers

  175 124,557 10.0 Python

  🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project