ChatRWKV VS SillyTavern

Most LLMs actually do a decent job out of the box if you ask them for step by step instructions. Tree of tough is one way to improve the results, reflexion is another that can be used separate or additionally. The downside is that most models will run quickly into their token limit (around 2k for most). However the new SuperHot models can handle up to 8k and then there are the RMVK-Raven models, they are RNNs and not transformers like all the other LLMs and can theoretically handle infinite context lengths (but they loose "focus" after a while).

RWKV models inference: https://github.com/BlinkDL/ChatRWKV (fast CUDA).

I'm most interested in that last one. I think I heard the RWKV models are very fast, don't need much Ram, and can have huge context tokens, so maybe their 14b can work for me. I wasn't sure how ready for use they were though, but looking more into it, stuff like rwkv.cpp and ChatRWKV and a whole lot of other community projects are mentioned on their github.

You can use ChatRWKV v2 (https://github.com/BlinkDL/ChatRWKV) to run Raven🐦 (compatible with vanilla RWKV):

I feel compelled to summon /u/bo_peng here and to mention his work on RWKV. (See https://github.com/BlinkDL/ChatRWKV and related repos.)

Please test https://github.com/BlinkDL/ChatRWKV which is a good chatbot despite only trained on the Pile :)

Right, but it's certainly easier for people who might not even know what "API" stands for, and that's quite nifty. As far as self-hosted frontends go, I can personally recommend SillyTavern[1] in the browser, ChatterUI[2] on mobile, and ShellGPT[3] for CLI. LobeChat looks pretty cool, though! I'll definitely check it out.

[1] https://github.com/SillyTavern/SillyTavern

[2] https://github.com/Vali-98/ChatterUI

[3] https://github.com/TheR1D/shell_gpt

I don't think there is any enumeration for {{char}} macros. Here is some good discussion on the subject.

As for SillyTavern, you need the main SillyTavern frontend and SillyTavern-extras (for TTS, STT, etc.) They're pretty easy to install. SillyTavern connects to oobabooga and SillyTavern-extras via API.

Finally, no matter what backend I use, I need it to be compatible with my power-user frontend, SillyTavern. That way I always use the same UI, with the characters I created and extensions I want, e. g. web search, XTTS text-to-speech and Whisper speech recognition for real-time voice chat - and all of that local!

SillyTavern v1.10.5 frontend (not the latest as I don't want to upgrade mid-test)