Our great sponsors
-
text-generation-webui
A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
LibreOffice
Read-only LibreOffice core repo - no pull request (use gerrit instead https://gerrit.libreoffice.org/) - don't download zip, use https://dev-www.libreoffice.org/bundles/ instead (by LibreOffice)
If you're gonna go with the first one then I'd recommend using Oobabooga since it natively supports loading on CPU and offloading to GPU (with a few extra steps to fix a library not being built properly). You may have them already but just to be sure, you will need to install Python 3.10 since 3.10 required for some of the libraries used by Oobabooga, and Git since Oobabooga uses Git to download and update itself. I'd recommend using the one-click installer for Oobabooga since it makes this way easier, too.
For the third, I don't think Oobabooga supports the horde but KoboldAI does. I won't go into how to install KoboldAI since Oobabooga should give you enough freedom with 7B, 13B and maybe 30B models (depending on available RAM), but KoboldAI lets you download some models directly from the web interface, supports using online service providers to run the models for you, and supports the horde with a list of available models to choose from.
SillyTavern, a popular role playing front end for AI language processing, also supports the horde and other online service providers, and will even let you know the expected waiting time and how busy each model provided by the horde is. If you're interested in role playing I'd also recommend running SillyTavern over top of Oobabooga/KoboldAI or through the horde, as it's purpose-built for role playing and includes a number of features not found in the actual programs like Oobabooga or KoboldAI. KoboldAI is dead simple to get working with SillyTavern, but Oobabooga is a little more complicated as you need to enable the API to let SillyTavern connect through.
I mean have you seen this nazi child rape fantasy generator? It's freely available for everyone and when I tried it I just couldn't believe the kind of vile shit it produces! It's evil I tell you!