text-generation-webui-docker
serge
| text-generation-webui-docker | serge | |
|---|---|---|
| 6 | 40 | |
| 355 | 5,576 | |
| - | 1.3% | |
| 7.4 | 9.8 | |
| 7 days ago | 4 days ago | |
| Dockerfile | Svelte | |
| GNU Affero General Public License v3.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
text-generation-webui-docker
-
How to run llama.cpp or something similar in docker w/ docker-compose ? Guide needed
I have been using docker images from here. It can be connected to SillyTavern with API.
-
Can't load exl2 model with 3060ti(8GB)
Hi guys, I am using Text Gen Webui-docker,I want to load an exl2 model with ExLlamav2_HF. I have tried 34B,13B and 7B,and none of them worked,while I can load it with llama.cpp with .gguf files. Everytime I load the models it gave me sam errors: ``` Traceback (most recent call last):
-
Docker container running on 127.0.0.1 while assigning a different subnet
I'm trying to get the container for text-generation-webui to run, where the setup itself was pretty straight forward.
- Generative AI workloads?
-
Are you selfhosting a ChatGPT alternative?
Here's the dockerized oogabooga link: https://github.com/Atinoda/text-generation-webui-docker
-
Atinoda Docker version on WSL
Has anyone gotten Atinoda/text-generation-webui-docker (github.com) working on WSL? I'm having port issues.
serge
- Show HN: I made an app to use local AI as daily driver
- chatgpt alternative
-
Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2
Very cool, this looks like a combination of chatbot-ui and llama-cpp-python? A similar project I've been using is https://github.com/serge-chat/serge. Nous-Hermes-Llama2-13b is my daily driver and scores high on coding evaluations (https://huggingface.co/spaces/mike-ravkine/can-ai-code-resul...).
-
LeCun: Qualcomm working with Meta to run Llama-2 on mobile devices
You might be pleased to hear that nothing really stops you from doing this today. If you ran Serge[0] on a Mac with Tailscale, you could hack together a decently-accelerated Llama chatbot.
[0] https://github.com/serge-chat/serge
-
Chatbot frontend library in Svelte?
Cannot help you with libraries specifically but both Serge and ChatUI are built using SvelteKit, so the code might be of some use to you.
- We’re back and…
-
Best way to use AMD CPU and GPU
Serge made it really easy for me to get started, but it all CPU-based.
-
Need Help
All that said this project probably solves your problem: https://github.com/serge-chat/serge
- Are you selfhosting a ChatGPT alternative?
-
What the hell??
You can play a little bit with more straightforward local models (the simplest to setup is https://github.com/nsarrazin/serge ), to see that any LLM is basically a party trick.
What are some alternatives?
chatgpt-telegram-bot - 🤖 A Telegram bot that integrates with OpenAI's official ChatGPT APIs to provide answers, written in Python
gpt4all - gpt4all: run open-source LLMs anywhere
LLaMA-Adapter - Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters [Moved to: https://github.com/OpenGVLab/LLaMA-Adapter]
langflow - ⛓️ Langflow is a dynamic graph where each node is an executable unit. Its modular and interactive design fosters rapid experimentation and prototyping, pushing hard on the limits of creativity.
guidance - A guidance language for controlling large language models. [Moved to: https://github.com/guidance-ai/guidance]
llama.cpp - LLM inference in C/C++
fuseai - Self-Hosted and Open-Source web app to interact with OpenAI APIs. Currently supports ChatGPT, but DALLE and Whisper support is coming.
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
FastChat - An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
llama-gpt - A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
chatbot-ui - AI chat for every model.
RWKV-LM - RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.