FastChat
CASALIOY
Our great sponsors
FastChat | CASALIOY | |
---|---|---|
82 | 6 | |
32,751 | 231 | |
4.5% | 1.3% | |
9.7 | 8.7 | |
7 days ago | 5 months ago | |
Python | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
FastChat
-
LLMs on your local Computer (Part 1)
FastChat
- FLaNK AI for 11 March 2024
- FLaNK 04 March 2024
- ChatGPT for Teams
-
LM Studio β Discover, download, and run local LLMs
How does it compare with something like FastChat? https://github.com/lm-sys/FastChat
Feature set seems like a decent amount of overlap. One limitation of FastChat, as far as I can tell, is that one is limited to the models that FastChat supports (though I think it would be minor to modify it to support arbitrary models?)
-
Video-LLaVA
Looks like the Vicuna repo is Apache 2.0 also[1].
What's the interpretation of copyright law that would prevent the code being Apache 2.0 based on the source of the fine-tuning dataset?
-
π₯π Top 10 Open-Source Must-Have Tools for Crafting Your Own Chatbot π€π¬
Check how to start with FastChat. Support FastChat on GitHub β
-
Show HN: ChatAPI β PWA to Use ChatGPT by API Build with Alpine.js
For something a little heavier but much more robust in terms of features/functionality I've been enjoying FastChat: https://github.com/lm-sys/FastChat
It allows you to plug in different backends so that you can use OpenAI compatible clients with various LLM's, selfhosted or otherwise.
- FLaNK Stack Weekly 09 Oct 2023
-
Show HN: Open-source proxy server for Llama2, GPT-4, Claude2 with Logging,Cache
If you do want to self-host - there's some great libraries like https://github.com/lm-sys/FastChat and https://github.com/ggerganov/llama.cpp that might be helpful
If none of these really solve your issue - feel free to email me and I'm happy to help you figure something out - [email protected]
CASALIOY
-
Open LLM suggestions
Also this is 50% slower at ingestion. We use multithreaded ingestion that sips a SOTU.txt in 50ms whereas privateGPT takes about 2 seconds. CASALIOY
-
ChatGPT on a Raspberry Pi Zero W with OLED display
If you have some spare time you could give Casalioy a try. This has not been tested on a RPi yet.
-
Air-gapped langchain Agent. Talk to your Data privately
Check it out HERE <-
check out this
Here's a demo screencast with an ingested text just as long as this comment (on my i5-9600k 16GB)
This should work
What are some alternatives?
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
llama.cpp - LLM inference in C/C++
gpt4all - gpt4all: run open-source LLMs anywhere
bitsandbytes - Accessible large language models via k-bit quantization for PyTorch.
LocalAI - :robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
llama-cpp-python - Python bindings for llama.cpp
mlc-llm - Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
llama - Inference code for Llama models
ollama - Get up and running with Llama 2, Mistral, Gemma, and other large language models.
OpenLLM - Operating LLMs in production
MiniGPT-4 - Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)
litellm - Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)