rellm
llama-api-server
| rellm | llama-api-server | |
|---|---|---|
| 7 | 1 | |
| 491 | 182 | |
| - | - | |
| 5.0 | 6.5 | |
| 9 months ago | 8 days ago | |
| Python | Python | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
rellm
-
Run and create custom ChatGPT-like bots with OpenChat
- https://github.com/r2d4/rellm
-
Forcing GPT-4 or GPT-3.5-turbo to adhere to a specific output format
MS guidance as mentioned and ReLLM
- GitHub - r2d4/rellm: Exact structure out of any language model completion.
- AI Showdown: Wizard Vicuna vs. Stable Vicuna, GPT-4 as the judge (test in comments)
-
ReLLM: Exact Structure for Large Language Model Completions
There's probably a better API that wraps generate, but there's a bit more work than the logit mask.
You have to go one token at a time, otherwise the masking becomes combinatoric rather than linear (two tokens at a time -- need to generate all two token pairs, etc.).
But otherwise, that's what the code does! https://github.com/r2d4/rellm/blob/main/rellm/rellm.py#L21
- r2d4/rellm: Exact structure out of any language model completion.
llama-api-server
-
Run and create custom ChatGPT-like bots with OpenChat
Disclaimer: I am curating LLM-tools on github [1]
A few thoughts:
* allow for custom endpoint URLs, this way people can use open source LLMs with a fake openAI API backend like basaran[2] or llama-api-server[3]
* look into better embedding methods for info-retrieval like InstructorEmbeddings or Document Summary Index
* Don't use a single embedding per content item, use multiple to increase retrieval quality
1 https://github.com/underlines/awesome-marketing-datascience/...
2 https://github.com/hyperonym/basaran
3 https://github.com/iaalm/llama-api-server
What are some alternatives?
OpenChat - LLMs custom-chatbots console ⚡
gpt-jargon - Jargon is a natural language programming language specified and executed by LLMs like GPT-4.
convostack - Plug and play embeddable AI chatbot widget and backend deployment framework
NeMo-Guardrails - NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
guidance - A guidance language for controlling large language models. [Moved to: https://github.com/guidance-ai/guidance]
basaran - Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.
awesome-ml - Curated list of useful LLM / Analytics / Datascience resources