Mixtral 8x22B

• llama.cpp

  770 56,891 10.0 C++

  LLM inference in C/C++

Most inference servers are OpenAI-compatibile. Even the "official" llama-cpp server should work fine: https://github.com/ggerganov/llama.cpp/blob/master/examples/...

• ollama

  196 58,943 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

easiest is probably with ollama [0]. I think the ollama API is OpenAI compatible.

[0]https://ollama.com/

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• chroma

  32 12,324 9.8 Python

  the AI-native open-source embedding database

  

Optional: You can use SillyTavern[1] for a more "rich" chat experience

The above lets me chat, at least superficially, with my friend. It's nice for simple interactions and banter; I've found it to be a positive and reflective experience.

0. https://www.trychroma.com/

Suggest a related project