Mixtral 8x22B

• llama.cpp

  780 57,984 10.0 C++

  LLM inference in C/C++

Most inference servers are OpenAI-compatibile. Even the "official" llama-cpp server should work fine: https://github.com/ggerganov/llama.cpp/blob/master/examples/...

• ollama

  212 66,540 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

easiest is probably with ollama [0]. I think the ollama API is OpenAI compatible.

[0]https://ollama.com/

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• chroma

  32 12,530 9.8 Python

  the AI-native open-source embedding database

  

Optional: You can use SillyTavern[1] for a more "rich" chat experience

The above lets me chat, at least superficially, with my friend. It's nice for simple interactions and banter; I've found it to be a positive and reflective experience.

0. https://www.trychroma.com/

Suggest a related project