Using Llamafiles for Embeddings in Local RAG Applications

• BinaryVectorDB

  1 178 6.7 Python

  Efficient vector database for hundred millions of embeddings.

  

This style of embeddings could be quite lightweight/cheap/efficient https://github.com/cohere-ai/BinaryVectorDB

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• ollama

  225 72,781 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

Could you sidestep inference altogether? Just return the top N results by cosine similarity (or full text search) and let the user find what they need?

https://ollama.com models also works really well on most modern hardware

Suggest a related project