Vector Search Indexes

• pgvector

  16 1,908 7.1 C

  Open-source vector similarity search for Postgres

  

  Apart from search, ANN could be use for recommendations, classification, and other information retrieval problems.

  Currently, ES and Solr, both based on Lucene, can't really manage vector representations, as they are mainly based on inverted indexes to n-grams.

  ANN potential applications extend to audio, bioinformatics, video, among any modality that can be represented as a vector. All you need is an encoder! How nice.

  Faiss is definitely powerful. I have been running experiments using 80 million vectors that map to legal documents, and vectorizing protein-folds (using Alphafold). While it is an interesting technology, at this moment is more a lib or tool than a full-featured product like ES or Solr.

  ATM, updating a Faiss index is a non trivial process, with many of the workflow tools you would see in ES missing.

  I recently been found with pgvector (postgres + vector support) https://github.com/ankane/pgvector. Perhaps less performant, but easier to work with for teams. With support of migrations, ORM, sharing, and all the postgres goodies.

• Weaviate

  14 3,919 8.5 Go

  Weaviate is an open source vector database that stores both objects and vectors, allowing for combining vector search with structured filtering with the fault-tolerance and scalability of a cloud-native database, all accessible through GraphQL, REST, and various language clients.

  

• SonarLint

  www.sonarlint.org

  sponsored

  Clean code begins in your IDE with SonarLint. Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.

  

• vald

  1 1,127 8.6 Go

  Vald. A Highly Scalable Distributed Vector Search Engine

  

• Milvus

  42 15,900 9.9 Go

  A cloud-native vector database, storage for next generation AI applications

  

• nmslib

  1 2,988 5.2 C++

  Non-Metric Space Library (NMSLIB): An efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.

  

  nmslib (https://github.com/nmslib/nmslib) supports sparse vectors for some of its spaces. It has fewer indexing methods than faiss, though.

  https://github.com/nmslib/nmslib/blob/master/manual/spaces.m...

• InfluxDB

  www.influxdata.com

  sponsored

  Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.

  

Suggest a related project