Storing word / document vectors in RDBMS

This page summarizes the projects mentioned and recommended in the original post on /r/LanguageTechnology

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • qdrant

    Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/

  • A better option would be using a dedicated open-source vector database like Qdrant, it is more efficient, scalable, and has a convenient API. https://github.com/qdrant/qdrant (disclaimer: I'm part of the team).

  • postgres-word2vec

    utils to use word embedding models like word2vec vectors in a PostgreSQL database

  • I've recently stumbled upon smaller projects, like FREDDY (https://github.com/guenthermi/postgres-word2vec), a Postgres extension that looks interesting. The ability to write ad-hoc similarity queries in SQL seems like it might be valuable in some circumstances. I'm not sure about performance or storage efficacy.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • tensorstore

    Library for reading and writing large multi-dimensional arrays.

  • There are tons of other ways to store vector data, one was just recently released - https://github.com/google/tensorstore

  • txtai

    šŸ’” All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows

  • I have project, txtai that supports vector/semantic/similarity search. It pairs an ANN index with a relational database to support SQL queries.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts