pg_cjk_parser
pgvector
Our great sponsors
| pg_cjk_parser | pgvector | |
|---|---|---|
| 1 | 10 | |
| 4 | 518 | |
| - | 3.7% | |
| 0.3 | 8.9 | |
| over 1 year ago | 2 months ago | |
| C | C | |
| - | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pg_cjk_parser
pgvector
-
Find anything fast with Google's vector search technology
Check out pgvector: https://github.com/ankane/pgvector (disclosure: am author)
It uses IVFFlat indexing, but could be extended to support product quantization / ScaNN.
-
PostgreSQL 14 Released
I recall seeing some library that adds vector search to Postgres. Maybe https://github.com/ankane/pgvector?
Also there's Pinecone (https://www.pinecone.io) which can sit alongside Postgres or any other data warehouse and ingest vector embeddings + metadata for vector search/retrieval.
-
Vector Search Indexes
Apart from search, ANN could be use for recommendations, classification, and other information retrieval problems.
Currently, ES and Solr, both based on Lucene, can't really manage vector representations, as they are mainly based on inverted indexes to n-grams.
ANN potential applications extend to audio, bioinformatics, video, among any modality that can be represented as a vector. All you need is an encoder! How nice.
Faiss is definitely powerful. I have been running experiments using 80 million vectors that map to legal documents, and vectorizing protein-folds (using Alphafold). While it is an interesting technology, at this moment is more a lib or tool than a full-featured product like ES or Solr.
ATM, updating a Faiss index is a non trivial process, with many of the workflow tools you would see in ES missing.
I recently been found with pgvector (postgres + vector support) https://github.com/ankane/pgvector. Perhaps less performant, but easier to work with for teams. With support of migrations, ORM, sharing, and all the postgres goodies.
-
Postgres Full-Text Search: A Search Engine in a Database
If you are looking to do semantic search (Cosine similarity) + filtering (SQL) on vectors (audio, text, video, basically anything that can be converted into a vector) I suggest, https://github.com/ankane/pgvector
- ankane/pgvector Open-source vector similarity search for Postgres
- Pgvector – vector similarity search for Postgres
- Introducing pgvector: open-source vector similarity search for Postgres Fire Great for recommendations, search, and more
What are some alternatives?
milvus - An open-source vector database for scalable similarity search and AI applications.
faiss - A library for efficient similarity search and clustering of dense vectors.
smlar - PostgreSQL extension for an effective similarity search || mirror of git://sigaev.ru/smlar.git || see https://www.pgcon.org/2012/schedule/track/Hacking/443.en.html
qdrant - Qdrant - vector similarity search engine with extended filtering support
Weaviate - Weaviate is a cloud-native, modular, real-time vector search engine
pg_search - pg_search builds ActiveRecord named scopes that take advantage of PostgreSQL’s full text search
hn-search - Hacker News Search
nmslib - Non-Metric Space Library (NMSLIB): An efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.
ann-benchmarks - Benchmarks of approximate nearest neighbor libraries in Python
nhost - The Open-Source GraphQL Backend (Firebase Alternative).
zombodb - Making Postgres and Elasticsearch work together like it's 2022