Weaviate
pgvector
Our great sponsors
Weaviate | pgvector | |
---|---|---|
76 | 77 | |
9,359 | 8,904 | |
4.0% | 7.3% | |
10.0 | 9.7 | |
6 days ago | 6 days ago | |
Go | C | |
BSD 3-clause "New" or "Revised" License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Weaviate
-
pgvecto.rs alternatives - qdrant and Weaviate
3 projects | 13 Mar 2024
- FLaNK Stack 29 Jan 2024
- Qdrant, the Vector Search Database, raised $28M in a Series A round
-
How to use Weaviate to store and query vector embeddings
In this tutorial, I introduce Weaviate, an open-source vector database, with the thenlper/gte-base embedding model from Alibaba, through Hugging Face's transformers library.
-
Choosing vector database: a side-by-side comparison
This will be solved in Weaviate https://github.com/weaviate/weaviate/issues/2424
-
Who's hiring developer advocates? (October 2023)
Link to GitHub -->
-
Do we think about vector dbs wrong?
Hey @rvrs, I work on Weaviate and we are doing some improvements around increasing write throughput:
1. gRPC. Using gRPC to write vectors has had a really nice performance boost. It is released in Weaviate core but here is still some work on do on the clients. Feel free to get in contact if you would like to try it out.
2. Parameter tuning. lowering `efConstruction` can speed up imports.
3. We are also working on async indexing https://github.com/weaviate/weaviate/issues/3463 which will further speed things up.
In comparison with pgvector, Weaviate has more flexible query options such as hybrid search and quantization to save memory on larger datasets.
-
Pros and cons of vector search in elastic?
Highly opinionated as I'm working for Weaviate, so take my comment with a large portion of salt.
My highly opinionated view is that for Elastic, they're not really open source and the dependency on Java of the Lucene ecosystem is a big disadvantage, so as you already said, speed, they're getting better at this, but if you need to scale, this problem scales with you.
So if you already have ELK stack and don't need to scale, sure go for it otherwise, Weaviate offers real open source, so use it for free on your own infrastructure https://github.com/weaviate/weaviate
-
Lost on LangChain: Can someone help with the Question Answer concept?
If you do not wish to store your private data on pinecone you can use open source alternatives like Weaviate where you can spin up your own instance. Other option could be to use Agents. You'll need to find sutaible agent for your database which will allow LLMs to directly query data from your private database.
-
Questions about memory, tree-of-thought, planning
I tried cromadb but had terrible performance and could not pin down the cause (likely a problem on my end). Weaviate was easy to setup and had excellent performance, this is probably what I will use in the future. Next on my list is txtinstruct, to finetune a model with data that does not change and using a vector db for everything else seems promising.
pgvector
-
Using pgvector To Locate Similarities In Enterprise Data
For this example, I wanted to focus on how pgvector – an open-source vector similarity search for Postgres – can be used to identify data similarities that exist in enterprise data.
-
Pg_vectorize: The simplest way to do vector search and RAG on Postgres
There's an issue in the pgvector repo about someone having several ~10-20million row tables and getting acceptable performance with the right hardware and some performance tuning: https://github.com/pgvector/pgvector/issues/455
I'm in the early stages of evaluating pgvector myself. but having used pinecone I currently am liking pgvector better because of it being open source. The indexing algorithm is clear, one can understand and modify the parameters. Furthermore the database is postgresql, not a proprietary document store. When the other data in the problem is stored relationally, it is very convenient to have the vectors stored like this as well. And postgresql has good observability and metrics. I think when it comes to flexibility for specialized applications, pgvector seems like the clear winner. But I can definitely see pinecone's appeal if vector search is not a core component of the problem/business, as it is very easy to use and scales very easily
- FLaNK 04 March 2024
-
Vector Database and Spring IA
The Spring AI project aims to streamline the development of applications that incorporate artificial intelligence functionality without unnecessary complexity. On this example we use features like: Embedding, Prompts, ETL and save all embedding on PGvector(Postgres Vector database)
-
Use pgvector for searching images on Azure Cosmos DB for PostgreSQL
Official GitHub repository of the pgvector extension
-
Store embeddings in Azure Cosmos DB for PostgreSQL with pgvector
The pgvector extension adds vector similarity search capabilities to your PostgreSQL database. To use the extension, you have to first create it in your database. You can install the extension, by connecting to your database and running the CREATE EXTENSION command from the psql command prompt:
-
Are we at peak vector database?
It’s about to get a lot better too. Pgvector now supports multi-threaded build
https://github.com/pgvector/pgvector/issues/409#issuecomment...
pgvector has you covered: https://github.com/pgvector/pgvector
-
Vector Databases: A Technical Primer [pdf]
You don't need a dedicated vector db, you can use pgvector.
You could maybe use Cube for euclidean space search, but you're better off using optimized algorithms for embedding space search.
-
How Modern SQL Databases Are Changing Web Development - #4 Into the AI Era
PostgreSQL's pgvector extension is probably the most widely used SQL solution for storing and searching vector data today. The extension introduces a "vector" type specialized for storing high-dimensional vector data. It allows you to create vector indices (in "IVFFlat" or "HNSW" format for different indexing/searching performance tradeoffs) and leverage them to do various types of similarity searches.
What are some alternatives?
Milvus - A cloud-native vector database, storage for next generation AI applications
faiss - A library for efficient similarity search and clustering of dense vectors.
Elasticsearch - Free and Open, Distributed, RESTful Search Engine
qdrant - Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/
ann-benchmarks - Benchmarks of approximate nearest neighbor libraries in Python
jina - ☁️ Build multimodal AI applications with cloud-native stack
pinecone - Peer-to-peer overlay routing for the Matrix ecosystem
smlar - PostgreSQL extension for an effective similarity search || mirror of git://sigaev.ru/smlar.git || see https://www.pgcon.org/2012/schedule/track/Hacking/443.en.html
vespa - AI + Data, online. https://vespa.ai