Weaviate VS awesome-vector-search

In this tutorial, I introduce Weaviate, an open-source vector database, with the thenlper/gte-base embedding model from Alibaba, through Hugging Face's transformers library.

This will be solved in Weaviate https://github.com/weaviate/weaviate/issues/2424

Link to GitHub -->

Hey @rvrs, I work on Weaviate and we are doing some improvements around increasing write throughput:

1. gRPC. Using gRPC to write vectors has had a really nice performance boost. It is released in Weaviate core but here is still some work on do on the clients. Feel free to get in contact if you would like to try it out.

2. Parameter tuning. lowering `efConstruction` can speed up imports.

3. We are also working on async indexing https://github.com/weaviate/weaviate/issues/3463 which will further speed things up.

In comparison with pgvector, Weaviate has more flexible query options such as hybrid search and quantization to save memory on larger datasets.

Probably. But there are a number of free open source ones. For example, I've got a document that I'm doing embedding-keys for that has about 8000 sentences. Here's a list of some [ https://github.com/currentslab/awesome-vector-search ]

note: search for vector databases not written in Go but with Go clients, in case there is anything more local/lightweight: https://github.com/currentslab/awesome-vector-search

https://github.com/currentslab/awesome-vector-search

I was surprised to see Elastic actually has ok support for some of this stuff, though it appears slower for most of the tasks.

Supporting sequence of vectors does seems like a fresh air to the vector search service. I have added marqo to the list of awesome vector search (disclosure: I am the maintainer of the list) to increase your exposure.

If you want a proper curated list of various libraries and standalone services of vector search engines, refer to this awesome GitHub repository by Currents API.