solr VS Weaviate

Using the Galaxy UI, knowledge workers can systematically review the best results from all configured services including Apache Solr, ChatGPT, Elastic, OpenSearch, PostgreSQL, Google BigQuery, plus generic HTTP/GET/POST with configurations for premium services like Google's Programmable Search Engine, Miro and Northern Light Research.

Apache Solr can be used to index and search text-based documents. It supports a wide range of file formats including PDFs, Microsoft Office documents, and plain text files. https://solr.apache.org/

Solr https://solr.apache.org/

if so, then https://solr.apache.org/ can be a solution, though there's a bit of setup involved. oh yea, you get to write your own "search interface" too which would end up calling solr's api to find stuff.

In this tutorial, I introduce Weaviate, an open-source vector database, with the thenlper/gte-base embedding model from Alibaba, through Hugging Face's transformers library.

This will be solved in Weaviate https://github.com/weaviate/weaviate/issues/2424

Link to GitHub -->

Hey @rvrs, I work on Weaviate and we are doing some improvements around increasing write throughput:

1. gRPC. Using gRPC to write vectors has had a really nice performance boost. It is released in Weaviate core but here is still some work on do on the clients. Feel free to get in contact if you would like to try it out.

2. Parameter tuning. lowering `efConstruction` can speed up imports.

3. We are also working on async indexing https://github.com/weaviate/weaviate/issues/3463 which will further speed things up.

In comparison with pgvector, Weaviate has more flexible query options such as hybrid search and quantization to save memory on larger datasets.