paradedb VS qdrant

Elasticsearch is good, and it does scale, but it is much more cumbersome and expensive to scale and operate than Postgres. If you use the managed service, you'll pay for the operational pain in the form of higher pricing.

The Postgres movement is strong and extensions like ParadeDB https://github.com/paradedb/paradedb are designed specifically to solve this pain point (Disclaimer: I work for ParadeDB)

No timeline yet, but we know it's a high-priority feature and are working hard on it. Best way would be to join our Slack (link here: https://github.com/paradedb/paradedb/blob/dev/README.md) to follow along. It will be in the coming weeks/months, though.

You're right. We're working on this currently. You can track the issue here: https://github.com/paradedb/paradedb/issues/717

There are definitely ways to cleanly make Postgres scale for analytics. We didn't discuss in this blog, but we will be writing about them in the future. For example, check out what the folks at ParadeDB are doing. https://github.com/paradedb/paradedb. Neon is doing an awesome job separating compute from storage. Supabase contributed foreign data wrappers make it super easy to read from S3 into Postgres. Lots of great work going out there :)

Check out https://github.com/paradedb/paradedb/tree/dev/pg_analytics, we're shipping this week

Shameless plug, but I'm one of the makers of `pg_bm25` (https://github.com/paradedb/paradedb). We're making a faster tsvector/tsrank as a Postgres extension. Maybe it can help, our benchmarks show much faster performance especially as row count increases

Great. Now that we have the embeddings, we need to store them in a vector database. We will be using Qdrant for this purpose. Qdrant is an open-source vector database that allows you to store and query high-dimensional vectors. The easiest way to get started with the Qdrant database is using the docker.

I took Qdrant for this project. The reason was that Qdrant stands for high-performance vector search, the best choice against use cases like finding similar function calls based on semantic similarity. Qdrant is not only powerful but also scalable to support a variety of advanced search features that are greatly useful to nuanced caching mechanisms like ours.

I'm currently looking to implement locally, using QDrant [1] for instance.

I'm just playing around, but it makes sense to have a runnable example for our users at work too :) [2].

[1]. https://qdrant.tech/

Also compare with qdrant's Rust implementation; they tout their performance. https://github.com/qdrant/qdrant/tree/master/lib/segment/src...

There are much better known examples, such as https://qdrant.tech/ and https://github.com/lancedb/lancedb

For more information, see our release notes. Qdrant is an open source project. We welcome your contributions; raise issues, or contribute via pull requests!

Initialize the Qdrant Client with in-memory storage. The collection name will be “imagebind_data” and we will be using cosine distance.

Qdrant is an open-source vector search engine optimized for performance and flexibility. It supports both exact and approximate nearest neighbor search, providing a balance between accuracy and speed for various AI and ML applications.