gramatika VS qdrant

I wrote a lexer generator. It's pretty limited and poorly architected tbh, but feel free to have a look: https://github.com/dannymcgee/gramatika

I'm a huge nerd for programming languages and rendering, and to that end I've been putting together a general-purpose parsing library inspired by syn and using that to power a language server for WGSL.

I'm probably in the minority on this one, but I really hate writing verbose, repetitive code, so I freaking love macros. I will frequently use a one-off macro just to make something like a dispatcher function easier to read by cutting down on all the pomp and circumstance. I'm also working on a small crate that makes heavy use of proc macros, which I've already gotten a ton of mileage out of since it allows me to spin up a serviceable lexer with just a few lines of code. A lot of people really dislike macros because the source is hard to read and they're onerous to debug. They're not wrong on either of those points.

Great. Now that we have the embeddings, we need to store them in a vector database. We will be using Qdrant for this purpose. Qdrant is an open-source vector database that allows you to store and query high-dimensional vectors. The easiest way to get started with the Qdrant database is using the docker.

I took Qdrant for this project. The reason was that Qdrant stands for high-performance vector search, the best choice against use cases like finding similar function calls based on semantic similarity. Qdrant is not only powerful but also scalable to support a variety of advanced search features that are greatly useful to nuanced caching mechanisms like ours.

I'm currently looking to implement locally, using QDrant [1] for instance.

I'm just playing around, but it makes sense to have a runnable example for our users at work too :) [2].

[1]. https://qdrant.tech/

Also compare with qdrant's Rust implementation; they tout their performance. https://github.com/qdrant/qdrant/tree/master/lib/segment/src...

There are much better known examples, such as https://qdrant.tech/ and https://github.com/lancedb/lancedb

For more information, see our release notes. Qdrant is an open source project. We welcome your contributions; raise issues, or contribute via pull requests!

Initialize the Qdrant Client with in-memory storage. The collection name will be “imagebind_data” and we will be using cosine distance.

Qdrant is an open-source vector search engine optimized for performance and flexibility. It supports both exact and approximate nearest neighbor search, providing a balance between accuracy and speed for various AI and ML applications.