Our great sponsors
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
qdrant
Qdrant - High-performance, massive-scale Vector Database for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/
-
openraft
An implementation of the Raft distributed consensus protocol using the Tokio framework. (by drmingdrmer)
Yup, the language is immature in some domains like data replication, compilation time, lifetime expressivity, not having a default async runtime and the incompatibility of all of them. But many of these domains are worked on and improved every day this is why I love this language so much. This is because there is space for improvement. I also hope that one day we will have a pure-Rust LMDB alternative that I can contribute to without fear of C.
The Language support is mainly handled by the tokenizer where you can find specialized Segmenters and Normalizers that target some Language subtilities: https://github.com/meilisearch/charabia
We are building open source vector search engine in Rust, if you are interested - https://github.com/qdrant/qdrant
I advise you to fill out a discussion on our product repository for us to evaluate your needs, and use case and then see what we plan about that.
We plan to implement a replication system in a futur version of Meilisearch but as it is something quite comple to do we need time, the Rust ecosystem around Raft/Paxos is immature, we must contribute to it! I would say that the most advanced and maintained Raft solution is openraft at the moment, we will probably look into that.
LMDB is much more same in the sense that it supports real ACID transaction instead of savepoints for RocksDB. The latter is heavy and consumes a lot more memory for a lot less read throughput. However, RocksDB has a much better parallel and concurrent write story, where you can merge entries with merge functions and therefore write from multiple CPUs.