heed VS redb

At Meilisearch we are currently trying to add a better error handling in heed v0.20, our LMDB key-value store wrapper. Unfortunately, when there are a lot of generics it can become harder to play with…

There are issues and pull requests but I advise you to look at the milli folder in the Meilisearch repository, it’s where all the logic is done. We extensively use RoaringBitmaps, heed the LMDB wrapper and grenad when indexing.

I’ll continue to introduce new features and new safety guards until v0.20.0. Can you tell me more about your project? Or is it private?

LMDB support multiple readers and one writer at the same time. It is ensured by the library. Note that LMDB is a key-value store. You can use the heed library which is the most maintained Rust wrapper.

Note that heed is ensuring that you are not trying to use transactions, databases and environments in the right way. I have added much more work in that regard in the important update that I am working on too!

I am currently working on exposing the new LMDB encryption feature from heed the safe LMDB wrapper with the help of the Cryptography community.

Have you considered heed or even sanakirja?

rkv hasn't been updated for a while. I recommend using heed - https://docs.rs/heed

For wrappers around LMBD, I'd recommend RKV or Heed https://github.com/mozilla/rkv https://github.com/Kerollmops/heed

I wrote redb (https://github.com/cberner/redb) using mmap, initially. However, I later removed it and switched to read()/write() with my own user space cache. I'm sure it's not as good as the OS page cache, but the difference was only 1.2-1.5x performance on the benchmarks I cared about, and the cache is less than 500 lines of code.

exposure of the redb builder functionalities.

redb has reached its 1.0 release. The file format is now gauranteed to be backward compatible, and the API is stable. I've run pretty extensive fuzz testing, but please report any bugs you encounter.

I'm just now updating the benchmarks in the readme with multi-threaded workloads :) https://github.com/cberner/redb/pull/576

redb, but that doesn't have a stable file format yet,

I also have to support web, so I'm considering a hybrid approach with a direct IndexedDB API there and something like sqlite, percy, sanakirja, or redb on native.

Nice list, but don’t forget persy, redb and sanakirja too!

check out https://github.com/cberner/redb! it is inspired by LMDB and has comparable performance, but is entirely built in rust!

Yup, the language is immature in some domains like data replication, compilation time, lifetime expressivity, not having a default async runtime and the incompatibility of all of them. But many of these domains are worked on and improved every day this is why I love this language so much. This is because there is space for improvement. I also hope that one day we will have a pure-Rust LMDB alternative that I can contribute to without fear of C.

they are, ya! I'm quite excited about that, and already have a PR open to use GATs. It's not enough to fix this issue though, because std needs to add support for LendingIterator too