redb VS nebari

I wrote redb (https://github.com/cberner/redb) using mmap, initially. However, I later removed it and switched to read()/write() with my own user space cache. I'm sure it's not as good as the OS page cache, but the difference was only 1.2-1.5x performance on the benchmarks I cared about, and the cache is less than 500 lines of code.

exposure of the redb builder functionalities.

redb has reached its 1.0 release. The file format is now gauranteed to be backward compatible, and the API is stable. I've run pretty extensive fuzz testing, but please report any bugs you encounter.

I'm just now updating the benchmarks in the readme with multi-threaded workloads :) https://github.com/cberner/redb/pull/576

redb, but that doesn't have a stable file format yet,

I also have to support web, so I'm considering a hybrid approach with a direct IndexedDB API there and something like sqlite, percy, sanakirja, or redb on native.

Nice list, but don’t forget persy, redb and sanakirja too!

check out https://github.com/cberner/redb! it is inspired by LMDB and has comparable performance, but is entirely built in rust!

Yup, the language is immature in some domains like data replication, compilation time, lifetime expressivity, not having a default async runtime and the incompatibility of all of them. But many of these domains are worked on and improved every day this is why I love this language so much. This is because there is space for improvement. I also hope that one day we will have a pure-Rust LMDB alternative that I can contribute to without fear of C.

they are, ya! I'm quite excited about that, and already have a PR open to use GATs. It's not enough to fix this issue though, because std needs to add support for LendingIterator too

Or, you could also look at Nebari, which underlies BonsaiDB https://github.com/khonsulabs/nebari/

There is Nebari which is the KV part of BonsaiDB I've used both successfully (and that is currently in production)

This looks like it could have been inspired by BonsaiDB's recent problems. I think the authors are now trying to achieve something similar to your project with a new storage layer called Sediment (which will become a part of Nebari if successful).

I ran my benchmark suite on my 2017 i7 MacBook Pro (Filevault enabled), and I was able to perform a random "get" operation out of a 1 million record dataset in an average of 18us (microseconds).

I haven't spent much time documenting the low-level structures outside of the code itself.

Nebari, the underlying storage layer, also has its own coverage report.

Quite well, but Sled doesn't support the same feature set. I've benchmarked our low-level storage layer against Sled in its own repostiory: Nebari Benchmark README. There are two links two criterion reports on that page.

I maintain several large codebases that have #![forbid(unsafe)] annotations, which prevent unsafe code from being written in those codebases directly. BonsaiDb clocks in at just shy of 30k LOC, and depends on Nebari which is another 12k LOC. Those two crates make up the bulk of a networked database implementation.

I have, and there's a set of microbenchmarks in the repository. There are two links to reports on that page -- one that's executed on GitHub Actions, which can have quite widely varying performance, and one that's executed on a "dedicated" VPS instance from Scaleway -- exact specs are on that page. I've kicked off another run on that VPS, since it hasn't been run since October. It takes a while to run, so if it still says October, try hitting refresh a little later.

I replaced Sled in BonsaiDb with a new storage layer I wrote: Nebari. I just wrote a post this morning recalling my struggles over the last month to feel comfortable sharing this project outside of a smaller group of people.