goleveldb VS bbolt

I wanted to get deeper insights into both indexeddb and leveldb, so decided to write an indexeddb wrapper around goleveldb.

https://github.com/syndtr/goleveldb for heavy-duty local data storage.

We're using go-leveldb for a reasonably high-load case here at my $dayjob.

I would used some embedded kv store like go-leveldb or bolt. Key is BigEnding timestamp + optional tail to allow duplicate timestamps.

I'm using the leveldb https://github.com/syndtr/goleveldb for the moment because I'm developing the architecture. Mh, leveldb support the only the inmem db? :/

For learning goroutines and channels, I usually recommend writing a program that reads from files and writes the data in a dummy database with something like https://github.com/etcd-io/bbolt. It's relatively simple and you're more likely to run into common manifestations of concurrency issues running disk operations.

If you would like to explore usage of channels, I highly recommend writing a program that reads from files and writes the data in a dummy database with something like https://github.com/etcd-io/bbolt.

No transactions, no consistency guarantees, no benchmarks, global locks in the storage implementation, a collection is copied in its entirety on every insertion to it...I realize it's not for the same use case as MySQL or MongoDB, but a more obvious comparison here is e.g. https://github.com/etcd-io/bbolt. So why should someone use this over bbolt?

use go-sqlite3 to work with sqlite3 is one choice.

https://github.com/etcd-io/bbolt is another pure go option.

cznic seems like an alternative to bbolt. nice to have some options.

This is the PR for the freelist improvement from that alibaba article: https://github.com/etcd-io/bbolt/pull/141

There is also an option to have a custom domain like go.etcd.io/bbolt

Remove sqlite or replace with bbolt

I would used some embedded kv store like go-leveldb or bolt. Key is BigEnding timestamp + optional tail to allow duplicate timestamps.

We initially went for using bolt (and later bbolt) which is the same store that powers etcd. It uses a B+tree approach. It worked great, but we're not happy with the write performance. We're currently in the process of switching to a custom LSM-tree-based approach, which is what you typically find in DBs with great write performance (e.g. Cassandra).

Debuggability: We also built etcd data storage analysis tool using boltdb library and flamegraph to visualize data storage breakdown. Page cache used by kube-apiserver is counted towards a cgroup’s memory limit, and anonymous memory usage can steal page cache usage for the same cgroup. (force cgroup to reclaim as many pages reclaimed as possible if memory usage is approaching limit.)