Our great sponsors
-
consul
Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
Consul is much older than 4 years old (public availability in 2014; 1.0 release in 2017, with a lot of sites using 0.x in production long before). And the fact that they didn't encounter this pathological case until Q4 2021 tells us that they got a lot of life out of it until now. They also were planning to switch over to bbolt back in 2020[1].
The developers at Hashicorp are top-tier, and this doesn't substantially change their reputation in my eyes. Hindsight is always 20/20.
Let's end this thread; blaming doesn't help anyone.
It is an interesting read. Here's the pdf:
https://github.com/captn3m0/google-sre-ebook/releases/downlo...
This is the PR for the freelist improvement from that alibaba article: https://github.com/etcd-io/bbolt/pull/141
https://github.com/ledgerwatch/erigon/wiki/Criteria-for-tran...
For now this is (relatively) easy since bindings for GoLang, Rust NodeJS/Demo, etc are available and the API is mostly the same in general.
---
The ideas that MDBX uses to solve these issues are simple: zero-cost micro-defragmentation, coalescing short GC/freelist records, chunking too long GC/freelist records, LIFO for GC/freelist reclaiming, etc.
Many of the ideas mentioned seem simple to implement in Bold DB. However the complete solution is not documented and too complicated (in accordance with the traditions inherited from LMDB ;)