Top 23 Key-Value Open-Source Projects

• Redis

  318 64,821 9.7 C

  Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes, Streams, HyperLogLogs, Bitmaps.

  

Project mention: Valkey Is Rapidly Overtaking Redis | news.ycombinator.com | 2024-04-19

One of the challenges Redis labs here have is that there's very little reason for their userbase to stay loyal to them.

antirez retired from Redis development a few years ago.

From https://github.com/redis/redis/graphs/contributors it looks like activity since he left has been mostly from people who didn't overlap with him much.

Redis Labs have not shown themselves to be outstanding stewards of the project as far as I can tell. Why shouldn't people support the fork?

• etcd

  61 46,345 9.9 Go

  Distributed reliable key-value store for the most critical data of a distributed system

  

Project mention: Oracle Linux 8.8'de PostgreSQL 13 Yedekli Yapı Nasıl Kurulur? - Patroni, ETCD, HAProxy | dev.to | 2023-12-07

sudo dnf -y install curl wget vim ETCD_RELEASE=$(curl -s https://api.github.com/repos/etcd-io/etcd/releases/latest|grep tag_name | cut -d '"' -f 4) echo $ETCD_RELEASE wget https://github.com/etcd-io/etcd/releases/download/${ETCD_RELEASE}/etcd-${ETCD_RELEASE}-linux-amd64.tar.gz tar xvf etcd-${ETCD_RELEASE}-linux-amd64.tar.gz cd etcd-${ETCD_RELEASE}-linux-amd64 sudo mv etcd* /usr/local/bin ls /usr/local/bin /usr/local/bin/etcd --version

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• MMKV

  5 16,821 8.7 C++

  An efficient, small mobile key-value storage framework developed by WeChat. Works on Android, iOS, macOS, Windows, and POSIX.

  

Project mention: Understanding security in React Native applications | dev.to | 2024-04-03

react-native-mmkv is a wrapper around MMKV that allows you to easily implement secure storage in your app. It is arguably the fastest key-value storage for React Native apps

• tikv

  21 14,512 9.7 Rust

  Distributed transactional key-value database, originally created to complement TiDB

  

Project mention: just wanted to ask is there an in memory database that uses s3 or gcp cloud storage as permanent storage | /r/Database | 2023-07-04

I know that very similar functionality to this is in TiDB Serverless ( https://tidbcloud.com ). TiDB is a distributed relational database. It uses TiKV ( which is a key/value engine ) as the storage engine. You could use SQL to access your K/V records. There is ongoing work in TiKV to support S3 directly as the storage backend ( https://github.com/tikv/tikv/issues/6506 ) .

• badger

  30 13,367 6.7 Go

  Fast key-value DB in Go.

  

Project mention: Anytype helper crashed | /r/Anytype | 2023-12-09

github.com/dgraph-io/badger/v3/table.OpenTable(0xc000bb4000, {0x0, 0x1, 0x200000, 0x0, 0x0, 0x3f847ae147ae147b, 0x1000, 0x0, 0x0, ...})

• ArangoDB

  17 13,340 9.9 C++

  🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.

  

Project mention: Ask HN: When is pure functional programming beneficial? | news.ycombinator.com | 2023-07-11

... or working in an environment or on a problem for which functional patterns apply.

Suppose you are writing a "CRUD" app that writes to a relational database, how do you apply functional programming to that? The whole point of an application like that is that it makes side effects.

In some cases you can break those problems down into functional pieces. Consider Python drivers for a product like

https://www.arangodb.com/

One major problem is that you want drivers that work synchronously and asynchronously, the structure of the average api call is something like

   def query(parameters):

• valkey

  7 12,754 8.3 C

  A new project to resume development on the formerly open-source Redis project. We're calling it Valkey, since it's a twist on the key-value datastore.

  

Project mention: Valkey Is Rapidly Overtaking Redis | news.ycombinator.com | 2024-04-19

Changelog line items is probably a better measure (assuming the line items are aligned to features and bugfixes and not just a list of PRs) https://github.com/valkey-io/valkey/releases

Maybe version number/release cadence is also helpful.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• Apache ZooKeeper

  36 11,925 8.3 Java

  Apache ZooKeeper

  

Project mention: On Implementation of Distributed Protocols | dev.to | 2024-04-05

Apache ZooKeeper — a distributed coordination, synchronization, and configuration service (written in Java);

• garnet

  4 9,063 9.3 C#

  Garnet is a remote cache-store from Microsoft Research that offers strong performance (throughput and latency), scalability, storage, recovery, cluster sharding, key migration, and replication features. Garnet can work with existing Redis clients.

  

Project mention: A MySQL compatible database engine written in pure Go | news.ycombinator.com | 2024-04-09

You would be surprised by performance of modern .NET :)

Writing no-alloc is oftentimes done by reducing complexity and not doing "stupid" tricks that actually work against JIT and CoreLib features.

For databases specifically, .NET is actually positioned very well with its low-level features (intrisics incl. SIMD, FFI, struct generics though not entirely low-level) and high-throughput GC.

Interesting example of this applied in practice is Garnet[0]/FASTER[1]. Keep in mind that its codebase still consist of un-idiomatic C# and you can do way better by further simplification, but it already does the job well enough.

[0] https://github.com/microsoft/garnet

[1] https://github.com/microsoft/FASTER

• immudb

  52 8,486 9.5 Go

  immudb - immutable database based on zero trust, SQL/Key-Value/Document model, tamperproof, data change history

  

Project mention: Ask HN: What is your experience of tamper proof systems? | news.ycombinator.com | 2024-01-05

• buntdb

  7 4,381 0.0 Go

  BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support

Project mention: PostgreSQL: No More Vacuum, No More Bloat | news.ycombinator.com | 2023-07-15

Experimental format to help readability of a long rant:

1.

According to the OP, there's a "terrifying tale of VACUUM in PostgreSQL," dating back to "a historical artifact that traces its roots back to the Berkeley Postgres project." (1986?)

2.

Maybe the whole idea of "use X, it has been battle-tested for [TIME], is robust, all the bugs have been and keep being fixed," etc., should not really be that attractive or realistic for at least a large subset of projects.

3.

In the case of Postgres, on top of piles of "historic code" and cruft, there's the fact that each user of Postgres installs and runs a huge software artifact with hundreds or even thousands of features and dependencies, of which every particular user may only use a tiny subset.

4.

In Kleppmann's DDOA [1], after explaining why the declarative SQL language is "better," he writes: "in databases, declarative query languages like SQL turned out to be much better than imperative query APIs." I find this footnote to the paragraph a bit ironic: "IMS and CODASYL both used imperative query APIs. Applications typically used COBOL code to iterate over records in the database, one record at a time." So, SQL was better than CODASYL and COBOL in a number of ways... big surprise?

Postgres' own PL/pgSQL [2] is a language that (I imagine) most people would rather NOT use: hence a bunch of alternatives, including PL/v8, on its own a huge mass of additional complexity. SQL is definitely "COBOLESQUE" itself.

5.

Could we come up with something more minimal than SQL and looking less like COBOL? (Hopefully also getting rid of ORMs in the process). Also, I have found inspiring to see some people creating databases for themselves. Perhaps not a bad idea for small applications? For instance, I found BuntDB [3], which the developer seems to be using to run his own business [4]. Also, HYTRADBOI? :-) [5].

6.

A usual objection to use anything other than a stablished relational DB is "creating a database is too difficult for the average programmer." How about debugging PostgreSQL issues, developing new storage engines for it, or even building expertise on how to set up the instances properly and keep it alive and performant? Is that easier?

I personally feel more capable of implementing a small, well-tested, problem-specific, small implementation of a B-Tree than learning how to develop Postgres extensions, become an expert in its configuration and internals, or debug its many issues.

Another common opinion is "SQL is easy to use for non-programmers." But every person that knows SQL had to learn it somehow. I'm 100% confident that anyone able to learn SQL should be able to learn a simple, domain-specific, programming language designed for querying DBs. And how many of these people that are not able to program imperatively would be able to read a SQL EXPLAIN output and fix deficient queries? If they can, that supports even more the idea that they should be able to learn something different than SQL.

----

1: https://dataintensive.net/

2: https://www.postgresql.org/docs/7.3/plpgsql-examples.html

3: https://github.com/tidwall/buntdb

4: https://tile38.com/

5: https://www.hytradboi.com/

• rosedb

  3 4,345 8.1 Go

  Lightweight, fast and reliable key/value storage engine based on Bitcask.

  

Project mention: rosedb: A Lightweight Key/Value Storage Engine in Go | /r/golang | 2023-06-30

• dynomite

  3 4,161 0.0 C

  A generic dynamo implementation for different k-v storage engines

  

• Riak

  1 3,902 2.3 Shell

  Riak is a decentralized datastore from Basho Technologies.

  

Project mention: Ask HN: Good examples of fault-tolerant Erlang code? | news.ycombinator.com | 2023-12-28

Step zero is definitely the OTP Design Principles doc (part of the OTP distribution):

https://www.erlang.org/doc/design_principles/users_guide

There are some good texts that have more examples:

Erlang & OTP in Action - https://www.manning.com/books/erlang-and-otp-in-action

Designing for Scalability with Erlang/OTP - https://www.oreilly.com/library/view/designing-for-scalabili...

One big example of distributed Erlang is Riak:

https://github.com/basho/riak

• Hive

  10 3,874 5.0 Dart

  Lightweight and blazing fast key-value database written in pure Dart. (by isar)

  

• nutsdb

  4 3,290 9.0 Go

  A simple, fast, embeddable, persistent key/value store written in pure Go. It supports fully serializable transactions and many data structures such as list, set, sorted set.

  

• ImmortalDB

  1 3,046 10.0 JavaScript

  :nut_and_bolt: A relentless key-value store for the browser.

• Olric

  28 3,005 5.7 Go

  Distributed in-memory object store. It can be used as an embedded Go library and a language-independent service.

Project mention: Olric: Distributed, embeddable in-memory data structures in Go | news.ycombinator.com | 2024-02-05

• Keyv

  1 2,481 8.8 TypeScript

  Simple key-value storage with support for multiple backends

• redka

  2 2,138 8.8 Go

  Redis re-implemented with SQLite

Project mention: Redis Re-Implemented with SQLite | news.ycombinator.com | 2024-04-14

I've decided to use the max connections approach for now.

[1]: https://github.com/nalgeon/redka/blob/main/internal/sqlx/db....

• FlashDB

  24 1,619 7.8 C

  An ultra-lightweight database that supports key-value and time series data | 一款支持 KV 数据和时序数据的超轻量级数据库

Project mention: FlashDB: NEW Data - star count:1406.0 | /r/algoprojects | 2023-10-20

• ejdb

  3 1,427 5.6 C

  :snowboarder: EJDB2 — Embeddable JSON Database engine C library. Simple XPath like query language (JQL).

  

• kv.js

  12 1,397 5.8 JavaScript

  ⚡️ Advanced in-memory caching for JavaScript.

  

Project mention: My first open-source project needs only 2 more stars to hit 1,000 stars! | /r/opensource | 2023-05-01

• SaaSHub

  www.saashub.com sponsored

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Key-Value related posts

• Valkey Is Rapidly Overtaking Redis
  4 projects | news.ycombinator.com | 19 Apr 2024
• Redis Is Forked
  1 project | news.ycombinator.com | 16 Apr 2024
• Handling Multiple requests with Redis and Bullmq
  1 project | dev.to | 13 Apr 2024
• New Redis Inc logo and branding [video]
  1 project | news.ycombinator.com | 10 Apr 2024
• Redis is not "open core" (2021)
  1 project | news.ycombinator.com | 10 Apr 2024
• Redict 7.3.0, a copyleft fork of Redis, is now available
  4 projects | news.ycombinator.com | 3 Apr 2024
• Redis' license change and forking are a mess that everybody can feel bad about
  1 project | news.ycombinator.com | 2 Apr 2024
• A note from our sponsor - SaaSHub
  www.saashub.com | 26 Apr 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com