Redis Re-Implemented with SQLite

• redka

  2 2,462 8.8 Go

  Redis re-implemented with SQLite

I've decided to use the max connections approach for now.

[1]: https://github.com/nalgeon/redka/blob/main/internal/sqlx/db....

• sqlite

  7 552 0.0 C

  Go SQLite3 driver (by crawshaw)

I’m not sure to what degree you want to follow the Redis no concurrency “everything serialized on one thread” model

You can get substantially better performance out of sqlite by using the lower level https://github.com/crawshaw/sqlite, turning on WAL etc, using a connection per goroutine for reads, and sending batches of writes over a buffered channel / queue to a dedicated writer thread. That way you can turn off SQLite’s built in per-connection mutex but still be thread safe since each connection is only used on a single thread at a time.

For this use-case you will also probably save a lot of time if you use some large arena-style circular buffers and copy incoming parameter bytes from the network request/socket to the buffer instead of allocating and passing around a bunch of individual strings.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• SSDB

  14 8,133 0.0 C++

  SSDB - A fast NoSQL database, an alternative to Redis

I've used SSDB[0] in the past for some really stupid large datasets (20TB)_and it worked really well in production

[0] https://github.com/ideawu/ssdb

• Sidekiq

  91 12,940 8.9 Ruby

  Simple, efficient background processing for Ruby

  

That depends on how the `maxmemory-policy` is configured, and queue systems based on Redis will tell you not to allow eviction. https://github.com/sidekiq/sidekiq/wiki/Using-Redis#memory (it even logs a warnings if it detects your Redis is misconfigured IIRC).

• kvrocks

  23 3,232 9.6 C++

  Apache Kvrocks is a distributed key value NoSQL database that uses RocksDB as storage engine and is compatible with Redis protocol.

  

I switched from SSDB to Kvrocks recently, because SSDB is abandoned and the author missing for 3 years now. I used to recommend SSDB, but now there's better alternatives available:

https://github.com/apache/kvrocks

https://github.com/sabledb-io/sabledb

• sabledb

  4 372 7.4 Rust

  Ultra fast, persistent database supporting Redis API

I switched from SSDB to Kvrocks recently, because SSDB is abandoned and the author missing for 3 years now. I used to recommend SSDB, but now there's better alternatives available:

https://github.com/apache/kvrocks

https://github.com/sabledb-io/sabledb

• s3ite

  1 12 5.4 Rust

In other abuses of SQLite, I wrote a tool [0] that exposes blobs in SQLite via an Amazon S3 API. It doesn't do expiry (but that would be easy enough to add if S3 does it).

We were using it to manage a millions of images for machine learning as many tools support S3 and the ability to add custom metadata to objects is useful (harder with files). It is one SQLite database per bucket but at the bucket level it is transactional.

0: https://github.com/seddonm1/s3ite

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• pocketbase

  177 33,169 9.7 Go

  Open Source realtime backend in 1 file

  

• tigerbeetle

  45 7,059 9.9 Zig

  The distributed financial transactions database designed for mission critical safety and performance.

  

I'm waiting for someone to implement the Redis API by swapping out the state machine in TigerBeetle (which was built modularly such that the state machine can be swapped out).

https://tigerbeetle.com/

• rq

  27 9,518 8.3 Python

  Simple job queues for Python

  

That's pretty cool. Reckon it would work with existing code that calls Redis over the wire for RQ?

  https://python-rq.org

• Memcached

  55 13,199 8.4 C

  memcached development tree

  

• redimo.go

  4 30 0.0 Go

  Use the power of DynamoDB with the ease of the Redis API

  

Did this for DynamoDB over the pandemic. Helped me learn both the Redis API and DynamoDB. https://github.com/dbProjectRED/redimo.go

Want to do the same thing with Postgres as well.

The Redis API and data structures are really nice and have been tuned to be useful in a wide variety of situations. Implementing these APIs is different concurrent, compute and storage models is a nice way to learn them and get options open.

• go-sqlite3

  39 7,446 6.3 C

  sqlite3 driver for go using database/sql

for what it's worth, the two pool approach is suggested here by a collaborator to github.com/mattn/go-sqlite3: https://github.com/mattn/go-sqlite3/issues/1179#issuecomment...

Suggest a related project