-
I've decided to use the max connections approach for now.
[1]: https://github.com/nalgeon/redka/blob/main/internal/sqlx/db....
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
I’m not sure to what degree you want to follow the Redis no concurrency “everything serialized on one thread” model
You can get substantially better performance out of sqlite by using the lower level https://github.com/crawshaw/sqlite, turning on WAL etc, using a connection per goroutine for reads, and sending batches of writes over a buffered channel / queue to a dedicated writer thread. That way you can turn off SQLite’s built in per-connection mutex but still be thread safe since each connection is only used on a single thread at a time.
For this use-case you will also probably save a lot of time if you use some large arena-style circular buffers and copy incoming parameter bytes from the network request/socket to the buffer instead of allocating and passing around a bunch of individual strings.
-
I've used SSDB[0] in the past for some really stupid large datasets (20TB)_and it worked really well in production
[0] https://github.com/ideawu/ssdb
-
That depends on how the `maxmemory-policy` is configured, and queue systems based on Redis will tell you not to allow eviction. https://github.com/sidekiq/sidekiq/wiki/Using-Redis#memory (it even logs a warnings if it detects your Redis is misconfigured IIRC).
-
kvrocks
Apache Kvrocks is a distributed key value NoSQL database that uses RocksDB as storage engine and is compatible with Redis protocol.
I switched from SSDB to Kvrocks recently, because SSDB is abandoned and the author missing for 3 years now. I used to recommend SSDB, but now there's better alternatives available:
https://github.com/apache/kvrocks
https://github.com/sabledb-io/sabledb
-
I switched from SSDB to Kvrocks recently, because SSDB is abandoned and the author missing for 3 years now. I used to recommend SSDB, but now there's better alternatives available:
https://github.com/apache/kvrocks
https://github.com/sabledb-io/sabledb
-
In other abuses of SQLite, I wrote a tool [0] that exposes blobs in SQLite via an Amazon S3 API. It doesn't do expiry (but that would be easy enough to add if S3 does it).
We were using it to manage a millions of images for machine learning as many tools support S3 and the ability to add custom metadata to objects is useful (harder with files). It is one SQLite database per bucket but at the bucket level it is transactional.
0: https://github.com/seddonm1/s3ite
-
-
tigerbeetle
The financial transactions database designed for mission critical safety and performance.
I'm waiting for someone to implement the Redis API by swapping out the state machine in TigerBeetle (which was built modularly such that the state machine can be swapped out).
https://tigerbeetle.com/
-
That's pretty cool. Reckon it would work with existing code that calls Redis over the wire for RQ?
https://python-rq.org -
-
Did this for DynamoDB over the pandemic. Helped me learn both the Redis API and DynamoDB. https://github.com/dbProjectRED/redimo.go
Want to do the same thing with Postgres as well.
The Redis API and data structures are really nice and have been tuned to be useful in a wide variety of situations. Implementing these APIs is different concurrent, compute and storage models is a nice way to learn them and get options open.
-
for what it's worth, the two pool approach is suggested here by a collaborator to github.com/mattn/go-sqlite3: https://github.com/mattn/go-sqlite3/issues/1179#issuecomment...
Related posts
-
Go queue v1.7.0: a new Delay queue for timers, retries, and TTLs
-
Go full stack web app tutorial with sqlc and htmx. Part 1
-
Build a Blazing-Fast TCP Server in Go: A Practical Guide
-
I benchmarked nine Go SQLite drivers and here are the results
-
Ask HN: I'm interested in something similar to Rails' Solid Queue for Go