Redis VS LevelDB

Redis

Since 2008 I have been an official Debian Developer and even the Debian Project Leader from 2017—2019. I have submitted over 4,000 bugs to the project, been part of several core teams and have released over 250 security updates as well (as part of the Debian Long Term Support initiative). Today, I am the release manager of the Lintian static analysis tool but I also maintain a number of popular packages such as Redis, Django and Memcached.

In addition, you need to have Redis installed and running on your machine to be able to use subscribers. So, if you do not have it installed and you want to follow along with the tutorial you should go ahead and install it.

Are your queries too slow? Do you have millions of users? To improve query performance, we can setup bloom filters with redis to reduce load the DB.

You can still use it as a whole and you will get a great development and user experience. The backend is built on Node.js, Express, and by default SQLite when you first install the server with the option to use PostgreSQL and Redis.

They've been pushing for things like this for years, but the George Floyd riots last year really emboldened this crowd. The Wokeness we've been seeing infecting all levels of society since 2014 went into absolute overdrive as every company tried to outdo the other with virtue signaling.

In addition to coding challenges, technical interviews can include architecture design. This interview tests the ability of the developer to design something from a technical perspective. For example, design a key-value store like Redis.

Redis module for graph - RedisGraph. RedisGraph uses Cypher for its query language.

Ah, fair enough, that's true. It's written to the AOF before processing the next query (https://github.com/redis/redis/blob/unstable/src/server.c#L4...), so I believe you get read-your-writes consistency per node, but I suppose it is technically written afterwards rather than before.

Redis

Here you go: https://github.com/spesmilo/electrumx/ You'll need python 3.7 or later, and LevelDB https://github.com/google/leveldb Both available for Windows so it should work fine :)

looking at https://github.com/google/leveldb it seems that leveldb will do a lot of disk i/o, which is something that involves the kernel. this appears to point towards some kind of LTO optimization maybe in the filesystem driver, or the actual passing around of the data to be written... something like that.

What about leveldb? Its a key-value store that has some python bindings as well. Might be something if you were gonna go with Mongo.

If you just do an occasional key/value lookup, you don't need 1.2 MiB of WebAssembly. [1]

I'd solve it via sharding: divide the database into N pieces via range- or mod-sharding. [1] Choose an N that's large enough for each piece to be reasonably small. When you look up a key, fetch the shard of interest.

You can put each piece into separate files (a little simpler to code, and most static servers will use pre-gzipped files for "Content-Encoding: gzip requests" easily, but you waste more disk space due to internal fragmentation) or one file (with range serving and an index of the byte range offset for each piece).

The format for each piece can be anything, eg json (simple) or an sstable-like format (more efficient). [3]

[1] Content-Length of https://phiresky.github.io/youtube-sponsorship-stats/sql-was...

[2] mod-sharding means: piece[i] has all the keys where hash(key) % N = i.

[3] https://github.com/google/leveldb/blob/master/doc/table_form...

Born off of that, came LevelDB (2011) by the same authors which uses LSM Tree as its underlying data structure. This gave rise to pluggable storage engines and embedded databases. Around the same time in 2007, Amazon came up with DynamoDB which uses the same underlying LSM Tree structure along with a masterless distributed database design.

To avoid some confusion, this is only the "distributed" part. The actual store is handled by the https://github.com/google/leveldb library.