dragonfly VS tarantool

https://github.com/dragonflydb/dragonfly is another option. Not a fork but API-compatible reimplementation.

Our journey will involve leveraging the capabilities of Dragonfly, a highly efficient drop-in replacement for Redis, known for its ultra-high throughput and multi-threaded share-nothing architecture. Specifically, we'll be utilizing two of Dragonfly's data types: Sorted-Set and Hash. These data structures are perfect for handling real-time data and ranking systems, making them ideal for our leaderboards.

2023 has been a year with remarkable advancements in AI capabilities, and at Dragonfly, we are thrilled to power new use cases with our latest release: Dragonfly Search. This new feature set, debuting in Dragonfly v1.13, is a subset of RediSearch-compatible commands implemented natively in Dragonfly, allowing for both vector search and faceted search use cases in the highly scalable and performant Dragonfly in-memory data store.

If you have not heard about Dragonfly - please check it out. It uses - what I hope - novel and interesting ideas backed up by the research from recent years [1] and [2]. It's meant to fix many problems that exist with Redis today. I have been working on Dragonfly for the last 7 months and it has been one of the more interesting and challenging projects I've ever done!

I recently ran across the the license for Dragonfly [1] which has some restrictions (rights reserved), but 5 years after the license date the license switches to Apache 2.0. Basically a timed-limited rights reservation. I don't hate it. I might even contribute to such a project for free.

I would consider something like this: When I release code, it's rights reserved for 5 years, then open-source (and this baked into an irrevocable license). Anyone may use the software for non-commercial purposes. Anyone may contribute, those who contribute will be granted permission for commercial use if I deem their contributions significant enough. Anyone may distribute the software under these terms.

If such a model became popular, I have a hard time imagining it could make things any worse. It might even accelerate open-source development. You might say, "but it's not open-source", fair enough, but we can view it as open-source contribution with a delay. For example, if this model became wildely popular this year, and we saw great progress with this model, then come 2028 we would be flooded with new open-source software and ultimately might be better off than it would have been without this model.

(And this whole thing makes me rethink copyright and patents and how much they really contribute to society. Perhaps they should be shortened?)

[1]: https://github.com/dragonflydb/dragonfly/blob/main/LICENSE.m...

Maybe check out https://www.dragonflydb.io/ It claims to have a full redis implementation.

The article describes that the new JIT is a "copy-and-patch JIT" (I've previously heard this called a "splat JIT"). This is a relatively simple JIT architecture where you have essentially pre-compiled blobs of machine code for each interpreter instruction that you patch immediate arguments into by copying over them.

I once wrote an article about very simple JITs, and the first example in my article uses this style: https://blog.reverberate.org/2012/12/hello-jit-world-joy-of-...

I take some issue with this statement, made later in the article, about the pros/cons vs a "full" JIT:

> The big downside with a “full” JIT is that the process of compiling once into IL and then again into machine code is slow. Not only is it slow, but it is memory intensive.

I used to think this was true also, because my main exposure to JITs was the JVM, which is indeed memory-intensive and slow.

But then in 2013, a miraculous thing happened. LuaJIT 2.0 was released, and it was incredibly fast to JIT compile.

LuaJIT is undoubtedly a "full" JIT compiler. It uses SSA form and performs many optimizations (https://github.com/tarantool/tarantool/wiki/LuaJIT-Optimizat...). And yet feels no more heavyweight than an interpreter when you run it. It does not have any noticeable warm up time, unlike the JVM.

Ever since then, I've rejected the idea that JIT compilers have to be slow and heavyweight.

Then you should like Tarantool which has a built-in Lua (LuaJIT) application server.

LuaJIT provides Lua language support, including both the language execution environment and the JIT tracer compiler. Our LuaJIT has long differed from the vanilla version in a set of patches adding features, such as the profiler, and new tests. That is why we test our fork thoroughly to prevent regression. LuaJIT source code is open and distributed under a free license, but it does not include regression tests. Therefore, we have assembled our regression test suite from PUC-Rio Lua tests, test suite by François Perrad, tests for other LuaJIT forks, and of course, our own tests.

Starting from 2.10.0-beta Tarantool can natively run on M1 chips. So far this is preliminary support: something may crash or run unstable. We have resolved almost all bugs we knew about, with a few minor ones left. For example, there are some issues with the JIT compiler. But this didn't prevent the team product manager from installing Tarantool on his new MacBook Air and running it every day.

A similar project built with intention around this idea is Tarantool[1]. I never hear much about it, but if you're interested in compute close to your data, this is definitely something that would warrant consideration.

[1]: https://github.com/tarantool/tarantool