Apache ZooKeeper VS Akka

You can install Kafka from https://kafka.apache.org/quickstart. Because Druid and Kafka both use Apache Zookeeper, I opted to use the Zookeeper deployment that comes with Druid, so didn’t start it with Kafka. Once running, I created two topics for me to post the data into, and for Druid to ingest from:

Please note that we use Zookeeper as the Governance Center.

Next, review deployment complexity such as DB-less versus database-backed deployments. For example, Kong does require running Cassandra or Postgres. Apigee requires Cassandra, Zookeeper, and Postgres to run, while other solutions like Express Gateway and Tyk only require Redis. Apache APISIX uses etcd as its data store, it stores and manages routing-related and plugin-related configurations in etcd in the Data Plane.

ZooKeeper, a system for coordinating distributed nodes, similar to Google's Chubby

ElasticJob achieves distributed coordination through ZooKeeper. In practical scenarios, users may start multiple jobs in the same project simultaneously, all of which use the same Apache Curator client. There are certain risks due to the nature of ZooKeeper and the callback method of Curator in a single event thread.

For now we just need to know that Kafka will be running n brokers acting as a cluster. The Kafka cluster will use Zookeeper to orchestrate the brokers.

We spin up a Zookeeper instance at port 2181 internally within the Docker network (i.e. not accessible from the host)

To solve this issue we can use a distributed system manager such as Zookeeper which can provide distributed synchronization. Zookeeper can maintain multiple ranges for our servers.

Support the deployment of the ZooKeeper cluster through Bitnami.

Note Akka, the Java & friends framework, is working with the actor model and have as main inspiration Erlang to mimic some features of the BEAM on top of the JVM.

If I hear about Actor model, the first thing that comes to my mind is the popular Open Source library Akka.

Lightbend, the company owning Akka, recently shared a blog post signed by the CEO announcing a license change from Apache 2.0 to Business Source License 1.1, a proprietary license. You can already find it in this PR, merged a couple days ago.

Akka 2.6 is on the open source Apache license, that is unchanged (its not possible for Lightbend to change an existing license). Its only the new Akka 2.7 which has the BSL license, so as long as you don't upgrade you are fine. See https://github.com/akka/akka/pull/31561.

Here is Akka's dependency specification: https://github.com/akka/akka/blob/main/project/Dependencies....

How many of these would have to opt for this source available approach before Akka would no longer be sustainable?

I don't think that the Akka change is aimed at cloud providers. Akka is a toolkit for building highly concurrent, distributed, and resilient message-driven applications for Java and Scala [1]. It's not possible for AWS to offer their own "Akka service" the way they offered Redis through ElastiCache. This change affects companies that build software using Akka, many of which may not be software-focused. For example, here's a case study of an oilfield services company that uses Akka:

https://www.lightbend.com/blog/low-cpu-utilization-fortune-5...

[1] As found on https://akka.io/

> So Amazon can take free code, sell it as a slickly-packaged service, and not need to share anything.

Right, because they didn't change it. So what would there be to share anyway? 'Go to https://github.com/akka/akka'?

> but it does mean that at least every user has access to the code to make their "Akka Service" as good as Amazon's.

That code is meaningless. In this scenario we already established that Amazon is selling an unmodified copy of Akka as a Service (which, by the way, we will revisit this point later). And we already established, in your own words, that it's not about the code:

> Their secret sauce is "we have a Borg Cube's worth of datacenters, so we can run the same code as you at a lower price".

So what would the SSPL require, they publish their deployment scripts to their cloud datacentres? Like I said, that's meaningless.

> "Akka Service"

Final point. There's no such thing as an 'Akka Service'. Akka is a bunch of libraries. You link it into your JVM application. You don't run Akka separately, so by definition you can't offer it as a service (even if you're Amazon).

Hence my suggestion of AGPL, and heck let's be generous, let's add a commercial use exception for businesses with less than $20m revenue, similar to what they're doing now with the BSL. Except with my suggestion the software would stay as strong copyleft, most businesses would be happy with the commercial use exception, and a few would have to pay their fair share.

[1] https://news.ycombinator.com/item?id=32751373

However, if you really expect a lot of users (especially concurrent ones) to use your API, you should delve into the world of reactive programming. Use tools like RxJS (JS/TS) or Project Reactor (Java) in such a case, preferably in combination with a broker like Kafka or RabbitMQ. R2DBC is also great for a data store. Then of course if you want to go one step further, there also exists the less popular but very interesting Actor model which Akka easily has ported into Java.

What you are describing is more or less the actor programming model. (If you’ve heard of Akka, then you’ve heard of the actor model.) Loom is an enabling technology for implementing the actor model on the (vanilla) JVM, and a lot of people are looking forward to that, including me! So that’s a perfectly valid approach.