Apache Flink VS Akka

7. Apache Flink | Github | tutorial

I'm trying to use https://github.com/mauricioaniche/ck to analyze the ck metrics of https://github.com/apache/flink. I have the latest version of java downloaded and I have the latest version of apache maven downloaded too. My environment variables are set correctly. I'm in the correct directory as well. However, when I run mvn clean compile package in powershell it always says build error. I've tried looking up the errors but there's so many. https://imgur.com/a/Zk8Snsa I'm very new to programming in general so any suggestions would be appreciated.

Apache Drill, Druid, Flink, Hive, Kafka, Spark

It took me some time to get a good grasp of the power of SQL; and it really kicked in when I learned about optimization rules. It's a program that you rewrite, just like an optimizing compiler would.

You state what you want; you have different ways to fetch and match and massage data; and you can search through this space to produce a physical plan. Hopefully you used knowledge to weight parts to be optimized (table statistics, like Java's JIT would detect hot spots).

I find it fascinating to peer through database code to see what is going on. Lately, there's been new advances towards streaming databases, which bring a whole new design space. For example, now you have latency of individual new rows to optimize for, as opposed to batch it whole to optimize the latency of a dataset. Batch scanning will be benefit from better use of your CPU caches.

And maybe you could have a hybrid system which reads history from a log and aggregates in a batched manner, and then switches to another execution plan when it reaches the end of the log.

If you want to have a peek at that here are Flink's set of rules [1], generic and stream-specific ones. The names can be cryptic, but usually give a good sense of what is going on. For example: PushFilterIntoTableSourceScanRule makes the WHERE clause apply the earliest possible, to save some CPU/network bandwidth further down. PushPartitionIntoTableSourceScanRule tries to make a fan-out/shuffle happen the earliest possible, so that parallelism can be made use of.

[1] https://github.com/apache/flink/blob/5f8fb304fb5d68cdb0b3e3c...

[1]: https://mvnrepository.com/artifact/org.apache.avro/avro-maven-plugin/1.8.2 [2]: https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/sink/FileSink.java [3]: https://ci.apache.org/projects/flink/flink-docs-master/docs/connectors/datastream/file_sink/ [4]: https://github.com/apache/flink/blob/c81b831d5fe08d328251d91f4f255b1508a9feb4/flink-end-to-end-tests/flink-file-sink-test/src/main/java/FileSinkProgram.java [5]: https://github.com/rajcspsg/streaming-file-sink-demo

Kotlin also has a construct for asynchronous collections/streams. Kotlin's version of AsyncSequence is called a Flow. Just as Swift's AsyncSequence builds upon prior experience with RxSwift and Combine, Kotlin's Flow APIs build upon earlier stream/collection APIs in the JVM ecosystem: Java's RxJava, Java8 Streams, Project Reactor, and Scala's Akka.

First-class distributed and multicore computing. Swift has first-class “actors” and “distributed” methods. Unison, Erlang, and Elixir are built with distributed being one of the #1 concerns. Though first-class is not super common and I don't really expect it to be because usually libraries are enough (e.g. Scala has Akka and is used WIDELY for distributed); whereas something like linear types and typed effects, you can't emulate in a library.

Akka is a library that implements the actor model for JVM languages. Mainly in Scala, but you can use it in Java too, and maybe others. It doesn't feel as ergonomic as Elixir, but if Elixir is too "out there" for the decision makers in your case, this might be a friendlier alternative.

Note Akka, the Java & friends framework, is working with the actor model and have as main inspiration Erlang to mimic some features of the BEAM on top of the JVM.

If I hear about Actor model, the first thing that comes to my mind is the popular Open Source library Akka.

Lightbend, the company owning Akka, recently shared a blog post signed by the CEO announcing a license change from Apache 2.0 to Business Source License 1.1, a proprietary license. You can already find it in this PR, merged a couple days ago.

Akka 2.6 is on the open source Apache license, that is unchanged (its not possible for Lightbend to change an existing license). Its only the new Akka 2.7 which has the BSL license, so as long as you don't upgrade you are fine. See https://github.com/akka/akka/pull/31561.

Here is Akka's dependency specification: https://github.com/akka/akka/blob/main/project/Dependencies....

How many of these would have to opt for this source available approach before Akka would no longer be sustainable?

I don't think that the Akka change is aimed at cloud providers. Akka is a toolkit for building highly concurrent, distributed, and resilient message-driven applications for Java and Scala [1]. It's not possible for AWS to offer their own "Akka service" the way they offered Redis through ElastiCache. This change affects companies that build software using Akka, many of which may not be software-focused. For example, here's a case study of an oilfield services company that uses Akka:

https://www.lightbend.com/blog/low-cpu-utilization-fortune-5...

[1] As found on https://akka.io/