sdk-go VS flink-kubernetes-operator

Plus, from what I've tried myself, the whole WASM experience is great only when used in combination with TS/wasm-bindgen, in the other scenarios it's a lot of manual tedious memory moving code involved. Maybe when WIT gets more broadly adopted, and there will be bindgens available in most languages/engines, this will be different.

If you wanna check out the projects and see how it works: https://github.com/restatedev/sdk-shared-core/, https://github.com/restatedev/sdk-go/ and https://github.com/restatedev/sdk-typescript/

With the explosion of IoT devices and demand for instant insights, real-time analytics is no longer optional. Technologies like Apache Kafka, Apache Flink, and Redpanda are at the forefront of this movement. Learning these platforms will help you design systems that process streaming data efficiently.

To do so, we will use Kinesis Data Analytics to run an Apache Flink application. To enhance our development experience, we will use Studio notebooks for Kinesis Data Analytics that are powered by Apache Zeppelin.

Apache Flink is a distributed data processing framework for both batch and streaming processing. It can be used to develop event-driven applications; perform batch and streaming data analysis; and can be used to develop ETL data pipelines.

all operator hubs on openshift are backed by the actual projects operator, ie https://github.com/apache/flink-kubernetes-operator

Apache Flink: A more general-purpose stream processing framework known for its low latency and advanced windowing capabilities. https://flink.apache.org/

Restate is built as a sharded replicated state machine similar to how TiKV (https://tikv.org/), Kudu (https://kudu.apache.org/kudu.pdf) or CockroachDB (https://github.com/cockroachdb/cockroach) are designed. Instead of relying on a specific consensus implementation, we have decided to encapsulate this part into a virtual log (inspired by Delos https://www.usenix.org/system/files/osdi20-balakrishnan.pdf) since it makes it possible to tune the system more easily for different deployment scenarios (on-prem, cloud, cost-effective blob storage). Moreover, it allows for some other cool things like seamlessly moving from one log implementation to another. Apart from that the whole system design has been influenced by ideas from stream processing systems such as Apache Flink (https://flink.apache.org/), log storage systems such as LogDevice (https://logdevice.io/) and others.

We plan to publish a more detailed follow-up blog post where we explain why we developed a new stateful system, how we implemented it, and what the benefits are. Stay tuned!

I’ve recently started my journey with Apache Flink. As I learn certain concepts, I’d like to share them. One such "learning" is the expansion of array type columns in Flink SQL. Having used ksqlDB in a previous life, I was looking for functionality similar to the EXPLODE function to "flatten" a collection type column into a row per element of the collection. Because Flink SQL is ANSI compliant, it’s no surprise this is covered in the standard.

You should let the Apache Flink team know, they mention exactly-once processing on their home page (under "correctness guarantees") and in their list of features.

[0] https://flink.apache.org/

[1] https://flink.apache.org/what-is-flink/flink-applications/#b...

Data scientists often prefer Python for its simplicity and powerful libraries like Pandas or SciPy. However, many real-time data processing tools are Java-based. Take the example of Kafka, Flink, or Spark streaming. While these tools have their Python API/wrapper libraries, they introduce increased latency, and data scientists need to manage dependencies for both Python and JVM environments. For example, implementing a real-time anomaly detection model in Kafka Streams would require translating Python code into Java, slowing down pipeline performance, and requiring a complex initial setup.