Data Engineering with Scala: Mastering Real-Time Data Processing with Apache Flink and Google Pub/Sub

1. flink-scala-api

   1 2 84 9.3 Scala

   Flink Scala API is a thin wrapper on top of Flink Java API which support Scala Types for serialisation as well the latest Scala version

   

   Second, although Apache Flink has an official API for the Scala language, it has been deprecated and will be removed in future versions. You can learn more about this here. However, since Scala is a JVM-based language and Apache Flink is developed in Java, it is perfectly possible to still use the Scala language for development with Apache Flink, but using the Java APIs. Yes, I also turned my nose up at that. Nobody deserves that! But, to make our lives easier, we will use the Flink Scala API library, which is nothing less than a fork of the official Flink Scala API, but completely maintained by the community. I highly recommend this library!

2. CodeRabbit

   coderabbit.ai featured

   CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

   

3. scala-cli

   2 40 584 9.7 Scala

   Scala CLI is a command-line tool to interact with the Scala language. It lets you compile, run, test, and package your Scala code (and more!)

   

   First of all, the development of our data pipeline will not be based on SBT projects. We will use the Scala CLI, a command-line tool that allows compile, run, test and package Scala code. Based on the Scala CLI, we can develop Scala scripts in a very practical and fast way!

4. flink-kubernetes-operator

   3 17 853 9.0 Java

   Apache Flink Kubernetes Operator

   

   Apache Flink is a distributed data processing framework for both batch and streaming processing. It can be used to develop event-driven applications; perform batch and streaming data analysis; and can be used to develop ETL data pipelines.

Suggest a related project