beam VS Laravel

The "streaming systems" book answers your question and more: https://www.oreilly.com/library/view/streaming-systems/97814.... It gives you a history of how batch processing started with MapReduce, and how attempts at scaling by moving towards streaming systems gave us all the subsequent frameworks (Spark, Beam, etc.).

As for the framework called MapReduce, it isn't used much, but its descendant https://beam.apache.org very much is. Nowadays people often use "map reduce" as a shorthand for whatever batch processing system they're building on top of.

Apache Beam is one of many tools that you can use

Flexible runtime ☁️: Temporian programs can run seamlessly in-process in Python, on large datasets using Apache Beam.

To perform the tests I'm using a Kafka cluster on Kubernetes from the Beam repo (here).

Apache Beam: Streaming framework which can be run on several runner such as Apache Flink and GCP Dataflow

Apache Beam: Batch/streaming data processing 🔗Link

What you are looking for is Dataflow. It can be a bit tricky to wrap your head around at first, but I highly suggest leaning into this technology for most of your data engineering needs. It's based on the open source Apache Beam framework that originated at Google. We use an internal version of this system at Google for virtually all of our pipeline tasks, from a few GB, to Exabyte scale systems -- it can do it all.

That being said, there is a learning curve in understanding how Apache Beam works. Take a look at the beam website for more information.

Laravel is a popular PHP framework known for its expressive syntax and rich ecosystem of features. Here's why it shines for building RESTful APIs:

Implementation In this article, we'll delve into the concept of rate limiting in Laravel; a popular PHP framework. We will explore how to set it up, customize it to suit your application's needs, and handle common scenarios. By the end, you'll have the knowledge and confidence to implement rate limiting in your Laravel applications, enhancing their security and stability.

Delving into PHP frameworks like Laravel or Symfony is like building a skyscraper, with Composer acting as your "construction foreman," guiding you step by step to ensure your code is robust and awe-inspiring. This stage involves getting familiar with popular PHP frameworks such as Laravel, Symfony, CodeIgniter, etc., and utilizing the functionalities provided by these frameworks to rapidly develop efficient, maintainable web applications. Tools to consider: Laravel, Composer.

Your very first starting point should be the Laravel documentation. Known for its clear explanations and user-friendly layout, the Laravel documentation makes setup a breeze, ensuring you get off to the best possible start.

In this tutorial, we will learn how to create a GraphQL API with Laravel, a popular PHP web framework. We will be creating a simple student model, seeding the database with dummy data, setting up a database connection, and creating a GraphQL server by defining our API's schema, queries, and mutations. We’ll also learn how to make requests to our API (test our endpoints) using a tool like Insomnia or Postman. By the end of this tutorial, you will have a working GraphQL API that you can continue to expand and improve.

Laravel is an open-source PHP framework on GitHub with 75.7k stars and 24.2k forks used for building web applications. It was first released in 2011 and follows the Model-View-Controller (MVC) architecture. It comes with an expressive and elegant syntax. It is fine-tuned for building professional web applications and ready to handle enterprise workloads. It achieves this by integrating the best packages from the PHP ecosystem and creating a framework that is also developer-friendly. Other than its foundational features, It also provides tools for dependency injection, unit testing, real-time events, and many more.

For this project, we will use Laravel, one of the biggest, most popular and most complete frameworks for PHP, along with Livewire, which will help us make our notifications real-time and reactive. Let's create a project from scratch and implement authentication and notification features for users tagged in posts.

Learning one, or more, frameworks is a good way to get hired in the future. If you know, and like, PHP you could give Laravel a try. It's a complete backend framework (can also be used for frontend). And then learn a frontend framework as well. I guess React is the more popular choice, but if you try Laravel, then they recommend and pair well with Vue.