bookkeeper
beam
Our great sponsors
bookkeeper | beam | |
---|---|---|
3 | 30 | |
1,844 | 7,445 | |
0.9% | 1.1% | |
9.1 | 10.0 | |
1 day ago | 4 days ago | |
Java | Java | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
bookkeeper
-
Apache Pulsar vs Apache Kafka - How to choose a data streaming platform
Is it possible to store data within Kafka and Pulsar? The answer is yes, both systems offer long-term storage solutions, but their underlying implementations differ widely. While Kafka uses logs that are distributed among brokers, Pulsar uses Apache BookKeeper for storage.
-
Bring Streaming to Apache Cassandra with Apache Pulsar
Apache BookKeeper®: An open-source storage service that handles persistent storage of messages.
beam
-
Ask HN: Does (or why does) anyone use MapReduce anymore?
The "streaming systems" book answers your question and more: https://www.oreilly.com/library/view/streaming-systems/97814.... It gives you a history of how batch processing started with MapReduce, and how attempts at scaling by moving towards streaming systems gave us all the subsequent frameworks (Spark, Beam, etc.).
As for the framework called MapReduce, it isn't used much, but its descendant https://beam.apache.org very much is. Nowadays people often use "map reduce" as a shorthand for whatever batch processing system they're building on top of.
-
beam VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
Releasing Temporian, a Python library for processing temporal data, built together with Google
Flexible runtime ☁️: Temporian programs can run seamlessly in-process in Python, on large datasets using Apache Beam.
-
Real Time Data Infra Stack
Apache Beam: Streaming framework which can be run on several runner such as Apache Flink and GCP Dataflow
-
Google Cloud Reference
Apache Beam: Batch/streaming data processing 🔗Link
-
Apache Beam Moved From Jira to GitHub Issues - You Can Too!
The Apache Beam community recently migrated to GitHub Issues after years of using Jira as our issue tracker. This post details why we made the move, how we did it, and how to decide if migrating is appropriate for your project.
-
A trick to have arbitrary infix operators in Python
Apache Beam works this way. Code sample: https://github.com/apache/beam/blob/master/examples/multi-la...
-
Jinja2 not formatting my text correctly. Any advice?
ListItem(name='Apache Beam', website='https://beam.apache.org/', category='Batch Processing', short_description='Apache Beam is an open source unified programming model to define and execute data processing pipelines, including ETL, batch and stream processing'),
-
The Data Engineer Roadmap 🗺
Apache Beam
-
Frameworks of the Future?
I asked a similar question in a different community, and the closest they came up with was the niche Apache Beam and the obligatory vague hand-waving about no-code systems. So, maybe DEV seeming to skew younger and more deliberately technical might get a better view of things? Is anybody using a "Framework of the Future" that we should know about?
What are some alternatives?
Apache Arrow - Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
Apache Hadoop - Apache Hadoop
Scio - A Scala API for Apache Beam and Google Cloud Dataflow.
Apache Spark - Apache Spark - A unified analytics engine for large-scale data processing
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Apache Hive - Apache Hive
Apache Accumulo - Apache Accumulo
Trino - Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
Apache ZooKeeper - Apache ZooKeeper
Apache HBase - Apache HBase
Ruby on Rails - Ruby on Rails
data-engineer-roadmap - Roadmap to becoming a data engineer in 2021