patterns-devkit
Dataplane
patterns-devkit | Dataplane | |
---|---|---|
5 | 1 | |
106 | 184 | |
0.0% | 1.6% | |
2.9 | 8.3 | |
about 1 year ago | 4 months ago | |
Python | Go | |
BSD 3-clause "New" or "Revised" License | Business Source License 1.1 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
patterns-devkit
Dataplane
-
Airflow VS dataplane - a user suggested alternative
2 projects | 3 May 2022
Dataplane is an Airflow inspired data platform to automate, schedule and design data pipelines and workflows written in Golang.
What are some alternatives?
pyspark-example-project - Implementing best practices for PySpark ETL jobs and applications.
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
pipebird - Pipebird is open source infrastructure for securely sharing data with customers.
dagu - Yet another cron alternative with a Web UI, but with much more capabilities. It aims to solve greater problems.
SmartPipeline - A framework for rapid development of robust data pipelines following a simple design pattern
transfer - Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.
hamilton - Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.
data-engineering-wiki - The best place to learn data engineering. Built and maintained by the data engineering community.
AWS Data Wrangler - pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
JDR - Job Dependency Runner
flowrunner - Flowrunner is a lightweight package to organize and represent Data Engineering/Science workflows
dagster - An orchestration platform for the development, production, and observation of data assets.