plugin-sdk
Dataplane
| plugin-sdk | Dataplane | |
|---|---|---|
| 1 | 1 | |
| 20 | 186 | |
| - | 2.7% | |
| 9.7 | 8.3 | |
| 4 days ago | 4 months ago | |
| Go | Go | |
| Mozilla Public License 2.0 | Business Source License 1.1 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
plugin-sdk
-
Building a High Performance Data Integration Framework in Go
Great question! We specifically started with connectors/plugins to cloud infrastructure providers and other infrastructure vendors - This is why most of our users migrated and/or compared us to the tools specified here (https://www.cloudquery.io/docs/cq-vs-others/overview).
There are no connectors for AWS, GCP, Azure in FiveTran, Airbyte, Meltano but as we extend the number of our source plugins I believe this question will come up more and we will add those products as well to our comparison list.
The protocol is just standard/plain gRPC (https://www.cloudquery.io/docs/developers/architecture). Protobuf is defined here - https://github.com/cloudquery/plugin-sdk/tree/main/internal/...
Basically just the normalized, verified data + metadata, types
Dataplane
-
Airflow VS dataplane - a user suggested alternative
2 projects | 3 May 2022
Dataplane is an Airflow inspired data platform to automate, schedule and design data pipelines and workflows written in Golang.
What are some alternatives?
Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
dagu - Yet another cron alternative with a Web UI, but with much more capabilities. It aims to solve greater problems.
transfer - Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.
data-engineering-wiki - The best place to learn data engineering. Built and maintained by the data engineering community.
JDR - Job Dependency Runner
dagster - An orchestration platform for the development, production, and observation of data assets.
starthinker - Reference framework for building data workflows provided by Google. Accelerates authentication, logging, scheduling, and deployment of solutions using GCP. To borrow a tagline.. "The framework for professionals with deadlines."
patterns-devkit - Data pipelines from re-usable components
amplify - Bacalhau Amplify: automatic enrichment, enhancement, and explanation of your data
GCPSteamAnalytics - API de fuente de datos eficiente para ELT Pipeline, creada con Go. Aproveche el poder de Go con esta API de alto rendimiento para una recuperación de datos perfecta, complementando Python y otras herramientas en su proceso ELT.
rtdl - rtdl makes it easy to build and maintain a real-time data lake