datajob
stepfunctions2processing
Our great sponsors
datajob | stepfunctions2processing | |
---|---|---|
4 | 2 | |
107 | 121 | |
- | - | |
0.0 | 0.0 | |
about 1 year ago | over 1 year ago | |
Python | Python | |
Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
datajob
- Build and deploy a serverless data pipeline on AWS with no effort.
-
Datajob: Build and deploy a serverless data pipeline on AWS with no effort.
Thanks! triggering a pipeline run based on a schedule is one of the ideas to implement next: https://github.com/vincentclaes/datajob#ideas
-
Aws Glue Environmentsdlc
I created an open source library called `datajob` to deploy and orchestrate glue jobs. You can find it on github https://github.com/vincentclaes/datajob and on pypi
stepfunctions2processing
-
Using the Serverless framework to deploy hybrid serverless/cluster workflows
git clone [https://github.com/ryfeus/stepfunctions2processing.git](https://github.com/ryfeus/stepfunctions2processing.git) cd docker docker build -t stepfunctiontest . $(aws ecr get-login --no-include-email --region us-east-1) aws ecr create-repository --repository-name stepfunctiontest docker tag stepfunctiontest:latest .dkr.ecr.us-east-1.amazonaws.com/stepfunctiontest:latest docker push .dkr.ecr.us-east-1.amazonaws.com/stepfunctiontest:latest
-
Using custom docker image with SageMaker + AWS Step Functions
Feel free to check the project repository at https://github.com/ryfeus/stepfunctions2processing.
What are some alternatives?
tributary - Streaming reactive and dataflow graphs in Python
betaflight_processing - This repository contains only processing charts based on latest Betaflight (4.3) version
DataEngineeringProject - Example end to end data engineering project.
aws-textract-e2e-processing - This repo contains all the code required to do an IDP solution on AWS from document splitting, classification to extraction. We use a sample commercial acord data set.
getting-started - This repository is a getting started guide to Singer.
AWS-Auto-Scaling-Fargate-Cluster - ☁️ 🚀 Example Auto Scaling Fargate Cluster Created on AWS using CloudFormation
Moto - A library that allows you to easily mock out tests based on AWS infrastructure.
gluonts - Probabilistic time series modeling in Python
stepview - All your AWS Stepfunctions at a glance in the terminal! 🧐