ELT of my own Strava data using the Strava API, MySQL, Python, S3, Redshift, and Airflow

• StravaDataPipline

  1 28 6.0 Python

  :arrows_counterclockwise: :running: EtLT of my own Strava data using the Strava API, MySQL, Python, S3, Redshift, and Airflow

The GitHub repo can be found here: https://github.com/jackmleitch/StravaDataPipline A corresponding blog post can also be found here: https://jackmleitch.com/blog/Strava-Data-Pipeline

• versatile-data-kit

  52 410 9.7 Python

  One framework to develop, deploy and operate data workflows with Python and SQL.

  

I believe that you would not need to build the docker image yourself. There are data engineering frameworks which allow you to build your data jobs yourself and take care of the containerisation of your pipeline. You can have a look at this ingest from rest API example. They would also allow you to schedule your data job using cron, while data job itself can contain SQL & Python.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project