Template Dagster repo using poetry and a single Docker container; works well with CICD
The associated code repo can be found here
An orchestration platform for the development, production, and observation of data assets.
I’ve been exploring dagster for some of Mile Two’s data orchestration needs and have been absolutely loving it. It hits all of the sweet spots for gradually developing data pipelines, but I found myself in a familiar situation: trying to logically structure my code such that it can easily be containerized and thrown into a CI/CD process. To that end, I’ve open-sourced a boilerplate project that enhances the dagster development experience with these valuable features:
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
ETL advice appreciated
1 project | reddit.com/r/ETL | 19 Jun 2022
Thinking of making a switch from actuarial science to data engineering
1 project | reddit.com/r/dataengineering | 15 Apr 2022
Am I learning DE in a too roundabout way?
1 project | reddit.com/r/dataengineering | 24 Jan 2022
What is this kind of texture called?
1 project | reddit.com/r/Design | 19 Dec 2021
Airflow 2.0 vs Prefect
1 project | reddit.com/r/dataengineering | 20 Oct 2021