Self managed Apache Airflow with Data on EKS

• data-on-eks

  2 500 9.0 HCL

  DoEKS is a tool to build, deploy and scale Data & ML Platforms on Amazon EKS

  

git clone https://github.com/awslabs/data-on-eks.git cd data-on-eks/schedulers/terraform/self-managed-airflow

• terraform-aws-eks-blueprints

  39 2,482 9.1 HCL

  Configure and deploy complete EKS clusters.

  

Initializing modules... Downloading git::https://github.com/aws-ia/terraform-aws-eks-blueprints.git?ref=v4.15.0 for airflow_irsa... - airflow_irsa in .terraform/modules/airflow_irsa/modules/irsa Downloading registry.terraform.io/terraform-aws-modules/s3-bucket/aws 3.8.2 for airflow_s3_bucket... - airflow_s3_bucket in .terraform/modules/airflow_s3_bucket Downloading registry.terraform.io/terraform-aws-modules/rds/aws 5.6.0 for db... - db in .terraform/modules/db .. .. Terraform has been successfully initialized! You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work. If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• airflow-dags

  1 0 4.2 Python

  Sample DAGs repo to use with Apache Airflow GitSync feature.

  

You can then pop this into a browser and then login using the Airflow default user (admin/admin). You should then see the Apache Airflow 2.4.3 UI, and also have a couple of workflows (DAGs) that have been populated from the git repository configured in our code (https://github.com/Hyper-Mesh/airflow-dags)

Suggest a related project