Airflow on Kubernetes (with Spark)

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • charts

    Bitnami Helm Charts (by bitnami)

  • There were already a number of Helm Charts available to deploy Airflow on Kubernetes, such as the Bitnami one and a user community one. These Helm Charts are definitions of Kubernetes resources which can be used to easily install applications on the Kubernetes cluster. Recently Apache Airflow released an own 'official' Helm Chart as well, with lots of functionality that can be enabled with minimal configuration. With this introduction it feels like deploying the tool on Kubernetes is a great option that has a great future ahead.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts