Best Open source no-code ELT tool for startup

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • Apache Spark

    Apache Spark - A unified analytics engine for large-scale data processing

    For my ETL/data warehouse/analytics needs, I've been very happy with Apache Airflow combined with Apache Spark.

  • Airflow

    Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

    For my ETL/data warehouse/analytics needs, I've been very happy with Apache Airflow combined with Apache Spark.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  • versatile-data-kit

    One framework to develop, deploy and operate data workflows with Python and SQL.

    Opensource, good for basic SQL and/or Python skills, extensible and provides support in setup/adoption of the framework. https://github.com/vmware/versatile-data-kit I'm the community manager for this project, I built my first full ELT pipeline (tracking GitHub stats) with no previous experience on my first month totally by myself. It's covering the full data journey. Oh, and it has Airflow integration, with that you can have a dashboard to see your jobs, dependencies but has better/more intuitive scheduling.

  • vector

    A high-performance observability data pipeline.

    NiFi is a beast. It can do just about anything and is pretty quick to get things up and running. I wanted something with a smaller footprint for some specific use cases and ended up moving to Benthos (https://www.benthos.dev/) for my pipelines. It supports a lot of inputs, outputs and processors by default. It uses config files to define your pipelines. I have found it very reliable and flexible. Great documentation and community as well. Also might want to check out the Vector project (https://github.com/vectordotdev/vector).

  • Benthos

    Fancy stream processing made operationally mundane

    NiFi is a beast. It can do just about anything and is pretty quick to get things up and running. I wanted something with a smaller footprint for some specific use cases and ended up moving to Benthos (https://www.benthos.dev/) for my pipelines. It supports a lot of inputs, outputs and processors by default. It uses config files to define your pipelines. I have found it very reliable and flexible. Great documentation and community as well. Also might want to check out the Vector project (https://github.com/vectordotdev/vector).

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts