Data Pipelines - how do you build data pipelines for sources not available in today’s ELT tools (Fivetran, Talend, Airbyte)? Old fashioned scripts and YOLO?

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering
event-driven Kafka Rabbitmq message-queue message-bus

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • plumber

    A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.

    • For CDC and event driven part of the stack, Plumber is a great free tool. That project is going to be adding sampling soon too - this can def help with the cost of ETL.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • plumber VS kaf - a user suggested alternative

    2 projects | 12 Jan 2024

  • batchcorp/plumber: A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.

    1 project | /r/devel | 9 Nov 2022

  • Plumber

    1 project | /r/devopspro | 25 Sep 2022

  • How do you handle integration tests on Github Action?

    1 project | /r/devops | 30 Mar 2021

  • UPDATE: Curl-like CLI tool for viewing/writing to and from Kafka, RabbitMQ and more

    2 projects | /r/devops | 29 Mar 2021