CDC Implementation

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering
change-data-capture kafka-connect Apache Kafka debezium Cdc

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • dbt-external-tables

    dbt macros to stage external sources

    • We keep track of the schema in git using dbt-external-tables. If a new column is added upstream, whoever made the change is responsible for updating the schema in dbt. It's just a few lines of code. Once PR is merged, the external table that reads the S3 files is refreshed. We never remove columns.

  • debezium

    Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

    • We use debezium at work to push data from operational databases to data warehouse.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • How the heck do I validate records with this kind of data??

    1 project | /r/AskProgramming | 5 Dec 2023

  • Managed Relational Databases with AWS RDS and Aurora

    1 project | dev.to | 24 Aug 2023

  • Postgresql to hadoop in real time

    1 project | /r/dataengineering | 26 Jun 2023

  • ClickHouse Advanced Tutorial: Apply CDC from MySQL to ClickHouse

    1 project | dev.to | 15 Jun 2023

  • Debezium: Stream Changes from Your Database

    1 project | news.ycombinator.com | 14 Jun 2023