Introducing Iceberg Table Engine in RisingWave: Manage Streaming Data in Iceberg with SQL

1. PostgreSQL

   1 510 18,031 10.0 C

   Mirror of the official PostgreSQL GIT repository. Note that this is just a *mirror* - we don't work with pull requests on github. To contribute, please see https://wiki.postgresql.org/wiki/Submitting_a_Patch

   

   Propagating changes from operational databases (Postgres, MySQL, etc.) via CDC into Iceberg requires careful handling. Since Iceberg tables typically lack primary key enforcement at the storage layer, updates and deletes must often be implemented via equality deletes: deleting the old row version and inserting the new one. A streaming ingest engine must reliably:

2. Stream

   getstream.io featured

   Stream - Scalable APIs for Chat, Feeds, Moderation, & Video. Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.

   

3. MySQL

   2 190 11,525 9.9 C++

   MySQL Server, the world's most popular open source database, and MySQL Cluster, a real-time, open source transactional database.

   

   Propagating changes from operational databases (Postgres, MySQL, etc.) via CDC into Iceberg requires careful handling. Since Iceberg tables typically lack primary key enforcement at the storage layer, updates and deletes must often be implemented via equality deletes: deleting the old row version and inserting the new one. A streaming ingest engine must reliably:

4. Trino

   3 52 11,577 10.0 Java

   Official repository of Trino, the distributed SQL query engine for big data, former

   

   However, Iceberg defines the storage format, leaving the complexities of data ingestion and processing, especially for real-time streams, to separate systems. While query engines like Trino or Athena excel with static datasets, they aren't designed for continuous, low-latency ingestion and transformation of streaming data into Iceberg. This often forces engineers to integrate multiple complex tools, increasing operational overhead and fragility.

5. iceberg

   4 24 7,705 9.9 Java

   Apache Iceberg

   

   Apache Iceberg has rapidly become the open standard for table formats in modern data lakes. Its powerful features—schema evolution, hidden partitioning, time travel, and ACID compliance—enable scalable, reliable, and vendor-neutral data architectures.

6. debezium

   5 90 11,602 9.9 Java

   Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

   

   Interpret the CDC format (e.g., Debezium).

7. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

Suggest a related project