Every Database Will Support Iceberg — Here's Why

1. PostgreSQL

   1 510 18,031 10.0 C

   Mirror of the official PostgreSQL GIT repository. Note that this is just a *mirror* - we don't work with pull requests on github. To contribute, please see https://wiki.postgresql.org/wiki/Submitting_a_Patch

   

   Traditional databases — PostgreSQL, MySQL, etc. — store their data in proprietary formats. That format is optimized for that engine and can’t be directly accessed by anything else. Even if something like Trino can connect to Postgres, it’s still running queries through Postgres itself, not reading its storage directly. You’re just a client.

2. Stream

   getstream.io featured

   Stream - Scalable APIs for Chat, Feeds, Moderation, & Video. Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.

   

3. MySQL

   2 190 11,538 9.9 C++

   MySQL Server, the world's most popular open source database, and MySQL Cluster, a real-time, open source transactional database.

   

   Traditional databases — PostgreSQL, MySQL, etc. — store their data in proprietary formats. That format is optimized for that engine and can’t be directly accessed by anything else. Even if something like Trino can connect to Postgres, it’s still running queries through Postgres itself, not reading its storage directly. You’re just a client.

4. Trino

   3 52 11,577 10.0 Java

   Official repository of Trino, the distributed SQL query engine for big data, former

   

   Traditional databases — PostgreSQL, MySQL, etc. — store their data in proprietary formats. That format is optimized for that engine and can’t be directly accessed by anything else. Even if something like Trino can connect to Postgres, it’s still running queries through Postgres itself, not reading its storage directly. You’re just a client.

5. Apache Spark

   4 123 41,486 10.0 Scala

   Apache Spark - A unified analytics engine for large-scale data processing

   

   Apache Iceberg defines a table format that separates how data is stored from how data is queried. Any engine that implements the Iceberg integration — Spark, Flink, Trino, DuckDB, Snowflake, RisingWave — can read and/or write Iceberg data directly.

6. Apache Pulsar

   5 34 14,740 9.8 Java

   Apache Pulsar - distributed pub-sub messaging system

   

   Ingest real-time data from Kafka, Pulsar, or CDC sources like Postgresand MySQL, with built-in support for Debezium.

7. ApacheKafka

   6 137 36 0.0

   A curated re-sources list for awesome Apache Kafka

   

   Ingest real-time data from Kafka, Pulsar, or CDC sources like Postgresand MySQL, with built-in support for Debezium.

8. iceberg

   7 24 7,705 9.9 Java

   Apache Iceberg

   

   If you follow me on LinkedIn or Medium, you’ve probably noticed I’ve been talking a lot about Apache Iceberg. And as the founder of RisingWave — a stream processing and management system — I get this question a lot:

9. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

10. Apache Flink

    8 24 25,052 9.9 Java

    Apache Flink

    

    Apache Iceberg defines a table format that separates how data is stored from how data is queried. Any engine that implements the Iceberg integration — Spark, Flink, Trino, DuckDB, Snowflake, RisingWave — can read and/or write Iceberg data directly.

11. debezium

    9 90 11,602 9.9 Java

    Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

    

    Ingest real-time data from Kafka, Pulsar, or CDC sources like Postgresand MySQL, with built-in support for Debezium.

12. bigquery-utils

    10 14 1,223 7.8 Jupyter Notebook

    Useful scripts, udfs, views, and other utilities for migration and data warehouse operations in BigQuery.

    

    This isn’t hypothetical. It’s already happening. Snowflake supports reading and writing Iceberg. Databricks added Iceberg interoperability via Unity Catalog. Redshift and BigQuery are working toward it.

Suggest a related project