Top 14 iceberg Open-Source Projects

• doris

  42 11,389 10.0 Java

  Apache Doris is an easy-to-use, high performance and unified analytics database.

  

Project mention: Variant in Apache Doris 2.1.0: a new data type 8 times faster than JSON for semi-structured data analysis | dev.to | 2024-03-27

As an open-source real-time data warehouse, Apache Doris provides semi-structured data processing capabilities, and the newly-released version 2.1.0 makes a stride in this direction. Before V2.1, Apache Doris stores semi-structured data as JSON files. However, during query execution, the real-time parsing of JSON data leads to high CPU and I/O consumption in addition to high query latency, especially when the dataset is huge and complicated. Moreover, the lack of a pre-defined schema means there is no handle for query optimization.

• Trino

  44 9,597 10.0 Java

  Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

  

Project mention: Trino: Fast distributed SQL query engine for big data analytics | news.ycombinator.com | 2024-03-19

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• starrocks

  12 7,910 10.0 Java

  StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries. InfoWorld’s 2023 BOSSIE Award for best open source software.

  

Project mention: A MySQL compatible database engine written in pure Go | news.ycombinator.com | 2024-04-09

tidb has been around for a while, it is distributed, written in Go and Rust, and MySQL compatible. https://github.com/pingcap/tidb

Somewhat relatedly, StarRocks is also MySQL compatible, written in Java and C++, but it's tackling OLAP use-cases. https://github.com/StarRocks/starrocks

• iceberg

  18 5,540 9.9 Java

  Apache Iceberg

  

Project mention: Iceberg won the table format war: But not in the way you thought it might | /r/dataengineering | 2023-07-06

• iceberg.vim

  22 2,106 0.0 Vim Script

  :antarctica: Bluish color scheme for Vim and Neovim

Project mention: Iceberg.nvim looking wrong in buffers, but not in Telescope previews | /r/neovim | 2023-06-09

I haven't tried the original https://github.com/cocopon/iceberg.vim yet since I wanted to keep a Lua config as much as possible. I'm using Nvim 0.9.

• nessie

  13 843 9.9 Java

  Nessie: Transactional Catalog for Data Lakes with Git-like semantics

  

Project mention: A deep dive into the concept and world of Apache Iceberg Catalogs | dev.to | 2024-03-01

Nessie is an innovative open-source catalog that extends beyond the traditional catalog capabilities in the Apache Iceberg ecosystem, introducing git-like features to data management. This catalog not only tracks table metadata but also allows users to capture commits at a holistic level, enabling advanced operations such as multi-table transactions, rollbacks, branching, and tagging. These features provide a new layer of flexibility and control over data changes, resembling version control systems in software development.

• iceberg-rust

  1 407 9.3 Rust

  Apache Iceberg (by apache)

  

Project mention: Apache Iceberg now has a native Rust implementation | news.ycombinator.com | 2024-02-20

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• ngods-stocks

  3 373 0.0 Jupyter Notebook

  New Generation Opensource Data Stack Demo

• puffin

  1 277 7.6

  Serverless HTAP cloud data platform powered by Arrow × DuckDB × Iceberg (by sutoiku)

  

• openhouse

  3 253 9.2 Java

  Open Control Plane for Tables in Data Lakehouse

  

Project mention: Linkedin OpenHouse: Control Plane for Tables in Data Lakehouses | news.ycombinator.com | 2024-03-11

• iceberg-python

  2 236 9.8 Python

  Apache PyIceberg

  

Project mention: Understanding Parquet, Iceberg and Data Lakehouses | news.ycombinator.com | 2023-12-29

You don't need a Spark deployment. The first reference implementations for reading and writing were in Spark.

Now, with PyIceberg, there is read support in Python. Write support should be merged very soon - https://github.com/apache/iceberg-python/pull/41

• dbt-athena

  1 187 9.2 Python

  The athena adapter plugin for dbt (https://getdbt.com) (by dbt-athena)

  

• data_origination_workshop

  1 11 6.3 Shell

  Hands-on workshop with Iceberg, Redpanda, Debezium and Kafka-Connect

• lastfm-iceberg

  1 9 4.4 JavaScript

  Generate an Iceberg-Chart based on your Last.fm music history

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

iceberg related posts

• A deep dive into the concept and world of Apache Iceberg Catalogs

  1 project | dev.to | 1 Mar 2024

• Iceberg won the table format war: But not in the way you thought it might

  2 projects | /r/dataengineering | 6 Jul 2023

• Why is Hive Metastore everywhere? (Especially Iceberg)

  1 project | /r/dataengineering | 30 Jun 2023

• Iceberg.nvim looking wrong in buffers, but not in Telescope previews

  1 project | /r/neovim | 9 Jun 2023

• Lakehouse using AWS Athena on Iceberg Concerns

  1 project | /r/dataengineering | 28 May 2023

• Matrix theme

  2 projects | /r/neovim | 23 Mar 2023

• rust diagnostics hides details of the problem

  3 projects | /r/neovim | 16 Mar 2023
• A note from our sponsor - SaaSHub
  www.saashub.com | 10 May 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com