Iceberg + Spark + Trino + Dagster: modern, open-source data stack installation

This page summarizes the projects mentioned and recommended in the original post on /r/bigdata

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • ngods-stocks

    New Generation Opensource Data Stack Demo

  • I’m guessing that you use the Spark JDBC dataframes. Trino is in my opinion easier to use. You get SQL access to all pgsql tables with this simple config file. No need to write a piece of code for each table. The config above just maps the pgsql schema to a Trino schema. Then you configure Iceberg with another config file and you can do cross-schema SQL queries like create table pgsql.xyz from select * from iceberg.abc. Or you can use dbt that is based on SQL.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • 4 Simple Steps to Develop a WhatsApp Support Chatbot (Using LLMs, OpenAI & Python)

    1 project | dev.to | 17 May 2024
  • LLM-Training-Puzzles: What would you do with 1000 H100s?

    1 project | news.ycombinator.com | 16 May 2024
  • Think DSP Is an Introduction to Digital Signal Processing in Python

    1 project | news.ycombinator.com | 15 May 2024
  • The Fundamentals of Modern Deep Learning with PyTorch

    1 project | news.ycombinator.com | 15 May 2024
  • Code a Neural Network from scratch to solve the binary MNIST problem

    1 project | dev.to | 14 May 2024