Using rust for DE activities?

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering
Rust Kafka deltalake AWS Delta

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • kafka-delta-ingest

    A highly efficient daemon for streaming data from Kafka into Delta Lake

    • Rust can offer incredible cost savings when you can use it in place of spark to interact with your delta lake. One such project was kafka-delta-ingest. The developers were able to reduce the cost of running the pipeline by over 90%. However, most of this stuff is still very experimental and not ready for production but you will definitely be seeing more projects like this just based on how much money can be saved.

  • matano

    Open source security data lake for threat hunting, detection & response, and cybersecurity analytics at petabyte scale on AWS

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Streaming Data and Postgres

    1 project | /r/dataengineering | 20 Dec 2022

  • process millions of events per sec

    1 project | /r/dataengineering | 9 Aug 2022

  • Exactly once delivery from Kafka to Delta Lake with Rust

    1 project | news.ycombinator.com | 15 Sep 2021

  • Which lakehouse table format do you expect your organization will be using by the end of 2023?

    2 projects | /r/dataengineering | 25 Dec 2022

  • Delta-rs – a Rust-based implementation of deltalake

    1 project | news.ycombinator.com | 8 Apr 2024