Useful libraries for data engineering in various programming languages

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • hamilton

    Discontinued A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton (by stitchfix)

  • Python - https://github.com/stitchfix/hamilton (author here). It's great if you want your code to be always unit testable and documentation friendly, and you want to be able to visualize execution. Blog post on using it with Pandas https://link.medium.com/XhyYD9BAntb.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • AlphaFold 3 predicts the structure and interactions of all of life's molecules

    5 projects | news.ycombinator.com | 8 May 2024
  • TimesFM (Time Series Foundation Model) for time-series forecasting

    4 projects | news.ycombinator.com | 8 May 2024
  • A library to assist writing memory-unsafe code in "pure" Python

    2 projects | news.ycombinator.com | 7 May 2024
  • 2D Gaussian Splatting for Geometrically Accurate Radiance Fields

    1 project | news.ycombinator.com | 8 May 2024
  • CDK CodeChecker v2: Breaking Free from Third-Party Dependencies

    3 projects | dev.to | 8 May 2024