Near Real Time Ingestion to DB using Python

This page summarizes the projects mentioned and recommended in the original post on /r/dataengineering
Web Frameworks Python Stream Processing Rust data-engineering

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • bytewax

    Python Stream Processing

    • You can probably use Python to solve your problem, there are many ways you can speed up your deserialization/flattening. I work on Bytewax (https://github.com/bytewax/bytewax) and I wouldn't mention it if it wasn't a good fit, but I think it's worth looking at here. It is a stream processor that makes it easy to scale, maintain order, track progress, and you just write native Python.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Building a streaming SQL engine with Arrow and DataFusion

    1 project | news.ycombinator.com | 18 Mar 2024

  • Stream processing framework for a new project in Python

    1 project | /r/dataengineering | 2 Jun 2023

  • A Python package for streaming synthetic data

    2 projects | /r/Python | 25 May 2023

  • Snowflake - what are the streaming capabilities it provides?

    3 projects | /r/dataengineering | 10 May 2023

  • FLaNK AI Weekly 18 March 2024

    39 projects | dev.to | 18 Mar 2024