Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Top 23 streaming-data Open-Source Projects
-
miller
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
-
-
-
readyset
Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
-
-
Memgraph
Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.
-
-
-
-
-
scikit-multiflow
A machine learning package for streaming data in Python. The other ancestor of River.
-
hstream
HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications. (by hstreamdb)
-
-
swim
Full stack application platform for building stateful microservices, streaming APIs, and real-time UIs
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: Ask HN: How Can I Make My Front End React to Database Changes in Real-Time? | news.ycombinator.com | 2024-04-17[2] https://materialize.com/
River is a Python library for online machine learning. Online machine learning can dynamically adapt to new patterns in the data, or when the data itself is generated as a function of time, e.g., stock price prediction, content personalization.
Project mention: Ask HN: How Can I Make My Front End React to Database Changes in Real-Time? | news.ycombinator.com | 2024-04-17- Some platforms like Supabase Realtime [3] and Firebase offer subscription models to database changes, but these solutions fall short when dealing with complex queries involving joins or group-bys.
My vision is that the modern frontend to behave like a series of materialized views that dynamically update as the underlying data changes. Current state management libraries handle state trees well but don't seamlessly integrate with relational or graph-like database structures.
The only thing I can think of is to implement it by myself, which sounds like a big PITA.
Anything goes, Brainstorm with me. Is it causing you headaches as well? Are you familiar with an efficient solution? how are you all tackling it?
[1] https://readyset.io/
Memgraph | Staff C++ Database Engineer | REMOTE (Central/Western Europe, LatAm, or North America) https://memgraph.com/
Memgraph is a Seed stage, open source graph database vendor. Graph DBs are a great solution for GenAI, logistics, cybersecurity and fintech so we are looking to grow aggressively this year.
We're looking for a staff-level engineer to set technical direction, mentor junior team members, and solve some very difficult problems.
Either DM me (the hiring manager) or apply here: https://join.com/companies/memgraph/10684850-staff-software-...
Project mention: Building a streaming SQL engine with Arrow and DataFusion | news.ycombinator.com | 2024-03-18
River is actually the merger between creme and scikit-multiflow, another great example of open source collaboration and continuation.
Project mention: Show HN: Streamdal – an open-source tail -f for your data | /r/hackernews | 2023-11-03
Project mention: Show HN: Kafbat UI for Apache Kafka v1.0 is out | news.ycombinator.com | 2024-03-22
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
streaming-data related posts
- Fancy stream processing made operationally mundane
- Benthos: Fancy stream processing made operationally mundane
- Need help on cleaning this data!!
- Running weekly average
- johnkerl/miller: Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
- Benthos: Open-source stream processing tool
- Go in depth youtube channels?
-
A note from our sponsor - InfluxDB
www.influxdata.com | 26 Apr 2024
Index
What are some of the best open-source streaming-data projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | awesome-bigdata | 12,792 |
| 2 | miller | 8,553 |
| 3 | kafka-ui | 8,458 |
| 4 | Benthos | 7,559 |
| 5 | materialize | 5,567 |
| 6 | river | 4,766 |
| 7 | readyset | 3,867 |
| 8 | smart_open | 3,091 |
| 9 | fluvio | 2,638 |
| 10 | Memgraph | 2,086 |
| 11 | Pravega | 1,966 |
| 12 | go-streams | 1,753 |
| 13 | Streamz | 1,217 |
| 14 | bytewax | 1,144 |
| 15 | zpl | 960 |
| 16 | OnlineStats.jl | 816 |
| 17 | scikit-multiflow | 739 |
| 18 | hstream | 691 |
| 19 | awesome-kafka | 565 |
| 20 | streamdal | 529 |
| 21 | swim | 473 |
| 22 | kafka-ui | 299 |
| 23 | kafka-streams-in-action | 259 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com