data-stream

Open-source projects categorized as data-stream
Edit details
Language: + Go + Java + XSLT + C# + JavaScript + TypeScript + Python + Swift + Rust
Topics: Kafka streaming-data Reactive Programming Stream Processing time-series
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
Sponsored

Top 12 data-stream Open-Source Projects

  • awesome-bigdata

    A curated list of awesome big data frameworks, ressources and other awesomeness.

    • Project mention: Good coding groups for black women? | news.ycombinator.com | 2024-01-13

  • strimzi-kafka-operator

    Apache Kafka® running on Kubernetes

    • Project mention: Knative switchboard series, part 1. Setup Knative Eventing with Kafka from scratch, scale based on events volume, and monitor | dev.to | 2024-01-04

    Knative dashboards together with Kafka's dashboards it sheds light on almost any aspect of what's going on in the system.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • go-streams

    A lightweight stream processing library for Go

  • boomfilters

    Probabilistic data structures for processing continuous, unbounded streams.

  • openscap

    NIST Certified SCAP 1.2 toolkit

  • Spreads

    Series and Panels for Real-time and Exploratory Analysis of Data Streams

    • Project mention: Beautiful branchless binary search | news.ycombinator.com | 2023-04-28

    Branchless or not in this case, it still touches memory in not so good pattern. I found that a significant speedup of a classic BS could be achieved by switching to linear SIMD search when the remaining range has a width of 3-4 SIMD lines (or maybe even a little more). The bounds of that range are likely already touched and in cache, then prefetching helps. It gives 30-50% gain on 1K items array of integers, 10-25% on 1M items, depending on data distribution. Here is an example in C#: https://github.com/Spreads/Spreads/blob/main/src/Spreads.Cor...

  • conduit

    Conduit streams data between data stores. Kafka Connect replacement. No JVM required. (by ConduitIO)

    • Project mention: Pulling CDC data from Postgres | /r/dataengineering | 2023-04-30

    I'd like to mention Conduit + its Postgres connector. The Pg connector comes built-in, so all that is needed is a single Conduit binary to get started. It relies on WAL, but the connector creates the replication slot itself (if needed).

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • scramjet

    Public tracker for Scramjet Cloud Platform, a platform that bring data from many environments together.

  • openHistorian

    The Open Source Time-Series Data Historian

  • kafka-ml

    Kafka-ML: connecting the data stream with ML/AI frameworks (now TensorFlow and PyTorch!)

    • Project mention: What are your favorite tools or components in the Kafka ecosystem? | /r/apachekafka | 2023-05-31

    For example, CLIs, UIs, monitoring tools / integrations, cluster administration, stream processing libraries (Flink, Kafka Streams, smaller / newer libs), etc? Anything in the ML / AI space (e.g. a quick Google search came up with https://github.com/ertis-research/kafka-ml).

  • CohesionKit

    Lighweight realtime data synchronisation library

  • open-transaction-pool

    A CKB Open Transaction solution based on memory pool.

    • Project mention: How to program with CKB: Layer 1.5 | /r/NervosNetwork | 2023-09-11

    EthanYuan/open-transaction-pool: A CKB Open Transaction solution based on memory pool.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

data-stream related posts

Index

What are some of the best open-source data-stream projects? This list will help you:

Project Stars
1 awesome-bigdata 12,792
2 strimzi-kafka-operator 4,456
3 go-streams 1,753
4 boomfilters 1,574
5 openscap 1,268
6 Spreads 417
7 conduit 345
8 scramjet 254
9 openHistorian 168
10 kafka-ml 144
11 CohesionKit 7
12 open-transaction-pool 3

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com