Go Stream Processing

Open-source Go projects categorized as Stream Processing

Top 18 Go Stream Processing Projects

Stream Processing
  1. connect

    Fancy stream processing made operationally mundane (by redpanda-data)

    Project mention: Kuvasz-streamer: open-source CDC for Postgres for low latency replication | news.ycombinator.com | 2025-01-03

    lots of Go based CDC stuff going on these days. Redpanda Connect (formerly benthos) recently added support for Postgres CDC [1] and MySQL is coming soon too [2].

    1: https://github.com/redpanda-data/connect/pull/2917

    2: https://github.com/redpanda-data/connect/pull/3014

  2. CodeRabbit

    CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.

    CodeRabbit logo
  3. watermill

    Building event-driven applications the easy way in Go.

    Project mention: Watermill – Building event-driven applications the easy way in Go | news.ycombinator.com | 2025-01-31
  4. peerdb

    Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

    Project mention: 7 Databases in 7 Weeks for 2025 | news.ycombinator.com | 2024-12-05

    Sai from ClickHouse here. Have been living and breathing past year helping customers integrating Postgres and ClickHouse to form their data stack. Totally agreed with this statement - there are numerous production grade workloads solving most of their data problems using these purpose-built Open Source database. My team at ClickHouse has been working to make the integration even seamless. :) We work on PeerDB, an open source tool enabling seamless Postgres CDC to ClickHouse https://github.com/PeerDB-io/peerdb/ This integration is now als natively available in the Cloud through ClickPipes. The private preview was released just last week https://clickhouse.com/blog/postgres-cdc-connector-clickpipe...

  5. go-streams

    A lightweight stream processing library for Go

  6. yomo

    🦖 Stateful Serverless Framework for Geo-distributed Edge AI Infra. with function calling support, write once, run on any model.

  7. core

    Build platforms that flexibly mix SQL, batch, and stream processing paradigms (by gazette)

    Project mention: Introducing S2 | news.ycombinator.com | 2024-12-21

    > To me the best solution seem like combining storing writes on EBS (or even NVMe) initially to minimize the time until writes can be acknowledged, and creating a chunk on S3 standard every second or so.

    Yep, this is approximately Gazette's architecture (https://github.com/gazette/core). It buys the latency profile of flash storage, with the unbounded storage and durability of S3.

    An addendum is there's no need to flush to S3 quite that frequently, if readers instead tail ACK'd content from local disk. Another neat thing you can do is hand bulk historical readers pre-signed URLs to files in cloud storage, so those bytes don't need to proxy through brokers.

  8. function-mesh

    The serverless framework purpose-built for event streaming applications.

  9. Nutrient

    Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.

    Nutrient logo
  10. machine

    Machine is a workflow/pipeline library for processing data (by whitaker-io)

  11. gostream

    A Go port of the Java Streams API. Type-safe and functional Go Streams processing for Go 1.18+.

  12. kawa

    A blazingly fast event stream processing library powering the reveald event processing daemon. (by runreveal)

  13. stream

    Go Stream, like Java 8 Stream. (by youthlin)

  14. goio

    An implementation of IO, Stream, Fiber for golang inspired by cats and fs2

  15. csvplus

    csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.

  16. stream

    stream is a concurrency pattern implementation using Go generics. a.k.a. my accidentally awesome library (by devnw)

  17. vaero

    Complete log pipeline solution in Python. Runs fast in Go.

  18. styx

    Simple, high-performance event streaming broker

  19. Integrand

    Production-ready, Light, and Flexible Webhook Infrastructure | Effortlessly Build Performant Webhook Integrations

    Project mention: Integrand VS webhook.site - a user suggested alternative | libhunt.com/r/integrand | 2024-06-26
  20. go-bench-stream

    🌊 Go Benchmarks for Stream Processing

  21. SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Go Stream Processing discussion

Log in or Post with

Go Stream Processing related posts

  • Bento, the open source fork of the project formerly known as Benthos

    4 projects | news.ycombinator.com | 31 May 2024
  • Benthos – Fancy stream processing made operationally mundane

    1 project | news.ycombinator.com | 15 May 2024
  • Hummock: A Storage Engine Designed for Stream Processing

    2 projects | dev.to | 17 Oct 2023
  • Fancy stream processing made operationally mundane

    1 project | news.ycombinator.com | 6 Aug 2023
  • Lakehouse - event data

    1 project | /r/dataengineering | 19 Jul 2023
  • Benthos: Fancy stream processing made operationally mundane

    1 project | news.ycombinator.com | 15 Jul 2023
  • Benthos: Open-source stream processing tool

    1 project | news.ycombinator.com | 5 Jun 2023
  • A note from our sponsor - CodeRabbit
    coderabbit.ai | 15 Feb 2025
    Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR. Learn more →

Index

What are some of the best open-source Stream Processing projects in Go? This list will help you:

# Project Stars
1 connect 8,240
2 watermill 8,076
3 peerdb 2,378
4 go-streams 1,978
5 yomo 1,707
6 core 738
7 function-mesh 214
8 machine 159
9 gostream 158
10 kawa 117
11 stream 91
12 goio 83
13 csvplus 67
14 stream 67
15 vaero 66
16 styx 64
17 Integrand 12
18 go-bench-stream 2

Sponsored
CodeRabbit: AI Code Reviews for Developers
Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
coderabbit.ai

Did you know that Go is
the 4th most popular programming language
based on number of references?