Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR. Learn more →
Top 18 Go Stream Processing Projects
-
Project mention: Kuvasz-streamer: open-source CDC for Postgres for low latency replication | news.ycombinator.com | 2025-01-03
lots of Go based CDC stuff going on these days. Redpanda Connect (formerly benthos) recently added support for Postgres CDC [1] and MySQL is coming soon too [2].
1: https://github.com/redpanda-data/connect/pull/2917
2: https://github.com/redpanda-data/connect/pull/3014
-
CodeRabbit
CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
-
Project mention: Watermill – Building event-driven applications the easy way in Go | news.ycombinator.com | 2025-01-31
-
peerdb
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
Sai from ClickHouse here. Have been living and breathing past year helping customers integrating Postgres and ClickHouse to form their data stack. Totally agreed with this statement - there are numerous production grade workloads solving most of their data problems using these purpose-built Open Source database. My team at ClickHouse has been working to make the integration even seamless. :) We work on PeerDB, an open source tool enabling seamless Postgres CDC to ClickHouse https://github.com/PeerDB-io/peerdb/ This integration is now als natively available in the Cloud through ClickPipes. The private preview was released just last week https://clickhouse.com/blog/postgres-cdc-connector-clickpipe...
-
-
yomo
🦖 Stateful Serverless Framework for Geo-distributed Edge AI Infra. with function calling support, write once, run on any model.
-
> To me the best solution seem like combining storing writes on EBS (or even NVMe) initially to minimize the time until writes can be acknowledged, and creating a chunk on S3 standard every second or so.
Yep, this is approximately Gazette's architecture (https://github.com/gazette/core). It buys the latency profile of flash storage, with the unbounded storage and durability of S3.
An addendum is there's no need to flush to S3 quite that frequently, if readers instead tail ACK'd content from local disk. Another neat thing you can do is hand bulk historical readers pre-signed URLs to files in cloud storage, so those bytes don't need to proxy through brokers.
-
-
Nutrient
Nutrient – The #1 PDF SDK Library, trusted by 10K+ developers. Other PDF SDKs promise a lot - then break. Laggy scrolling, poor mobile UX, tons of bugs, and lack of support cost you endless frustrations. Nutrient’s SDK handles billion-page workloads - so you don’t have to debug PDFs. Used by ~1 billion end users in more than 150 different countries.
-
-
gostream
A Go port of the Java Streams API. Type-safe and functional Go Streams processing for Go 1.18+.
-
kawa
A blazingly fast event stream processing library powering the reveald event processing daemon. (by runreveal)
-
-
-
csvplus
csvplus extends the standard Go encoding/csv package with fluent interface, lazy stream operations, indices and joins.
-
stream
stream is a concurrency pattern implementation using Go generics. a.k.a. my accidentally awesome library (by devnw)
-
-
-
Integrand
Production-ready, Light, and Flexible Webhook Infrastructure | Effortlessly Build Performant Webhook Integrations
Project mention: Integrand VS webhook.site - a user suggested alternative | libhunt.com/r/integrand | 2024-06-26 -
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Go Stream Processing discussion
Go Stream Processing related posts
-
Bento, the open source fork of the project formerly known as Benthos
-
Benthos – Fancy stream processing made operationally mundane
-
Hummock: A Storage Engine Designed for Stream Processing
-
Fancy stream processing made operationally mundane
-
Lakehouse - event data
-
Benthos: Fancy stream processing made operationally mundane
-
Benthos: Open-source stream processing tool
-
A note from our sponsor - CodeRabbit
coderabbit.ai | 15 Feb 2025
Index
What are some of the best open-source Stream Processing projects in Go? This list will help you:
# | Project | Stars |
---|---|---|
1 | connect | 8,240 |
2 | watermill | 8,076 |
3 | peerdb | 2,378 |
4 | go-streams | 1,978 |
5 | yomo | 1,707 |
6 | core | 738 |
7 | function-mesh | 214 |
8 | machine | 159 |
9 | gostream | 158 |
10 | kawa | 117 |
11 | stream | 91 |
12 | goio | 83 |
13 | csvplus | 67 |
14 | stream | 67 |
15 | vaero | 66 |
16 | styx | 64 |
17 | Integrand | 12 |
18 | go-bench-stream | 2 |