Top 7 Go Cdc Projects

peerdb

5 1,615 9.9 Go

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

Project mention: Pgwire: a Rust library for PostgreSQL compatible application | news.ycombinator.com | 2024-03-20

We at PeerDB (https://github.com/PeerDB-io/peerdb) were early adopters of Pgwire to implement our Postgres-compatible SQL Layer to do ETL. Very easy to work with. Saved us multiple months of effort to build it from scratch.

transfer

7 527 9.4 Go

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.

Project mention: Migrate mongodb Datawarehouse to snowflake | /r/snowflake | 2023-12-04

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
tiflow

1 411 9.7 Go

This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)
pgcapture

2 174 8.0 Go

A scalable Netflix DBLog implementation for PostgreSQL
go-benchmarks

1 9 1.9 Go

Comprehensive and reproducible benchmarks for Go developers and architects. (by skerkour)

Project mention: Reasons to Prefer Blake3 over Sha256 | news.ycombinator.com | 2023-11-13

At the end of the day, what matters really for most people is
1) Certifications (FIPS...)
2) Speed.
SHA-256 is fast enough for maybe 99,9% of use cases as you will saturate your I/O way before SHA-256 becomes your bottleneck[0][1]. Also, from my experience with the different available implementations, SHA-256 is up to 1.8 times faster than Blake3 on arm64.
[0] https://github.com/skerkour/go-benchmarks/blob/main/results/...
[1] https://kerkour.com/fast-hashing-algorithms

imsy

1 8 10.0 Go

simple incremental pull of immutable large files
kuvasz-streamer

1 5 9.0 Go

Kuvasz-Streamer is a Postgres-to-Postgres data consolidation and change data capture project.

Project mention: Show HN: Kuvasz-Streamer a Change Data Capture for Postgres Tool | news.ycombinator.com | 2024-02-16

InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).