InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now. Learn more →
Top 19 Go Replication Projects
-
seaweedfs
SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
I’m interested in how it is compared to seaweedfs[1], which we use for storing weather data (about 3 PB) for ML training.
[1] https://github.com/seaweedfs/seaweedfs
-
InfluxDB
InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
-
Project mention: A faster way to copy SQLite databases between computers | news.ycombinator.com | 2025-05-01
seems like a new release is being worked on: https://github.com/benbjohnson/litestream/pull/636
-
cloudnative-pg
CloudNativePG is a comprehensive platform designed to seamlessly manage PostgreSQL databases within Kubernetes environments, covering the entire operational lifecycle from initial deployment to ongoing maintenance
To install CloudNativePG, first you should check the latest version on GitHub.
-
Project mention: The definitive guide to using Django with SQLite in production 💡 | dev.to | 2025-01-18
marmot: A distributed SQLite replicator built on top of NATS
-
Hey HN! We’re Evgeny and Arjun. We’ve built a managed version of BemiDB that syncs with your existing PostgreSQL database and gives you fast analytical queries without heavy ETL pipelines.
BemiDB Cloud automatically replicates your data into an Apache Iceberg table stored in your own S3-compatible bucket. The data is compressed columnar Parquet under the hood. We embed DuckDB for query execution, and we speak the Postgres wire protocol so you can use all your existing ORMs, BI tools, or notebooks.
We’ve seen teams push Postgres read replicas too far for analytics or wrangle big ETL flows. We’re aiming for something simpler. Instead of standing up a data warehouse with complex pipelines, you can point BemiDB at your Postgres instance, click to sync, and start querying. All data remains in an open format so you aren’t locked in.
We’d love your feedback! Check us out at https://bemidb.com and our open source repo at https://github.com/BemiHQ/BemiDB. What do you think?
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
olake
Fastest open-source tool for replicating Databases to Data Lake in Open Table Formats like Apache Iceberg. ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Supporting Postgres, MongoDB and MySQL
Project mention: Debezium to olake.io – PhysicsWallah switch for CDC | news.ycombinator.com | 2025-04-30-> Automatic schema evolution: new MongoDB fields appear as nullable columns; complex sub-docs land as JSON strings you can parse later
-> Resumable, chunked full loads: a pod crash resumes instead of restarting
-> Runs as either a Kubernetes CronJob or an Airflow task; config is one YAML/JSON file.
Their stack in one line: MongoDB → OLake writer → Iceberg on S3 → Spark jobs → Trino / occasional Redshift, all orchestrated by Airflow and/or K8s.
Posting here because many of us still bolt Kafka onto CDC just to land files. If you only need Iceberg tables, a simpler path might exist now. Curious to hear others’ experiences with broker-less CDC tools.
(Disclaimer: I work on OLake and hosted the meetup, but the talk is purely technical.)
Check out github repo - https://github.com/datazip-inc/olake
-
https://github.com/lovelaze/nebula-sync can sync configs with Pi-hole 6 now, but it’s quite a bit of code for what looks like just a few HTTP requests to get the config from one using the teleporter feature, then restore it on the others using the same.
-
Have you seen pgstream? https://github.com/xataio/pgstream
-
-
Project mention: Pgcapture – CDC Framework for PostgreSQL in Golang | news.ycombinator.com | 2024-09-02
-
-
kuvasz-streamer
Kuvasz-Streamer is a Postgres-to-Postgres data consolidation and change data capture project.
Project mention: Kuvasz-streamer: open-source CDC for Postgres for low latency replication | news.ycombinator.com | 2025-01-03One will want to be cognizant of its AGPLv3 license https://github.com/kuvasz-io/kuvasz-streamer/blob/v1.19.2/LI...
-
-
kivi
Dynamo-inspired distributed leader-less key-value database that has no unique features and no apparent reason to exist
-
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Go Replication discussion
Go Replication related posts
-
A faster way to copy SQLite databases between computers
-
Siren Call of SQLite on the Server
-
Stream Processing with DuckDB/Polars?
-
Limbo: A complete rewrite of SQLite in Rust
-
Using SQLite as Storage for Web Server Static Content
-
Introducing pgstream: Postgres replication with DDL changes
-
Pgstream – Postgres replication that includes schema definitions
-
A note from our sponsor - InfluxDB
www.influxdata.com | 17 May 2025
Index
What are some of the best open-source Replication projects in Go? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | seaweedfs | 24,296 |
| 2 | litestream | 11,984 |
| 3 | cloudnative-pg | 5,937 |
| 4 | marmot | 2,020 |
| 5 | BemiDB | 1,369 |
| 6 | adguardhome-sync | 1,146 |
| 7 | zrepl | 1,020 |
| 8 | olake | 852 |
| 9 | nebula-sync | 784 |
| 10 | pgstream | 523 |
| 11 | pglogrepl | 381 |
| 12 | pgcapture | 266 |
| 13 | go-craq | 140 |
| 14 | kuvasz-streamer | 134 |
| 15 | psql-streamer | 59 |
| 16 | kivi | 42 |
| 17 | rawkv | 11 |
| 18 | imsy | 10 |
| 19 | binlog | 9 |
Sponsored
InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com