Top 7 Go data-integration Projects

cloudquery

102 5,581 10.0 Go

The open source high performance ELT framework powered by Apache Arrow

Project mention: We might want to regularly keep track of how important each server is | news.ycombinator.com | 2024-02-06

Check out CloudQuery - https://github.com/cloudquery/cloudquery for an easy cloud asset inventory.

Rudderstack

83 3,919 9.8 Go

Privacy and Security focused Segment-alternative, in Golang and React

Project mention: Rudderstack Switches to Elastic License | news.ycombinator.com | 2023-09-08

InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
incubator-devlake

10 2,424 9.9 Go

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
transfer

7 527 9.4 Go

Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.

Project mention: Migrate mongodb Datawarehouse to snowflake | /r/snowflake | 2023-12-04

conduit

7 342 9.4 Go

Conduit streams data between data stores. Kafka Connect replacement. No JVM required. (by ConduitIO)

Project mention: Pulling CDC data from Postgres | /r/dataengineering | 2023-04-30

I'd like to mention Conduit + its Postgres connector. The Pg connector comes built-in, so all that is needed is a single Conduit binary to get started. It relies on WAL, but the connector creates the replication slot itself (if needed).

Dataplane

1 182 8.3 Go

Dataplane is a data platform that makes it easy to construct a data mesh with automated data pipelines and workflows.
plugin-sdk

1 20 9.8 Go

CloudQuery Go SDK for source and destination plugins
WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).