conduit
Conduit streams data between data stores. Kafka Connect replacement. No JVM required. (by ConduitIO)
sqlpipe
SQLpipe makes it easy to move the result of one query from one database to another. (by sqlpipe)
| conduit | sqlpipe | |
|---|---|---|
| 7 | 9 | |
| 348 | 61 | |
| 3.2% | - | |
| 9.5 | 7.0 | |
| about 8 hours ago | about 2 months ago | |
| Go | PLpgSQL | |
| Apache License 2.0 | GNU General Public License v3.0 or later |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
conduit
Posts with mentions or reviews of conduit.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-04-30.
-
Pulling CDC data from Postgres
I'd like to mention Conduit + its Postgres connector. The Pg connector comes built-in, so all that is needed is a single Conduit binary to get started. It relies on WAL, but the connector creates the replication slot itself (if needed).
-
How to connect already setup kafka cluster to mongodb?
GitHub - ConduitIO/Conduit: Data Integration for Production Data Stores. Conduit is meant to be a bit more general-purpose than Kafka Connect and is an easy drop-in replacement. We're working hard to make that even easier. We are still in the early stages of this project and are trying to build more and more connectors. You can find out more about our connector roadmap on the Github repo. Our connector philosophy is to be real-time first, and double down on change data capture (CDC) capabilities, all with permissive licensing.
-
What services you guys used for CDC (Change Data capture) for Sql as well as no sql databases ?
If you're looking for a tool with a UI and in which you can also easily extend the functionality with your own, custom data connectors, you might also want take a look at Conduit which is another open-source tool we've developed to make building and running real-time data infrastructure more straightforward and less time consuming.
-
Alternative Kafka Integration Framework to Kafka Connect?
You might want to check out: https://github.com/conduitio/conduit
-
Where is the modern data stack for software engineers?
This is why we are working on a project called Conduit at Meroxa. We hope to change the experience software engineers have with data.
- Conduit: Data Integration for Production Data Stores
- Conduit: Data Integration Tool for Production Data Stores written in Go
sqlpipe
Posts with mentions or reviews of sqlpipe.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2022-11-14.
- [Postgresql] Détecter les changements de schéma via les journaux WAL
-
tasq - my first published open-source Go module
I run sqlpipe.com. It's a relatively new company that builds open source data engineering / developer tools in Go. The site is seeing over 1,000 unique visitors per month.
-
How to successfully transition from principal engineer to consultant?
My product is SQLpipe.
-
Any open source projects need help ?
Hi, I maintain an open-source data movement tool called SQLpipe. The repo is here: github.com/sqlpipe/sqlpipe.
-
Best ‘E’TL tools for extracting data from on-prem SQL databases
Hi, this is exactly what my product SQLpipe does. It runs SQL against a source, and transfers the result to a target.
-
Sync Snowflake to PostgreSQL
Check it out at https://sqlpipe.com
-
Do you see a place for custom ETL software?
Background: I built an open-source ETL tool called SQLpipe and am trying to come up with "win-win" ways to monetize it.
-
Detecting schema changes through WAL logs
I'm building an open-source data engineering tool (SQLpipe), and am now trying to incorporate WAL CDC into the product.
-
Options for doing moves of PostgreSQL to SQL Server
My open-source tool SQLpipe might be a good fit for this job. It is available for Windows, and can definitely be run by scheduled job.
What are some alternatives?
When comparing conduit and sqlpipe you can also consider the following projects:
turbine-go - Turbine Library for Go
devtron - Tool integration platform for Kubernetes
dozer - Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks.
tasq - Database based task queue implemented in Go
bytewax - Python Stream Processing
airbyte - The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
Benthos - Fancy stream processing made operationally mundane
Gitea - Git with a cup of tea! Painless self-hosted all-in-one software development service, including Git hosting, code review, team collaboration, package registry and CI/CD
deprecated-core - 🔮 Instill Core contains components for supporting Instill VDP and Instill Model
pond - 🔘 Minimalistic and High-performance goroutine worker pool written in Go
conduit-connector-postgres - Conduit connector for PostgreSQL
bubbletea - A powerful little TUI framework 🏗