connectors VS airbyte

Compare connectors vs airbyte and see what are their differences.

connectors

Connectors for capturing data from external data sources (by estuary)

airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted. (by airbytehq)
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
connectors airbyte
3 139
33 14,217
- 3.5%
9.9 10.0
2 days ago about 11 hours ago
Go Python
GNU General Public License v3.0 or later GNU General Public License v3.0 or later
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.

connectors

Posts with mentions or reviews of connectors. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-09-22.
  • All the ways to capture changes in Postgres
    12 projects | news.ycombinator.com | 22 Sep 2023
    No. We implemented our own [1] for a few reasons:

    * Scaling well to multi-TB DBs without pinning the write-ahead log (potentially filling your DB's disk) while the backfill is happening. Instead, our connector constantly reads the WAL and works well in setups like Supabase that have very restrictive WAL sizes (1GB iirc).

    * Incremental fault-tolerant backfills that can be stopped and resumed at will.

    * Being able to offer "precise" captures which are logically consistent in terms of the sequence of create/update/delete events.

    The last one becomes really interesting when paired with REPLICA IDENTITY FULL, because you can feed it into an incremental computation (perhaps differential dataflow) for streaming updates of a continuous computation.

    Our work is based off of the Netflix DBLog paper, which we took and ran with.

    [1] https://github.com/estuary/connectors/tree/main/source-postg...

  • Why would you ever not use CDC for ELT?
    1 project | /r/dataengineering | 7 May 2023
    Our connectors themselves are fully OSS (for example, here's PostgreSQL)
  • What Is Dbt and Why Are Companies Using It?
    3 projects | news.ycombinator.com | 2 Dec 2021
    We've used https://github.com/estuary/connectors/pkgs/container/source-... to load data sets in the many terabytes. Caveat that, while it's implemented to Airbyte's spec, we've only used it with Flow.

airbyte

Posts with mentions or reviews of airbyte. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-10-02.

What are some alternatives?

When comparing connectors and airbyte you can also consider the following projects:

walex - Postgres change events (CDC) in Elixir

Airflow - Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

temporal_tables - Temporal Tables PostgreSQL Extension

dagster - An orchestration platform for the development, production, and observation of data assets.

pg-event-proxy-example - Send NOTIFY and WAL events from PostgreSQL to upstream services (amqp / redis / mqtt)

Prefect - The easiest way to build, run, and monitor data pipelines at scale.

temporal_tables - Postgresql temporal_tables extension in PL/pgSQL, without the need for external c extension.

meltano

maxwell - Maxwell's daemon, a mysql-to-json kafka producer

jitsu - Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days

debezium - Change data capture for a variety of databases. Please log issues at https://issues.redhat.com/browse/DBZ.

spark-rapids - Spark RAPIDS plugin - accelerate Apache Spark with GPUs