duckdb_fdw vs postgres-binary-parser

duckdb_fdw

DuckDB Foreign Data Wrapper for PostgreSQL (by alitrack)

Source Code

Suggest alternative

Edit details

postgres-binary-parser

Cython implementation of a parser for PostgreSQL's COPY WITH BINARY format (by spitz-dan-l)

Suggest topics

Source Code

Suggest alternative

Edit details

Our great sponsors

InfluxDB - Power Real-Time Data Analytics at Scale

WorkOS - The modern identity platform for B2B SaaS

SaaSHub - Software Alternatives and Reviews

Our great sponsors

duckdb_fdw		postgres-binary-parser
	Project
4	Mentions	2
235	Stars	6
-	Growth	-
7.3	Activity	0.0
2 months ago	Latest Commit	about 3 years ago
PLpgSQL	Language	Python
MIT License	License	-

The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.

duckdb_fdw

Posts with mentions or reviews of duckdb_fdw. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-09-30.

Querying Postgres Tables Directly from DuckDB
6 projects | news.ycombinator.com | 30 Sep 2022
Postgres and Parquet in the Data Lke
7 projects | news.ycombinator.com | 3 May 2022
DuckDB quacks Arrow: A zero-copy data integration between Arrow and DuckDB
1 project | news.ycombinator.com | 3 Dec 2021

I should also add that there is a duckdb fdw, so you could have DuckDB read from your parquet files and do faster transformations before you pull your data into Postgres!
https://github.com/alitrack/duckdb_fdw
DuckDB Postgres Foreign Data Wrapper
1 project | news.ycombinator.com | 20 Feb 2021

postgres-binary-parser

Posts with mentions or reviews of postgres-binary-parser. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-09-30.

Querying Postgres Tables Directly from DuckDB
6 projects | news.ycombinator.com | 30 Sep 2022

I was trying https://github.com/sfu-db/connector-x and hacking around with this https://github.com/spitz-dan-l/postgres-binary-parser but it turned out that a COPY to csv using asyncpg and then converting to parquet was the fastest.
Fastest way to read bytes as a scalar?
1 project | /r/learnpython | 10 Mar 2021

Have a look at this repo - might save you some work.

What are some alternatives?

When comparing duckdb_fdw and postgres-binary-parser you can also consider the following projects:

subzero-starter-kit - Starter Kit and tooling for authoring GraphQL/REST API backends with subZero

connector-x - Fastest library to load data from DB to DataFrames in Rust and Python

odbc2parquet - A command line tool to query an ODBC data source and write the result into a parquet file.

aquameta - Web development platform built entirely in PostgreSQL

postgres_vectorization_test - Vectorized executor to speed up PostgreSQL

parquet_fdw - Parquet foreign data wrapper for PostgreSQL

parquet_s3_fdw - ParquetS3 Foreign Data Wrapper for PostgresSQL

cstore_fdw - Columnar storage extension for Postgres built as a foreign data wrapper. Check out https://github.com/citusdata/citus for a modernized columnar storage implementation built as a table access method.

geoparquet - Specification for storing geospatial vector data (point, line, polygon) in Parquet