postgres-binary-parser vs duckdb_fdw

postgres-binary-parser

Cython implementation of a parser for PostgreSQL's COPY WITH BINARY format (by spitz-dan-l)

Suggest topics

Source Code

Suggest alternative

Edit details

duckdb_fdw

DuckDB Foreign Data Wrapper for PostgreSQL (by alitrack)

duckdb foreign-data-wrapper Postgresql duckdb-fdw fdw

Source Code

Suggest alternative

Edit details

InfluxDB - Power Real-Time Data Analytics at Scale

Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

www.influxdata.com

featured

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

featured

postgres-binary-parser		duckdb_fdw
	Project
2	Mentions	4
6	Stars	247
-	Growth	-
0.0	Activity	7.3
about 3 years ago	Latest Commit	3 months ago
Python	Language	PLpgSQL
-	License	MIT License

The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.

postgres-binary-parser

Posts with mentions or reviews of postgres-binary-parser. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-09-30.

Querying Postgres Tables Directly from DuckDB
6 projects | news.ycombinator.com | 30 Sep 2022

I was trying https://github.com/sfu-db/connector-x and hacking around with this https://github.com/spitz-dan-l/postgres-binary-parser but it turned out that a COPY to csv using asyncpg and then converting to parquet was the fastest.
Fastest way to read bytes as a scalar?
1 project | /r/learnpython | 10 Mar 2021

Have a look at this repo - might save you some work.

duckdb_fdw

Posts with mentions or reviews of duckdb_fdw. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2022-09-30.

Querying Postgres Tables Directly from DuckDB
6 projects | news.ycombinator.com | 30 Sep 2022
Postgres and Parquet in the Data Lke
7 projects | news.ycombinator.com | 3 May 2022
DuckDB quacks Arrow: A zero-copy data integration between Arrow and DuckDB
1 project | news.ycombinator.com | 3 Dec 2021

I should also add that there is a duckdb fdw, so you could have DuckDB read from your parquet files and do faster transformations before you pull your data into Postgres!
https://github.com/alitrack/duckdb_fdw
DuckDB Postgres Foreign Data Wrapper
1 project | news.ycombinator.com | 20 Feb 2021

What are some alternatives?

When comparing postgres-binary-parser and duckdb_fdw you can also consider the following projects:

connector-x - Fastest library to load data from DB to DataFrames in Rust and Python

subzero-starter-kit - Starter Kit and tooling for authoring GraphQL/REST API backends with subZero

odbc2parquet - A command line tool to query an ODBC data source and write the result into a parquet file.

aquameta - Web development platform built entirely in PostgreSQL

postgres_vectorization_test - Vectorized executor to speed up PostgreSQL

parquet_fdw - Parquet foreign data wrapper for PostgreSQL

parquet_s3_fdw - ParquetS3 Foreign Data Wrapper for PostgresSQL

cstore_fdw - Columnar storage extension for Postgres built as a foreign data wrapper. Check out https://github.com/citusdata/citus for a modernized columnar storage implementation built as a table access method.

geoparquet - Specification for storing geospatial vector data (point, line, polygon) in Parquet