pqrs
duckdb_fdw
| pqrs | duckdb_fdw | |
|---|---|---|
| 4 | 4 | |
| 251 | 235 | |
| - | - | |
| 4.0 | 7.3 | |
| about 1 month ago | 2 months ago | |
| Rust | PLpgSQL | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pqrs
-
Querying Postgres Tables Directly from DuckDB
This won't let you ETL from PG to parquet, but I used this in anger the other day https://github.com/manojkarthick/pqrs Worked quite well for my purposes!
-
Holy Uck Fast Analysis with DuckDB and Pyarrow
Shameless plug, my project exposes the common Parquet operations using a Rust CLI tool using the Rust API for Apache Arrow and can be used without any Java/Hadoop/Spark dependencies. Also available a static binary.
https://github.com/manojkarthick/pqrs
- Pqrs: A Command line tool for inspecting Parquet files
- pqrs: A parquet-tools replacement in Rust using Apache Arrow
duckdb_fdw
- Querying Postgres Tables Directly from DuckDB
- Postgres and Parquet in the Data Lke
-
DuckDB quacks Arrow: A zero-copy data integration between Arrow and DuckDB
I should also add that there is a duckdb fdw, so you could have DuckDB read from your parquet files and do faster transformations before you pull your data into Postgres!
https://github.com/alitrack/duckdb_fdw
- DuckDB Postgres Foreign Data Wrapper
What are some alternatives?
Apache Parquet - Apache Parquet
subzero-starter-kit - Starter Kit and tooling for authoring GraphQL/REST API backends with subZero
deno - A modern runtime for JavaScript and TypeScript.
odbc2parquet - A command line tool to query an ODBC data source and write the result into a parquet file.
ballista - Distributed compute platform implemented in Rust, and powered by Apache Arrow.
aquameta - Web development platform built entirely in PostgreSQL
roapi - Create full-fledged APIs for slowly moving datasets without writing a single line of code.
postgres_vectorization_test - Vectorized executor to speed up PostgreSQL
parquet2 - Fastest and safest Rust implementation of parquet. `unsafe` free. Integration-tested against pyarrow
parquet_fdw - Parquet foreign data wrapper for PostgreSQL
rust - Rust for the xtensa architecture. Built in targets for the ESP32 and ESP8266
parquet_s3_fdw - ParquetS3 Foreign Data Wrapper for PostgresSQL