pqrs
postgres-binary-parser
pqrs | postgres-binary-parser | |
---|---|---|
4 | 2 | |
251 | 6 | |
- | - | |
4.0 | 0.0 | |
about 1 month ago | about 3 years ago | |
Rust | Python | |
Apache License 2.0 | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
pqrs
-
Querying Postgres Tables Directly from DuckDB
This won't let you ETL from PG to parquet, but I used this in anger the other day https://github.com/manojkarthick/pqrs Worked quite well for my purposes!
-
Holy Uck Fast Analysis with DuckDB and Pyarrow
Shameless plug, my project exposes the common Parquet operations using a Rust CLI tool using the Rust API for Apache Arrow and can be used without any Java/Hadoop/Spark dependencies. Also available a static binary.
https://github.com/manojkarthick/pqrs
- Pqrs: A Command line tool for inspecting Parquet files
- pqrs: A parquet-tools replacement in Rust using Apache Arrow
postgres-binary-parser
-
Querying Postgres Tables Directly from DuckDB
I was trying https://github.com/sfu-db/connector-x and hacking around with this https://github.com/spitz-dan-l/postgres-binary-parser but it turned out that a COPY to csv using asyncpg and then converting to parquet was the fastest.
-
Fastest way to read bytes as a scalar?
Have a look at this repo - might save you some work.
What are some alternatives?
Apache Parquet - Apache Parquet
connector-x - Fastest library to load data from DB to DataFrames in Rust and Python
deno - A modern runtime for JavaScript and TypeScript.
ballista - Distributed compute platform implemented in Rust, and powered by Apache Arrow.
roapi - Create full-fledged APIs for slowly moving datasets without writing a single line of code.
parquet2 - Fastest and safest Rust implementation of parquet. `unsafe` free. Integration-tested against pyarrow
rust - Rust for the xtensa architecture. Built in targets for the ESP32 and ESP8266
raku-Dan - Top level raku Data ANalysis Module
alacritty - A cross-platform, OpenGL terminal emulator.
skar - Ethereum (EVM) indexer using Apache Arrow/Parquet
duckdb - DuckDB is an in-process SQL OLAP Database Management System