Powerful document editing and collaboration in your app or environment. Ultimate security, API and 30+ ready connectors, SaaS or on-premises Learn more →
Top 9 Rust Dataframe Projects
Fast multi-threaded, hybrid-out-of-core DataFrame library in Rust | Python | Node.jsProject mention: Benchmarking for Pandas and Polars Using CSV and Parquet File | reddit.com/r/Python | 2023-05-15
I have updated this issue https://github.com/pola-rs/polars/issues/8533, please kindly help to solve it. I have also sent similar issues to Pandas https://github.com/pandas-dev/pandas/issues/53249
Apache Arrow DataFusion SQL Query EngineProject mention: DuckDB 0.8.0 | news.ycombinator.com | 2023-05-17
DuckDB is a great piece of software if you are
If you are looking for a query engine implemented in a safe language (Rust) I definitely suggest checking out DataFusion. It is comparable to DuckDB in performance, has all the standard built in SQL functionality, and is extensible in pretty much all areas (query language, data formats, catalogs, user defined functions, etc)
Disclaimer I am a maintainer of DataFusion
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
📺(tv) Tidy Viewer is a cross-platform CLI csv pretty printer that uses column styling to maximize viewer enjoyment.Project mention: Are there any TUI apps you recommend outside of ncdu / nnn / htop / vim / bat / fd / tig / duf? | reddit.com/r/commandline | 2022-10-12
I work with data a lot so I use the sqlite cli. I also made tv (self-promotion) to view csvs.
Fastest library to load data from DB to DataFrames in Rust and PythonProject mention: I used multiprocessing and multithreading at the same time to drop the execution time of my code from 155+ seconds to just over 2+ seconds | reddit.com/r/Python | 2023-05-29
There's packages like connector-x and polars that do a lot of what you're mentioning out of the box. I used these two to massively speed up an SQLalchemy + Pandas based ETL in the past as well.
Apache Arrow Ballista Distributed Query EngineProject mention: Evolution and Trends of Data Engineering 2022/23 | reddit.com/r/dataengineering | 2023-05-19
Ballista (Arrow-Rust), which is largely inspired by Apache Spark, there are some interesting differences.
Rust numeric library with R, MATLAB & Python syntaxProject mention: Hey Rustaceans! Got a question? Ask here! (39/2022)! | reddit.com/r/rust | 2022-09-26
Rust’s standard library is relatively small by design and doesn’t contain any tools for numeric integration. However, you can probably find a crate on crates.io that does what you need. A quick search suggests Peroxide.
Lightweight Apache Arrow data frame for RustProject mention: myval - lightweight Apache Arrow data frame for Rust | reddit.com/r/rust | 2023-04-21
Clean code begins in your IDE with SonarLint. Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.
A dataframe manipulation tool inspired by dplyr and powered by polars.Project mention: A dplyr interpreter powered by Polars | reddit.com/r/rust | 2023-05-10
I have added documentation for all supported functions here.
UX-Dataflow is a streaming capable data multiplexer that allows you to aggregate data and then process it using a Chain of Responsibility design pattern.
Rust Dataframe related posts
I used multiprocessing and multithreading at the same time to drop the execution time of my code from 155+ seconds to just over 2+ seconds
1 project | reddit.com/r/Python | 29 May 2023
Evolution and Trends of Data Engineering 2022/23
1 project | reddit.com/r/dataengineering | 19 May 2023
Polars CLI is now available!
2 projects | reddit.com/r/rust | 9 May 2023
Data Engineering with Rust
5 projects | reddit.com/r/rust | 9 May 2023
Polars query engine 0.29.0 released
3 projects | reddit.com/r/rust | 8 May 2023
Polars: Computing a new column from multiple columns - there must be a better way
1 project | reddit.com/r/rust | 4 May 2023
Pandas Dataframe alternative for update a fixed size dataframe without copying
1 project | reddit.com/r/learnpython | 19 Apr 2023
A note from our sponsor - ONLYOFFICE
www.onlyoffice.com | 1 Jun 2023
What are some of the best open-source Dataframe projects in Rust? This list will help you: