Top 5 Rust Data processing Projects

rust-ndarray

7 2,594 7.5 Rust

ndarray: an N-dimensional array with array views, multidimensional slicing, and efficient operations

Project mention: How does explicit unrolling differ from iterating through elements one-by-one? (ndarray example) | reddit.com/r/rust | 2023-01-13

While looking through ndarrays src, I came across a set of functions that explicitly unroll 8 variables on each iteration of a loop, with the comment eightfold unrolled so that floating point can be vectorized (even with strict floating point accuracy semantics). I don't understand why floats would be affected by unrolling, and in general I'm confused as to how explicit unrolling differs from iterating through each element one by one. I assumed this would be a scenario where the compiler would optimize best anyway, which seems to be confirmed (at least in the context of using iter() rather than for) here. Could anyone give a little context into what this, or any explicit unrolling achieves?
bytewax

6 323 9.6 Rust

Python Stream Processing

Project mention: “Streaming” data processing using asynchronous generators in python | reddit.com/r/dataengineering | 2023-02-04

This the approach used for streaming data with bytewax. Some examples on the GitHub repo.
SonarQube

www.sonarqube.org
sponsored

Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
mech

4 176 8.7 Rust

🦾 Main repository for the Mech programming language. Start here!

Project mention: What features would you want in a new programming language? | reddit.com/r/ProgrammingLanguages | 2023-01-03

You should take a look at the language I’m developing, Mech: https://github.com/mech-lang/mech
utah

0 137 0.0 Rust

Dataframe structure and operations in Rust
ux-dataflow

0 7 0.0 Rust

UX-Dataflow is a streaming capable data multiplexer that allows you to aggregate data and then process it using a Chain of Responsibility design pattern.
InfluxDB

www.influxdata.com
sponsored

Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-02-04.