Rust Data processing

Open-source Rust projects categorized as Data processing

Top 5 Rust Data processing Projects

  • rust-ndarray

    ndarray: an N-dimensional array with array views, multidimensional slicing, and efficient operations

    Project mention: How does explicit unrolling differ from iterating through elements one-by-one? (ndarray example) | reddit.com/r/rust | 2023-01-13

    While looking through ndarrays src, I came across a set of functions that explicitly unroll 8 variables on each iteration of a loop, with the comment eightfold unrolled so that floating point can be vectorized (even with strict floating point accuracy semantics). I don't understand why floats would be affected by unrolling, and in general I'm confused as to how explicit unrolling differs from iterating through each element one by one. I assumed this would be a scenario where the compiler would optimize best anyway, which seems to be confirmed (at least in the context of using iter() rather than for) here. Could anyone give a little context into what this, or any explicit unrolling achieves?

  • bytewax

    Python Stream Processing

    Project mention: “Streaming” data processing using asynchronous generators in python | reddit.com/r/dataengineering | 2023-02-04

    This the approach used for streaming data with bytewax. Some examples on the GitHub repo.

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • mech

    🦾 Main repository for the Mech programming language. Start here!

    Project mention: What features would you want in a new programming language? | reddit.com/r/ProgrammingLanguages | 2023-01-03

    You should take a look at the language I’m developing, Mech: https://github.com/mech-lang/mech

  • utah

    Dataframe structure and operations in Rust

  • ux-dataflow

    UX-Dataflow is a streaming capable data multiplexer that allows you to aggregate data and then process it using a Chain of Responsibility design pattern.

  • InfluxDB

    Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Platform where developers build real-time applications for analytics, IoT and cloud-native services. Easy to start, it is available in the cloud or on-premises.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-02-04.

Rust Data processing related posts

Index

What are some of the best open-source Data processing projects in Rust? This list will help you:

Project Stars
1 rust-ndarray 2,594
2 bytewax 323
3 mech 176
4 utah 137
5 ux-dataflow 7
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com