Rust Data processing

Open-source Rust projects categorized as Data processing

Top 4 Rust Data processing Projects

Data processing
  • rust-ndarray

    ndarray: an N-dimensional array with array views, multidimensional slicing, and efficient operations

    Project mention: Some Reasons to Avoid Cython | news.ycombinator.com | 2023-09-22

    I would love some examples of how to do non-trivial data interop between Rust and Python. My experience is that PyO3/Maturin is excellent when converting between simple datatypes but conversions get difficult when there are non-standard types, e.g. Python Numpy arrays or Rust ndarrays or whatever other custom thing.

    Polars seems to have a good model where it uses the Arrow in memory format, which has implementations in Python and Rust, and makes a lot of the ndarray stuff easier. However, if the Rust libraries are not written with Arrow first, they become quite hard to work with. For example, there are many libraries written with https://github.com/rust-ndarray/ndarray, which is challenging to interop with Numpy.

    (I am not an expert at all, please correct me if my characterizations are wrong!)

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • mech

    🦾 Main repository for the Mech programming language. Start here!

    Project mention: Reactive Programming Without Functions | news.ycombinator.com | 2024-03-24

    There's also https://github.com/mech-lang/mech which is a sort of descendant of Eve https://witheve.com/ . That too seems to be getting close to hiatus. It's a bit of a shame since it seems like quite a nice paradigm for some stuff like GUIs, interactive stuff, and discrete event simulation, but I suppose the paradigm is both a bit obscure and different enough from everything else that it becomes a "boil the ocean" situation where one or a few people try and hack away but aren't really able to get much traction and eventually tired themselves out.

  • utah

    Dataframe structure and operations in Rust

  • ux-dataflow

    UX-Dataflow is a streaming capable data multiplexer that allows you to aggregate data and then process it using a Chain of Responsibility design pattern.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Rust Data processing discussion

Log in or Post with

Rust Data processing related posts

  • Some Reasons to Avoid Cython

    5 projects | news.ycombinator.com | 22 Sep 2023
  • Stream processing framework for a new project in Python

    1 project | /r/dataengineering | 2 Jun 2023
  • Helper crate for working with image data of varying type?

    1 project | /r/rust | 29 May 2023
  • A Python package for streaming synthetic data

    2 projects | /r/Python | 25 May 2023
  • What is the most efficient way to study Rust for scientific computing applications?

    1 project | /r/rust | 23 May 2023
  • Snowflake - what are the streaming capabilities it provides?

    3 projects | /r/dataengineering | 10 May 2023
  • faer 0.8.0 release

    6 projects | /r/rust | 21 Apr 2023
  • A note from our sponsor - InfluxDB
    www.influxdata.com | 23 Jul 2024
    Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

What are some of the best open-source Data processing projects in Rust? This list will help you:

Project Stars
1 rust-ndarray 3,437
2 mech 200
3 utah 142
4 ux-dataflow 9

Sponsored
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com

Did you konow that Rust is
the 5th most popular programming language
based on number of metions?