Writing Rust libraries for the Python scientific computing ecosystem

• maturin

  37 3,232 9.4 Rust

  Build and publish crates with pyo3, cffi and uniffi bindings as well as rust binaries as python packages

  

One of Rust's many strengths is that it can be seamlessly integrated with Python and speed up critical code sections. I recently wrote a small library with an efficient ragged array datatype, and I figured it would make for a good example of how to set up a Rust Python package with PyO3 and maturin that interoperates with numpy. There are a lot of little details that took me quite a while to figure out:

• PyO3

  147 10,997 9.8 Rust

  Rust bindings for the Python interpreter

  

One of Rust's many strengths is that it can be seamlessly integrated with Python and speed up critical code sections. I recently wrote a small library with an efficient ragged array datatype, and I figured it would make for a good example of how to set up a Rust Python package with PyO3 and maturin that interoperates with numpy. There are a lot of little details that took me quite a while to figure out:

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• polars

  144 26,043 10.0 Rust

  Dataframes powered by a multithreaded, vectorized query engine, written in Rust

  

All the problems you describe are thing we also encountered in writing polars. Take a look at the py-polars directory in root. I believe we have done all you listed. :)

• maturin-action

  2 116 7.8 TypeScript

  GitHub Action to install and run a custom maturin command with built-in support for cross compilation

  

Messense has written the great messense/maturin-action github action. It was already linked in the readme, I've moved it up in #751.

• ragged-buffer

  2 19 3.8 Rust

  Efficient numpy-like ragged array datatype for Python.

  

One of Rust's many strengths is that it can be seamlessly integrated with Python and speed up critical code sections. I recently wrote a small library with an efficient ragged array datatype, and I figured it would make for a good example of how to set up a Rust Python package with PyO3 and maturin that interoperates with numpy. There are a lot of little details that took me quite a while to figure out:

• rust-numpy

  10 1,015 6.7 Rust

  PyO3-based Rust bindings of the NumPy C-API

  

Integration with numpy uses the rust-numpy crate: Example of method that accepts numpy arrays as arguments Example of a method that returns a numpy array to Python (this performs a copy, there ought to be a way to avoid it but the current implementation has been plenty fast for my use case so far)

• pyron

  1 8 2.4 Rust

  Python bindings for the Rusty Object Notation.

Haha yes, debugging GitHub actions is fun. I've not yet made any serious attempts to get cross-compilation working.

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

Suggest a related project