Extending Python with Rust

• Numba

  124 9,432 9.9 Python

  NumPy aware dynamic Python compiler using LLVM

  

This blog article was missing the first step of using Numba on numpy https://numba.pydata.org/

• python-qubit-setup

  4 8 10.0 C++

  All scripts for controlling the instruments and acquiring data in our qubit setup.

A long time (10 years) ago I wrote some custom C++ code to perform relatively simple arithmetic operations on very large arrays, which was considerably slower in numpy for some reason [1]. As I said that was long ago so maybe Numpy has improved by now, but back then there was a significant penalty when doing these operations directly in Python using Numpy, even though they were only a handful of things (e.g. calculate asin(x)+bcos(y) on some large array), so I didn't expect much overhead from going back and forth between F.

1: https://github.com/adewes/python-qubit-setup/blob/master/lib...

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• greptimedb

  16 3,743 9.9 Rust

  An open-source, cloud-native, distributed time-series database with PromQL/SQL/Python supported. Available on GreptimeCloud.

  

This is truly a fantastic combination -- implement the logic in Rust and use it in Python. GreptimeDB also implements a similar functionality that allows writing Python script to do post-process of SQL query results, with the help of RustPython and Arrow. Maybe this combination can bring a sweet point between performance and efficiency.

docs: https://docs.greptime.com/user-guide/coprocessor-and-scripti...

code: https://github.com/GreptimeTeam/greptimedb/tree/develop/src/...

• julia

  350 44,510 10.0 Julia

  The Julia Programming Language

  

There are some nice tools for 3D PDEs which connect to DiffEq like GridAP (https://docs.sciml.ai/Gridap/stable/) and Ferrite (https://docs.sciml.ai/Ferrite/stable/). PDE tooling is where focus has been moving to as things evolve.

As for JIT, just today there was a PR that was merged that makes Julia cache and reuse binaries of packages (https://github.com/JuliaLang/julia/pull/47184). It won't be out until the next release of Julia, but it's a pretty major improvement to not JIT fully inferred package calls.

• NumPy

  272 26,290 10.0 Python

  The fundamental package for scientific computing with Python.

  

Definitely. However, the equivalent for the array multiplication in numpy looks like this:

https://github.com/numpy/numpy/blob/22e683d84f2584a6f9a57b2c... + https://github.com/numpy/numpy/blob/22e683d84f2584a6f9a57b2c...

To know what they do, you need the source for INPLACE_GIVE_UP_IF_NEEDED (https://github.com/numpy/numpy/blob/b222eb66c79b8eccba39f46f...) and PyArray_GenericInplaceBinaryFunction (I don't even know where that's coming from, it's not defined in Numpy, maybe it's part of the Python interface?).

In the end, both are unreadable in their own way. I personally prefer the Rust version above to the macrofied C version that's in Numpy but that's a matter of taste. I'd also trust the safe Rust implementations more than the C implementations because of the memory management guarantees Rust provides, though I suppose for simple operations like multiplication it'll be easier to make the program safe enough in C.

• graphics_wgpu

  1 2 2.8 Rust

I made my own, using WGPU and this basic graphics engine: https://github.com/David-OConnor/graphics_wgpu

Note that there's currently no docs or template/examples, and I'm rapidly breaking the API.

The surface plots are just meshes of a grid divided into triangles.

So, not a plotting lib at all; a flexible 3d and UI lib. Could probably made usable by others with a basic example of how you interact with the render and UI.

• progenitor

  1 1 8.3 Rust

  rule-based growth of hexagonal cells (by martinxyz)

Part of that is the ndarray crate, which IMO is too generic often, making syntax complicated. It's not going to get pythonic but the nalgebra crate is a bit nicer IMO, example:

https://github.com/martinxyz/progenitor/blob/85260/crates/pr...

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• numexpr

  4 2,138 8.3 Python

  Fast numerical array expression evaluator for Python, NumPy, Pandas, PyTables and more

  

• vswhere

  5 896 5.1 C++

  Locate Visual Studio 2017 and newer installations

  

Finding where & how to use an installed VS instance (or selecting one) in automated tooling is solved by the criminally unknown, MIT licensed, MS supported, redistributable, vswhere tool: https://github.com/microsoft/vswhere

• pygfx

  3 352 8.8 Python

  A python render engine running on wgpu.

  

Rather than using matplotlib, you could try either pygfx (https://github.com/pygfx/pygfx) or fastplotlib (https://github.com/kushalkolar/fastplotlib) to make higher performance graphics using Python.

However, it won't solve your problem of Python not being fast enough doing the calculations.

• fastplotlib

  2 324 8.9 Python

  Next-gen fast plotting library running on WGPU using the pygfx rendering engine

  

Rather than using matplotlib, you could try either pygfx (https://github.com/pygfx/pygfx) or fastplotlib (https://github.com/kushalkolar/fastplotlib) to make higher performance graphics using Python.

However, it won't solve your problem of Python not being fast enough doing the calculations.

Suggest a related project