CheeseShop
whatlang-pyo3
Our great sponsors
| CheeseShop | whatlang-pyo3 | |
|---|---|---|
| 2 | 2 | |
| 1 | 11 | |
| - | - | |
| 3.8 | 4.6 | |
| 7 months ago | 4 months ago | |
| Rust | Rust | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
CheeseShop
-
Apache Spark UDFs in Rust
By comparison, PyO3 handles virtually all that boilerplate, so your Rust functions can accept and return many native Rust types and everything just works (for example). Or maybe I'm missing some fundamental difference with how JVM data are handled versus Python.
-
PyO3: Rust Bindings for the Python Interpreter
At work, I'm using PyO3 for a project that churns through a lot of data (step 1) and does some pattern mining (step 2). This is the second generation of the project and is on-demand compared with the large, batch project in Spark that it is replacing. The Rust+Python project has really good performance, and using Rust for the core logic is such a joy compared with Scala or Python that a lot of other pieces are written in.
Learning PyO3, I cobbled together a sample project[0] to demonstrate how some functionality works. It's a little outdated (uses PyO3 0.11.0 compared with the current 0.13.1) and doesn't show everything, but I think it's reasonably clear.
One thing I noticed is that passing very large data from Rust and into Python's memory space is a bit of a challenge. I haven't quite grokked who owns what when and how memory gets correctly dropped, but I think the issues I've had are with the amount of RAM used at any moment and not with any memory leaks.
[0] https://github.com/aeshirey/CheeseShop
whatlang-pyo3
What are some alternatives?
ffi-overhead - comparing the c ffi (foreign function interface) overhead on various programming languages
whichlang - A blazingly fast and lightweight language detection library for Rust
dtparse - Fast datetime parser for Python written in Rust
erg - A statically typed language compatible with Python
rust-numpy - PyO3-based Rust bindings of the NumPy C-API
py3langid - Faster, modernized fork of the language identification tool langid.py
pythran - Ahead of Time compiler for numeric kernels
lingua-rs - The most accurate natural language detection library for Rust, suitable for short text and mixed-language text
rayon - Rayon: A data parallelism library for Rust
whatlang-rs - Natural language detection library for Rust. Try demo online: https://whatlang.org/
py2many - Transpiler of Python to many other languages
mamba - 🐍 The Mamba programming language, because we care about safety