rayon
PyO3
Our great sponsors
rayon | PyO3 | |
---|---|---|
66 | 146 | |
10,082 | 10,791 | |
2.9% | 4.4% | |
9.0 | 9.8 | |
3 days ago | 2 days ago | |
Rust | Rust | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
rayon
- Too Dangerous for C++
-
What Are The Rust Crates You Use In Almost Every Project That They Are Practically An Extension of The Standard Library?
rayon: Async CPU runtime for parallelism.
-
Moving from Typescript and Langchain to Rust and Loops
In the quest for more efficient solutions, the ONNX runtime emerged as a beacon of performance. The decision to transition from Typescript to Rust was an unconventional yet pivotal one. Driven by Rust's robust parallel processing capabilities using Rayon and seamless integration with ONNX through the ort crate, Repo-Query unlocked a realm of unparalleled efficiency. The result? A transformation from sluggish processing to, I have to say it, blazing-fast performance.
-
The Rust I Wanted Had No Future
(see https://github.com/rayon-rs/rayon/tree/master/src/iter/plumbing)
-
General Recommendations: Should I Use Tree-sitter as the AST for the LSP I am developing?
Sequentially, generating tree-sitter AST for each file and querying for the links of each file takes around 2.3 seconds. However, I randomly remembered this crate rayon, and I decided to test it. It ended up improving the performance (just by changing 2 lines of code) to 200-300ms by parallelizing the iterators and tree-sitter queries. MAJOR.
-
python to rust migration
Now if you really want to use Rust, you can rewrite only the part that are slowing down your consumer. It's easy by using Py03 and maturin. Maybe also rayon to parallelize.
-
AI learns to play flappy bird (code in comments)
Maybe rayon could make some loops there and there faster if needed.
-
Making Python 100x faster with less than 100 lines of Rust
One of the others understated pros of rewriting some parts in Rust, it's that you can parallelize easily with Rayon[0]
-
Trying to learn by tutorials, for cannot find a single Actix/Diesel tutorial that actually compiles
On that topic, have you heard about our lord and savior rayon? 😊
-
Brett Slatkin: Why am I building a new functional programming language?
> He spoke of the potential for functional languages to provide a significant, intrinsic advantage when it comes to parallel computing.
> (...)
> If that were true, you'd expect that the many existing functional programming languages would have already satisfied this need. But in my opinion, they haven't
Well there is https://futhark-lang.org/ - it runs on the GPU, and is awesome.
On the CPU side, I think that Rust plus https://github.com/rayon-rs/rayon was a huge breakthrough on writing parallel programs using both functional and imperative programming, and future languages should learn from its successes. The ownership system & the borrow checker, plus other type-level features like the Send and Sync traits, were essential to enable sharing read-only data between threads without synchronization, or sharing read-write data with synchronization, all checked at compile time for data races (which is a huge problem to solve, and is something that neither Go nor Java protects against at compile time)
Indeed Futhark shares a key feature with Rust: it uses uniqueness types to enable in-place updates, which is kind like a limited form of Rust ownership: if you are the sole user of some memory, you can update it and other code will be none the wiser. This kind of thing is very important to build functional programs that are performant in practice.
PyO3
- Polars – A bird's eye view of Polars
-
In Rust for Python: A Match from Heaven
This story unfolds as a captivating journey where the agile Flounder, representing the Python programming language, navigates the vast seas of coding under the wise guidance of Sebastian, symbolizing Rust. Central to their adventure are three powerful tridents: cargo, PyO3, and maturin.
- Segunda linguagem
-
Calling Rust from Python
I would not recommend FFI + ctypes. Maintaining the bindings is tedious and error-prone. Also, Rust FFI/unsafe can be tricky even for experienced Rust devs.
Instead PyO3 [1] lets you "write a native Python module in Rust", and it works great. A much better choice IMO.
-
Python 3.12
Same w/ Rust and Python, this is really neat because now each thread could have a GIL without doing exactly what you said. The pyO3 commit to allow subinterpreters was merged 21 days ago, so this might "just work" today: https://github.com/PyO3/pyo3/pull/3446
-
Removing Garbage Collection from the Rust Language (2013)
I expected someone to write a rust-based scripting language which tightly integrated with rust itself.
In reality, it seems like the python developers and toolchain are embracing rust enough to reduce the benefits to a new alternative.
-
Bytewax: Stream processing library built using Python and Rust
Hey HN! I am one of the people working on Bytewax. Bytewax came out of our experience working with ML infrastructure at GitHub. We wanted to use Python because we could move fast, the team was very fluent in it, and the rest of our tooling was Python-native already. We didn't want to introduce JVM-based solutions into our stack because of the lack of experience and the friction we had trying to get Python-centric tooling working with existing solutions like Flink.
In our research, we found Timely Dataflow (https://timelydataflow.github.io/timely-dataflow/, https://news.ycombinator.com/item?id=24837031) and the Naiad project (https://www.microsoft.com/en-us/research/project/naiad/) as well as PyO3 (https://github.com/PyO3/pyo3) and we thought we found a match made in heaven :). Bytewax leverages both of these projects and builds on them to provide a clean API (at least we think so) and table stakes features like connectors, state recovery, and cloud-native scaling. It has been really cool to learn about the dataflow computation model, Rust, and how to wrangle the GIL with Rust and Python :P.
Would love to get your feedback :).
`pip install bytewax` to get started. We have a page of guides (https://www.bytewax.io/guides) with ready-to-run examples.
-
Tell HN: Rust Is the Superglue
You can practice your Rust skills by writing performant and/or gluey extensions for higher-level language such as NodeJS (checkout napi-rs) and Python or complementing JS in the browser if you target Webassembly.
For instance, checkout Llama-node https://github.com/Atome-FE/llama-node for an involved Rust-based NodeJS extension. Python has PyO3, a Rust-Python extension toolset: https://github.com/PyO3/pyo3.
They can help you leverage your Rust for writing cool new stuff.
-
Writing Python Like Rust
(2020).
Things have arguably become even nicer (although slightly more divergent between the two) since then: Python's `Optional[T]` can now be written as `T | None`, and the core container types can now be annotated directly (e.g. `List[T]` becomes `list[T]`).
Combined via pyO3[1], Python and Rust are a real joy to write together.
- 🚀 GoRules Zen Engine: Rules Engine for Node.js
What are some alternatives?
crossbeam - Tools for concurrent programming in Rust
tokio - A runtime for writing reliable asynchronous applications with Rust. Provides I/O, networking, scheduling, timers, ...
rust-cpython - Rust <-> Python bindings
pybind11 - Seamless operability between C++11 and Python
RxRust - The Reactive Extensions for the Rust Programming Language
RustPython - A Python Interpreter written in Rust
rust-numpy - PyO3-based Rust bindings of the NumPy C-API
milksnake - A setuptools/wheel/cffi extension to embed a binary data in wheels
bincode - A binary encoder / decoder implementation in Rust.
tokio-rayon - Mix async code with CPU-heavy thread pools using Tokio + Rayon
uniffi-rs - a multi-language bindings generator for rust
wasmtime - A fast and secure runtime for WebAssembly