libfringe VS rayon

There’s a bunch of library-based implementations of coroutines for rust. I recall https://github.com/edef1c/libfringe being the most interesting one, but it is quite dated. I don’t think there’s a lot of community interest in stackfull coroutines at this point.

As we saw earlier, scheduling threads is a hard task for the operating system. To replace one thread that's being executed with another one, a lot of work needs to be done (including saving all the registers and some thread state). However, switching between Lunatic Processes does only the minimal amount of work possible. With an idea pioneered by the libfringe library and using some asm! macro magic, Lunatic lets the Rust compiler figure out the minimal number of registers to be preserved during context switches. This makes scheduling Lunatic processes zero-cost. On my machine usually 1ns, equivalent to a function call.

On that topic, have you heard about our lord and savior rayon? 😊

> He spoke of the potential for functional languages to provide a significant, intrinsic advantage when it comes to parallel computing.

> (...)

> If that were true, you'd expect that the many existing functional programming languages would have already satisfied this need. But in my opinion, they haven't

Well there is https://futhark-lang.org/ - it runs on the GPU, and is awesome.

On the CPU side, I think that Rust plus https://github.com/rayon-rs/rayon was a huge breakthrough on writing parallel programs using both functional and imperative programming, and future languages should learn from its successes. The ownership system & the borrow checker, plus other type-level features like the Send and Sync traits, were essential to enable sharing read-only data between threads without synchronization, or sharing read-write data with synchronization, all checked at compile time for data races (which is a huge problem to solve, and is something that neither Go nor Java protects against at compile time)

Indeed Futhark shares a key feature with Rust: it uses uniqueness types to enable in-place updates, which is kind like a limited form of Rust ownership: if you are the sole user of some memory, you can update it and other code will be none the wiser. This kind of thing is very important to build functional programs that are performant in practice.

If you don't know Rust, but know Python, you can install Python libraries written in Rust with pip. Like, pip install polars or pip install robyn. In this case you follow the two bottom links. But then you don't write your own libraries and stuff so.. I guess that's not what you want.

But, if you want to learn Rust, you probably wouldn't start out with pyo3. You first install Rust with https://rustup.rs/ and then check out the official book, and the book rust by example, that you can find here https://www.rust-lang.org/learn - and maybe write some code on the Rust playground https://play.rust-lang.org/ - then, you use pyo3 to build Python libraries in Rust, and then use maturin https://www.maturin.rs/ to build and publish them to Pypi.

But if you still prefer to begin with Rust by writing Python libraries (it's a valid strategy if you are very comfortable with working with multiple stacks), the Maturin link has a tutorial that setups a program that is half written in python, half written in Rust, https://www.maturin.rs/tutorial.html (well the pyo3 link I sent also has one too. You should refer to the documentation of both, because you will use the two together)

After learning Rust, the next step is looking for libraries that you could leverage to make Python programs ultra fast. Here https://github.com/rayon-rs/rayon is an obvious choice, see some examples from the Rust cookbook https://rust-lang-nursery.github.io/rust-cookbook/concurrenc... - when you create a parallel iterator, it will distribute the processing to many threads (by default, one per core). The rust cookbook, by the way, is a nice reference to see the most used crates (Rust libraries) in the Rust ecosystem.

Anyway there are some posts about pyo3 on the web, like this blog post https://boring-guy.sh/posts/river-rust/ (note: it uses an outdated version of pyo3, and doesn't seem to use maturin which is a newer tool). This post was written by the developers of https://github.com/online-ml/river - another Python library written in Rust

My solution is just a bruteforce BFS, with some parallelism thrown in. The key idea for me was to realize you can skip building robots if you earn more than the cost of that material per round.

My algorithm is really bad. It tries to scan all y range 0..=4000000 in part2. However my program can solve part2 within 200ms thanks to rayon's parallel iterators and 20 CPU cores.

Not sure if there's anything in the standard library, but I recall this as the definitive library for data parallelism in Rust: https://github.com/rayon-rs/rayon

Related issue: #885

Instead, we use rayon, a data-parallelism library for Rust.

3680 in azul 147 in rayon 2 functions and 1 pattern match in ripgrep 25 in rust-crypto