Rayon Alternatives
Similar projects and alternatives to rayon
-
tokio
A runtime for writing reliable asynchronous applications with Rust. Provides I/O, networking, scheduling, timers, ...
-
-
-
-
-
sqlx
🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, SQLite, and MSSQL. (by launchbadge)
-
libfringe
a Rust library implementing safe, lightweight context switches, without relying on kernel services
-
-
Scout APM
Scout APM: A developer's best friend. Try free for 14-days. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
-
-
maturin
Build and publish crates with pyo3, rust-cpython and cffi bindings as well as rust binaries as python packages
-
Bus Writer
Single-reader, multi-writer & single-reader, multi-verifier; broadcasts reads to multiple writeable destinations in parallel
-
-
tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
-
-
tokio-rayon
Mix async code with CPU-heavy thread pools using Tokio + Rayon (by andybarron)
-
-
-
-
actix-web
Actix Web is a powerful, pragmatic, and extremely fast web framework for Rust.
-
-
Reviews and mentions
-
Is there an inverse to the Iterator trait?
From what you said in the other post, it sounds like something that would work using iterators. You don't necessarily need to be pushing into a sink. Have you looked at Rayon? Seems like you could just stream your items through Rayon iterators and get it parallelized with very little extra effort.
-
Use `.skip()` on a `rayon::iter::Flatten`
A relevant issue: https://github.com/rayon-rs/rayon/issues/210
-
Python stands to lose its GIL, and gain a lot of speed
I'm looking forward to something like the brilliant rayon parallel operations library https://github.com/rayon-rs/rayon
We won't get data race free guarantees but if built into pandas or Vaex we can have a near transparent API.
It'll really open things up for those apps running on 32 core machines. They're out there, I deploy these things frequently (Plotly Dash framework for large Enterprise customers).
-
Rust for Data Science: Tutorial 1
Additionally, ndarray has got also some nice extra, such as support for rayon for parallelization, or the popular BLAS low-level specs, through one of the working back-ends (using blas-src ).
-
Rust: First Thoughts
With Rust, thread-safe code become much easier to write, which follows from Rust’s core principle of ownership (In short, you can only have one mutable reference to any given piece of data). Simple map and reduce operations can be parallelized with great ease by dropping in Rayon. For other use cases you can wrap your code in an Arc (Atomic reference counter) and proceed a bit more manually. It’s still possible to shoot your self in the foot, but Rust protects you from many mistakes at compile time. I was able to parallelize some geospatial operations and graph traversal in a period of hours thanks to Rust’s robust tools for concurrency and parallelism.
-
Is there an asynchronous Hashmap or equivalent local DB?
Note: What about the approach of using a thread poll, do we even need async? While a thread poll would provide an improvement over your current approach (and is easy to scale using rayon) in this situation it would be sub-optimal if the round trip time (RTT) for each request is long. There is a portion of time where your computer is doing nothing, e.g. as the request gets routed to S3 and as S3 gathers the data to send back, your computer is sitting idle. But if you use an async approach, during this idle time, your computer can send more requests.
-
The Sad Truth About C++ Copy Elision
A more practical example is the rayon library which lets you write parallelized multithreaded code simply by turning this:
-
tokio-rayon: Mix Tokio async and CPU-bound work using Rayon thread pools
FWIW, I prototyped rayon#679 to spawn futures, but we didn't come to consensus on the design, and I'm didn't have enough interest to keep pushing on it. That was roughly a hand-rolled one-shot channel, but it looks a lot nicer that you used a real channel.
-
Hey Rustaceans! Got an easy question? Ask here (12/2021)!
You might want to look at Rayon instead, which is designed for compute-heavy parallelism. If your algorithm can be expressed using iterators, it's likely pretty straightforward to parallelize it with Rayon. Otherwise, you might look at rayon::join() which you can call recursively.
-
World’s First 1k-Processor Chip, Powered by a Single AA Battery (2016)
Rust's Rayon library is pretty good for these use-cases https://github.com/rayon-rs/rayon
use rayon::prelude::*; - rayon-rs/rayon data-parallelism library for Rust
- Rayon: A data parallelism library for Rust
- PyO3: Rust Bindings for the Python Interpreter
-
Tokio 1.0.0 has been released!
I think it should really only be used to import traits and macros into scope. Crates rayon and tide are good examples I look at frequently.
-
Rusticles #19 - Wed Nov 11 2020
rayon-rs/rayon (Rust): Rayon: A data parallelism library for Rust
Stats
rayon-rs/rayon is an open source project licensed under GNU General Public License v3.0 or later which is an OSI approved license.
Popular Comparisons
Are you hiring? Post a new remote job listing for free.