flume VS reqwest

I also like flume, it has impressive performance (although not the best). More importantly, it's written only with safe rust. https://github.com/zesterer/flume

The most commonly suggested replacement for mspc is crossbeam-channel; flume is also relatively popular.

It's not officially deprecated, but the alternatives on crates.io are considered better. flume and crossbeam-channel feature less unsafe code and offer better performance. Benchmarks.

it's slow (checkout flume's benchmarks for example)

For concurrency/parallelism, you launch at most 2 * CPU Cores, PIN them and use a fast broker to spread the task (like a ring buffer or an MPSC). But you keep linear scan, tight loops, SIMD friendly data, on each. You are not switching context that much, and instead, bet you will process the batch fast. (CPUs are fast today!)

First, I was becoming more and more confident that the channel library Daxpedda and I fell in love with, flume, was misbehaving, but I couldn't seem to reproduce it outside of the massive PliantDB codebase. I finally called up Daxpedda on Discord and screen shared my debugging session, showing him how the tests succeeded if I retained a channel. If I allowed the sender to drop after successfully sending, sometimes the tests would fail. He agreed, something was odd. It took me a while, but I finally whittled it down to about 30 lines of code and reported the issue. In an amazingly quick fashion, the maintainer fixed the issue and released an update. And for the record, I still fully love and recommend this library if you're mixing async and non-async code using channels. It's a wonderful implementation.

Sort of? Here's the closest thing you'll probably find: https://crates.io/crates/flume

My first try was with flume, which uses a single producer, multi-consumer setup, and while it compiles and runs, it's magnitudes slower than the serial version. Here's a psuedocode rendition:

ReductStore was rewritten from C++ to Rust a few months ago. We are delighted to be part of the Rust community and have taken a new step towards Rust with the Client SDK. The SDK is powered by reqwest and enables asynchronous integration of the database into Rust applications:

From what I could turn up when googling the specific error lines (here), it has something to do with the crate mio not having support for WASM, but I don’t understand what’s being said on this thread.

When starting out, we made some design decisions at the backend. The decision will allow us to independently test the service without interfering with the real application using a term called integration testing. We'll utilize two "dev" packages: reqwest and fake. Dev dependencies only get introduced into your application in development or during testing. In production, they are not included:

Rust allows me to mainly only run the application to confirm things work from a business perspective.

For people starting out building stuff in rust - understand that there is a distinction of async code and libraries and can lead to confusing compiler errors if you don't realize there is a distinction. It's simple in hindsight but did cause me to waste hours barking up the wrong trees at first. Other wise just learn about `match` and Result/Option types asap, they're fundamental.

https://github.com/http-rs/tide tide is great to create an http server / routes

https://github.com/djc/askama I use this to template out HTML and it checks all my boxes, dynamic data, passing in functions, control flow.

https://github.com/launchbadge/sqlx sql interface for a variety of backend, async safe.

https://github.com/seanmonstar/reqwest http client to make requests

Rust is amazing, don't let the initial few speed bumps discourage you - building real things with rust is no more challenging today than any other modern language stack.

Is there any this month in reqwest? I would like to show my interest in https://github.com/seanmonstar/reqwest/issues/39

Reqwest for HTTP connections

It works by fetching a crate's metadata from crates.io, downloading the sources using reqwest, uncompressing them using flate2, extracting them (using tar), and finally rendering a diff (using similar). In the UI you can switch between different versions of the crate to diff against and it also has a search bar to enter the crate name.

I am wondering this, too. Making a request is mostly about one line of code and as you said it is documented and the documentation refers to examples.

request

In total for scraping: reqwest, tokio, and scraper-main. Those are I use to get the scraping started.