glommio VS Seastar

I feel a bit paralyzed by Fear Of Missing Io_Uring. There's so much awesome streaming stuff about (RisingWave, Materialize, NATS, DataFusion, Velox, many more), but it all feels built on slower legacy system libraries.

It's not heavily used yet, but Rust has a bunch of fairly high visibility efforts. Situation sort of feels similar with http3, where the problem is figuring out what to pick. https://github.com/tokio-rs/tokio-uring https://github.com/bytedance/monoio https://github.com/DataDog/glommio

I used glommio as the async executor (instead of something like tokio), and it is wonderful. For people wondering whether it's "good enough" or to use C++ and seastar (as I have thought about a lot before starting this project), take the leap of faith, it's fast - both in terms of run time and to code.

My understanding is you always need a runtime, somethings needs to drive the async flow. But there are others on the market, just not without the.. market domination... of tokio.

https://github.com/smol-rs/smol looks promising simply for being minimal

https://github.com/bytedance/monoio looks potentially easier to work with than tokio

https://github.com/DataDog/glommio is built around linux io_uring and seems somewhat promising for performance reasons.

I haven't played with any of these yet, because Tokio is unfortunately the path of least resistance. And a bit viral in how it's infected tings.

I think you missed one which is based on io_uring [1].

In my benchmarks with a slightly tweaked version it was 2x faster than Nginx and and 30x faster than Python's SimpleHttpServer.

[1] https://github.com/DataDog/glommio/blob/master/examples/hype...

b) It's proven now e.g Seastar, Glommio that the fastest way to run a multi-threaded application is to have one instance with one thread pinned per CPU core. Then to have fibers/lightweight threads on top handling all of the asynchronous code. Your approach of lots of instances is the slowest so there will be a ton of unnecessary thread context-switching.

I assume Tokio itself, see e.g monoio or glommio, but also Seastar for C++.

https://github.com/DataDog/glommio Rust thread per core library.

> Few of us have really figured out io_uring. But that doesn't mean it is slower.

seastar.io is a high level framework that I believe has "figured out" io_uring, with additional caveats the framework imposes (which is honestly freeing).

Additionally the rust equivalent: https://github.com/DataDog/glommio

This use case is perfect for https://github.com/DataDog/glommio which is a thread-per-core runtime that is appropriate for latency sensitive code.

It's worth mentioning: Under "Async Executors", for "io_uring" there is only "Glommio"

I recently found out that ByteDance has a competitor library which supposedly has better performance:

https://github.com/bytedance/monoio

https://github.com/DataDog/glommio/issues/554

I used glommio as the async executor (instead of something like tokio), and it is wonderful. For people wondering whether it's "good enough" or to use C++ and seastar (as I have thought about a lot before starting this project), take the leap of faith, it's fast - both in terms of run time and to code.

b) It's proven now e.g Seastar, Glommio that the fastest way to run a multi-threaded application is to have one instance with one thread pinned per CPU core. Then to have fibers/lightweight threads on top handling all of the asynchronous code. Your approach of lots of instances is the slowest so there will be a ton of unnecessary thread context-switching.

The most common example is DPDK [1]. It's a framework for building bespoke networking stacks that are usable from userspace, without involving the kernel.

You'll find DPDK mentioned a lot in the networking/HPC/data center literature. An example of a backend framework that uses DPDK is the seastar framework [2]. Also, I recently stumbled upon a paper for efficient RPC networks in data centers [3].

If you want to learn more, the p99 conference by ScyllaDB has tons of speakers talking about some interesting challenges.

[1] https://www.dpdk.org/.

[2] https://github.com/scylladb/seastar

[3] https://github.com/erpc-io/eRPC

I assume Tokio itself, see e.g monoio or glommio, but also Seastar for C++.

https://core.dpdk.org/supported/ lists supported nics. You're best just reading material from the dpdk website for figuring out roughly what it is. It is used for a lot of different goals. For most web C++ stuff it's mainly used because you can avoid round trips of data passing through the kernel and can reference network data without tons of copying. For an example check out the SeaStar framework, https://seastar.io/, which is under the hood of ScyllaDB.

As this is a Kafka sub, this may be a good opportunity to mention that Redpanda is based on the same framework (seastar) as Scylla. The idea of sharding work to CPU cores turns out to apply very well to the Kafka data model, too!

Seastar which is a thread per core runtime written by the Scylla devs thats used in both Redpanda and Scylla as the underlying runtime. https://github.com/scylladb/seastar

ScyllaDB is, ironically, maybe one of the worst examples the author could have come up with for "abstraction" in the article.

If folks aren't familiar with their work/internal tech, go check out some of their repos like Seastar. They have some of the most talented systems programmers on the planet writing thin veneers over kernel and hardware API's to squeeze every ounce out of performance.

https://github.com/scylladb/seastar

I know it's beside the point, but I just had to share because I thought that was funny

I’ve seen frameworks for c++ (https://seastar.io/) and rust (https://github.com/actix/actix) which support what you’re describing out of the box.

If you're interested in scaling and asynchronous programming in c++ I highly recommend you investigate the SeaStar application framework. You wouldn't build a web service with SeaStar, rather you would build the infrastructure that you would use to build the web service on top of. https://github.com/scylladb/seastar