Seastar
abseil-cpp
Our great sponsors
Seastar | abseil-cpp | |
---|---|---|
25 | 54 | |
7,987 | 13,878 | |
1.2% | 2.1% | |
9.7 | 9.5 | |
4 days ago | 7 days ago | |
C++ | C++ | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Seastar
-
I want to share my latest hobby project, dbeel: A distributed thread-per-core nosql db written in rust
I used glommio as the async executor (instead of something like tokio), and it is wonderful. For people wondering whether it's "good enough" or to use C++ and seastar (as I have thought about a lot before starting this project), take the leap of faith, it's fast - both in terms of run time and to code.
-
How much reason is there to be multi-threaded in the k8s environment
b) It's proven now e.g Seastar, Glommio that the fastest way to run a multi-threaded application is to have one instance with one thread pinned per CPU core. Then to have fibers/lightweight threads on top handling all of the asynchronous code. Your approach of lots of instances is the slowest so there will be a ton of unnecessary thread context-switching.
-
Are You Sure You Want to Use MMAP in Your Database Management System?
The most common example is DPDK [1]. It's a framework for building bespoke networking stacks that are usable from userspace, without involving the kernel.
You'll find DPDK mentioned a lot in the networking/HPC/data center literature. An example of a backend framework that uses DPDK is the seastar framework [2]. Also, I recently stumbled upon a paper for efficient RPC networks in data centers [3].
If you want to learn more, the p99 conference by ScyllaDB has tons of speakers talking about some interesting challenges.
-
Why does Actix-web's handler not require Send?
I assume Tokio itself, see e.g monoio or glommio, but also Seastar for C++.
-
What is DPDK library in C and how to learn it?
https://core.dpdk.org/supported/ lists supported nics. You're best just reading material from the dpdk website for figuring out roughly what it is. It is used for a lot of different goals. For most web C++ stuff it's mainly used because you can avoid round trips of data passing through the kernel and can reference network data without tons of copying. For an example check out the SeaStar framework, https://seastar.io/, which is under the hood of ScyllaDB.
-
How Numberly Replaced Kafka with a Rust-Based ScyllaDB Shard-Aware Application
As this is a Kafka sub, this may be a good opportunity to mention that Redpanda is based on the same framework (seastar) as Scylla. The idea of sharding work to CPU cores turns out to apply very well to the Kafka data model, too!
-
What are some C++ projects with high quality code that I can read through?
Seastar which is a thread per core runtime written by the Scylla devs thats used in both Redpanda and Scylla as the underlying runtime. https://github.com/scylladb/seastar
-
Abstraction Is Expensive
ScyllaDB is, ironically, maybe one of the worst examples the author could have come up with for "abstraction" in the article.
If folks aren't familiar with their work/internal tech, go check out some of their repos like Seastar. They have some of the most talented systems programmers on the planet writing thin veneers over kernel and hardware API's to squeeze every ounce out of performance.
https://github.com/scylladb/seastar
I know it's beside the point, but I just had to share because I thought that was funny
-
Modern JVM Multithreading • Paweł Jurczenko • Devoxx Poland 2021
I’ve seen frameworks for c++ (https://seastar.io/) and rust (https://github.com/actix/actix) which support what you’re describing out of the box.
-
Who is using C++ for web development?
If you're interested in scaling and asynchronous programming in c++ I highly recommend you investigate the SeaStar application framework. You wouldn't build a web service with SeaStar, rather you would build the infrastructure that you would use to build the web service on top of. https://github.com/scylladb/seastar
abseil-cpp
- Sane C++ Libraries
- Open source collection of Google's C++ libraries
- Is Ada safer than Rust?
-
Appending to an std:string character-by-character: how does the capacity grow?
Yeah, it's nice! And Abseil does it, IFF you use LLVM libc++.
https://github.com/abseil/abseil-cpp/blob/master/absl/string...
The standard adopted it as resize_and_overwrite. Which I think is a little clunky.
-
Shaving 40% Off Google’s B-Tree Implementation with Go Generics
This may be confusing to those familiar with Google's libraries. The baseline is the Go BTree, which I personally never heard of until just now, not the C++ absl::btree_set. The benchmarks aren't directly comparable, but the C++ version also comes with good microbenchmark coverage.
https://github.com/google/btree
https://github.com/abseil/abseil-cpp/blob/master/absl/contai...
- Faster Sorting Beyond DeepMind’s AlphaDev
-
“Once” one-time concurrent initialization with an integer
An implementation of call_once that accommodates callbacks that throw: https://github.com/abseil/abseil-cpp/blob/master/absl/base/c...
-
[R] AlphaDev discovers faster sorting algorithms
I wouldn't say it's that cryptic. It's just a few bitwise rotations/shifts/xor operations.
-
Deepmind Alphadev: Faster sorting algorithms discovered using deep RL
You can see hashing optimizations as well https://www.deepmind.com/blog/alphadev-discovers-faster-sort..., https://github.com/abseil/abseil-cpp/commit/74eee2aff683cc7d...
I was one of the members who reviewed expertly what has been done both in sorting and hashing. Overall it's more about assembly, finding missed compiler optimizations and balancing between correctness and distribution (in hashing in particular).
It was not revolutionary in a sense it hasn't found completely new approaches but converged to something incomprehensible for humans but relatively good for performance which proves the point that optimal programs are very inhuman.
Note that for instructions in sorting, removing them does not always lead to better performance, for example, instructions can run in parallel and the effect can be less profound. Benchmarks can lie and compiler could do something differently when recompiling the sort3 function which was changed. There was some evidence that the effect can come from the other side.
For hashing it was even funnier, very small strings up to 64 bit already used 3 instructions like add some constant -> multiply 64x64 -> xor upper/lower. For bigger ones the question becomes more complicated, that's why 9-16 was a better spot and it simplified from 2 multiplications to just one and a rotation. Distribution on real workloads was good, it almost passed smhasher and we decided it was good enough to try out in prod. We did not rollback as you can see from abseil :)
But even given all that, it was fascinating to watch how this system was searching and was able to find particular programs can be further simplified. Kudos to everyone involved, it's a great incremental change that can bring more results in the future.
-
Backward compatible implementations of newer standards constructs?
Check out https://abseil.io. It offers absl::optional, which is a backport of std::optional.
What are some alternatives?
Folly - An open-source C++ library developed and used at Facebook.
glommio - Glommio is a thread-per-core crate that makes writing highly parallel asynchronous applications in a thread-per-core architecture easier for rustaceans.
Boost - Super-project for modularized Boost
Boost.Asio - Asio C++ Library
spdlog - Fast C++ logging library.
Qt - Qt Base (Core, Gui, Widgets, Network, ...)
ffead-cpp - Framework for Enterprise Application Development in c++, HTTP1/HTTP2/HTTP3 compliant, Supports multiple server backends
EASTL - Obsolete repo, please go to: https://github.com/electronicarts/EASTL
BDE - Basic Development Environment - a set of foundational C++ libraries used at Bloomberg.