abseil-cpp
Disruptor
abseil-cpp | Disruptor | |
---|---|---|
54 | 30 | |
13,989 | 17,037 | |
1.6% | 0.4% | |
9.5 | 5.4 | |
5 days ago | 4 months ago | |
C++ | Java | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
abseil-cpp
- Sane C++ Libraries
- Open source collection of Google's C++ libraries
- Is Ada safer than Rust?
-
Appending to an std:string character-by-character: how does the capacity grow?
Yeah, it's nice! And Abseil does it, IFF you use LLVM libc++.
https://github.com/abseil/abseil-cpp/blob/master/absl/string...
The standard adopted it as resize_and_overwrite. Which I think is a little clunky.
-
Shaving 40% Off Google’s B-Tree Implementation with Go Generics
This may be confusing to those familiar with Google's libraries. The baseline is the Go BTree, which I personally never heard of until just now, not the C++ absl::btree_set. The benchmarks aren't directly comparable, but the C++ version also comes with good microbenchmark coverage.
https://github.com/google/btree
https://github.com/abseil/abseil-cpp/blob/master/absl/contai...
- Faster Sorting Beyond DeepMind’s AlphaDev
-
“Once” one-time concurrent initialization with an integer
An implementation of call_once that accommodates callbacks that throw: https://github.com/abseil/abseil-cpp/blob/master/absl/base/c...
-
[R] AlphaDev discovers faster sorting algorithms
I wouldn't say it's that cryptic. It's just a few bitwise rotations/shifts/xor operations.
-
Deepmind Alphadev: Faster sorting algorithms discovered using deep RL
You can see hashing optimizations as well https://www.deepmind.com/blog/alphadev-discovers-faster-sort..., https://github.com/abseil/abseil-cpp/commit/74eee2aff683cc7d...
I was one of the members who reviewed expertly what has been done both in sorting and hashing. Overall it's more about assembly, finding missed compiler optimizations and balancing between correctness and distribution (in hashing in particular).
It was not revolutionary in a sense it hasn't found completely new approaches but converged to something incomprehensible for humans but relatively good for performance which proves the point that optimal programs are very inhuman.
Note that for instructions in sorting, removing them does not always lead to better performance, for example, instructions can run in parallel and the effect can be less profound. Benchmarks can lie and compiler could do something differently when recompiling the sort3 function which was changed. There was some evidence that the effect can come from the other side.
For hashing it was even funnier, very small strings up to 64 bit already used 3 instructions like add some constant -> multiply 64x64 -> xor upper/lower. For bigger ones the question becomes more complicated, that's why 9-16 was a better spot and it simplified from 2 multiplications to just one and a rotation. Distribution on real workloads was good, it almost passed smhasher and we decided it was good enough to try out in prod. We did not rollback as you can see from abseil :)
But even given all that, it was fascinating to watch how this system was searching and was able to find particular programs can be further simplified. Kudos to everyone involved, it's a great incremental change that can bring more results in the future.
-
Backward compatible implementations of newer standards constructs?
Check out https://abseil.io. It offers absl::optional, which is a backport of std::optional.
Disruptor
-
Gnet is the fastest networking framework in Go
https://lmax-exchange.github.io/disruptor/#_what_is_the_disr.... Unfortunately IIUC writing this in Go still prevents the spin-locked acceptor thread from achieving the kind of performance you could get in a non-GC language, unless you chose to disable GC, so I'd guess Envoy is still faster.
https://gnet.host/docs/quickstart/ it's nice that you can use this simply though. Envoy is kind of tricky to setup with custom filters, so most of the time it's just a standalone binary.
[0] https://blog.envoyproxy.io/envoy-threading-model-a8d44b92231...
[1] https://lmax-exchange.github.io/disruptor/#_what_is_the_disr...
-
A lock-free ring-buffer with contiguous reservations (2019)
See also the Java LMAX Disruptor https://github.com/LMAX-Exchange/disruptor
I've built a similar lock-free ring buffer in C++11 https://github.com/posterior/loom/blob/master/doc/adapting.m...
-
JEP Draft: Deprecate Memory-Access Methods in Sun.misc.Unsafe for Removal
"Why we chose Java for our High-Frequency Trading application"
https://medium.com/@jadsarmo/why-we-chose-java-for-our-high-...
LMAX Disruptor customers
https://lmax-exchange.github.io/disruptor/
Among many other examples.
-
LMAX Disruptor – High Performance Inter-Thread Messaging Library
Current documentation
https://lmax-exchange.github.io/disruptor/
-
Progress on No-GIL CPython
LMAX Disruptor has on their wiki that average latency to send a message from one thread to another at 53 nanoseconds. For comparison a mutex is like 25 nanoseconds and more if Contended but a mutex is point to point synchronization.
The great thing about it is that multiple threads can receive the same message without much more effort.
https://github.com/LMAX-Exchange/disruptor/wiki/Performance-...
https://gist.github.com/rmacy/2879257
I am dreaming of language that is similar to Smalltalk that stays single threaded until it makes sense to parallise.
I am looking for problems to parallelism that are not big data. Parallelism is like adding more cars to the road rather than increasing the speed of the car. But what does a desktop or mobile user need to do locally that could take advantage of the mathematical power of a computer? I'm still searching.
- Disruptor 4.0.0 Released
-
Anything can be a message queue if you use it wrongly enough
Database config should be two connection strings, 1 for the admin user that creates the tables and anther for the queue user. Everything else should be stored in the database itself. Each queue should be in its own set of tables. Large blobs may or may not be referenced to an external file.
Shouldn't a message send be worst case a CAS. It really seems like all the work around garbage collection would have some use for in-memory high speed queues.
Are you familiar with the LMAX Disruptor? Is is a Java based cross thread messaging library used for day trading applications.
https://lmax-exchange.github.io/disruptor/
-
Any library you would like to recommend to others as it helps you a lot? For me, mapstruct is one of them. Hopefully I would hear some other nice libraries I never try.
Disruptor for inter-thread messaging
-
Measuring how much Rust's bounds checking actually costs
I have never worked in any industries where a perf margin was that small. It is funny, in HFT there are folks using Lmax (Java) and then you have folks writing their own TCP/IP stacks on FPGAs to do trading.
What are some alternatives?
Folly - An open-source C++ library developed and used at Facebook.
JCTools
Boost - Super-project for modularized Boost
Agrona - High Performance data structures and utility methods for Java
spdlog - Fast C++ logging library.
fastutil - fastutil extends the Java™ Collections Framework by providing type-specific maps, sets, lists and queues.
Qt - Qt Base (Core, Gui, Widgets, Network, ...)
MPMCQueue.h - A bounded multi-producer multi-consumer concurrent queue written in C++11
EASTL - Obsolete repo, please go to: https://github.com/electronicarts/EASTL
Eclipse Collections - Eclipse Collections is a collections framework for Java with optimized data structures and a rich, functional and fluent API.
BDE - Basic Development Environment - a set of foundational C++ libraries used at Bloomberg.
Javolution