SBE
Aeron
Our great sponsors
SBE | Aeron | |
---|---|---|
7 | 20 | |
3,022 | 7,054 | |
1.1% | 1.1% | |
8.7 | 9.8 | |
2 days ago | 3 days ago | |
Java | Java | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
SBE
- Simple Binary Encoding (SBE) – High Performance Message Codec
- Simple Binary Encoding (SBE)
-
Possibly stupid question, is java the right language for low latency and high throughput web servers?
I was about to suggest Chronicle, but it looks like they have gone closed-source. The older version is still interesting to look through though. Aeron / Disruptor / SBE are good projects for inspiration as well.
-
GitHub - realtimetech-solution/opack: Fast object or data serialize and deserialize library
Could you evaluate how it compares with SBE?
-
Simple Binary Encoding (SBE) now supports Rust
The Simple Binary Encoding (SBE) project now includes support for generating Rust code. Generated code produced does not use unsafe and has no dependencies on any other crates.
-
I made an NBT-based data format, but a little more general purpose
SBE
-
Parsing Protobuf at 2+GB/S: How I Learned to Love Tail Calls in C
Consider a valid protobuf message with such a field. If you can locate the field value bytes, you can write a new value to the same location without breaking the message. It's obviously possible to the same with the varint type too, as long as you don't change the number of bytes - not so practical, but useful for enum field which has a limited set of useful values (usually less than 128).
Pregenerating protobuf messages you want to send and then modifying the bytes in-place before sending is going to give you a nice performance boost over "normal" protobuf serialization. It can be useful if you need to be protobuf compatible, but it's obviously better to use something like SBE - https://github.com/real-logic/simple-binary-encoding
Aeron
-
LMAX Disruptor – High Performance Inter-Thread Messaging Library
Semi-related is the Aeron project: https://github.com/real-logic/aeron
-
Nálatok mi a helyzet?
- ez itt most egy izgalmasabb product (trading/matching engine, low latency code, aeron alapokon)
-
How do you do UDP Flow control?
Look into Aeron for examples of high performance UDP message sending. We use it for high performance audio messaging, and I previously used it in high frequency trading https://github.com/real-logic/aeron. It is written in Java/C, but the general concepts of back pressure and reliable delivery over UDP are well documented.
- Aeron: Efficient reliable UDP unicast, UDP multicast, and IPC message transport
-
Experience taking the training offer from real-logic Aeron framework creators?
They mention their training offer on the Aeron GitHub page here: https://github.com/real-logic/aeron
-
Low Latency C++ programs for High Frequency Trading (HFT)
Yup the Disruptor paper actually shocked the industry a bit, b/c it was so out of place. BTW, Martin Thompson went on improving the Disruptor, and the result is the Aeron Protocol: https://github.com/real-logic/aeron
- What network messaging library do you recommend?
-
Possibly stupid question, is java the right language for low latency and high throughput web servers?
I was about to suggest Chronicle, but it looks like they have gone closed-source. The older version is still interesting to look through though. Aeron / Disruptor / SBE are good projects for inspiration as well.
-
Looking similar framework with Aeron ( Java) to do benchmark test
We are using this Java Aeron (https://github.com/real-logic/aeron) to build our production distributed messaging cluster. As a Rust lover, Is there any similar lib or framework in our ecosystem to test benchmark with it?
-
if you had to restart at 0 knowledge what would you do?
Java: In the past years C++ in finance has been rapidly supplanted by Java thanks to breakthrough technologies in the past decade like LMAX Disruptor, Chronicle Queue, Azul JVM, and Aeron (not the ergonomic chair, but this one, the transport protocol that breaks kafka performance records out of the park - not really a full kafka replacement, as Kafka enforces subscriber GD and aeron is more of an OSI layer 4 better than TCP; google "Best-effort delivery vs reliable delivery"). There's plenty more but thanks to these technologies, they allowed a Java based stack to perform the latency and throughput requirements needed for high frequency trading/HFT. From top trading firms like Two Sigma to the New York Stock Exchange, they're in Java. For banks, large modern western banks worth their salt and have modernized their systems are dominated by Java, especially thanks to Azul. To list a few banks, ING, Wells Fargo, Credit Suisse, and Barclays are all in Azul. Even at work Java still dominates.
What are some alternatives?
Protobuf - Protocol Buffers - Google's data interchange format
Apache Kafka - Mirror of Apache Kafka
FlatBuffers - FlatBuffers: Memory Efficient Serialization Library
Embedded RabbitMQ - A JVM library to use RabbitMQ as an embedded service
Apache Avro - Apache Avro is a data serialization system.
Apache Pulsar - Apache Pulsar - distributed pub-sub messaging system
MessagePack - MessagePack implementation for C and C++ / msgpack.org[C/C++]
Apache ActiveMQ - Mirror of Apache ActiveMQ
Boost.Serialization - Boost.org serialization module
JeroMQ - Pure Java ZeroMQ
Cap'n Proto - Cap'n Proto serialization/RPC system - core tools and C++ library
Apache Camel - Apache Camel is an open source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data.