SBE
rr
SBE | rr | |
---|---|---|
7 | 102 | |
3,022 | 8,640 | |
1.2% | 1.3% | |
8.7 | 9.6 | |
6 days ago | 6 days ago | |
Java | C++ | |
Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
SBE
- Simple Binary Encoding (SBE) – High Performance Message Codec
- Simple Binary Encoding (SBE)
-
Possibly stupid question, is java the right language for low latency and high throughput web servers?
I was about to suggest Chronicle, but it looks like they have gone closed-source. The older version is still interesting to look through though. Aeron / Disruptor / SBE are good projects for inspiration as well.
-
GitHub - realtimetech-solution/opack: Fast object or data serialize and deserialize library
Could you evaluate how it compares with SBE?
-
Simple Binary Encoding (SBE) now supports Rust
The Simple Binary Encoding (SBE) project now includes support for generating Rust code. Generated code produced does not use unsafe and has no dependencies on any other crates.
-
I made an NBT-based data format, but a little more general purpose
SBE
-
Parsing Protobuf at 2+GB/S: How I Learned to Love Tail Calls in C
Consider a valid protobuf message with such a field. If you can locate the field value bytes, you can write a new value to the same location without breaking the message. It's obviously possible to the same with the varint type too, as long as you don't change the number of bytes - not so practical, but useful for enum field which has a limited set of useful values (usually less than 128).
Pregenerating protobuf messages you want to send and then modifying the bytes in-place before sending is going to give you a nice performance boost over "normal" protobuf serialization. It can be useful if you need to be protobuf compatible, but it's obviously better to use something like SBE - https://github.com/real-logic/simple-binary-encoding
rr
- rr: Lightweight Recording and Deterministic Debugging
-
Hermit is a hermetic and reproducible sandbox for running programs
I think this tool must share a lot techniques and use cases with rr. I wonder how it compares in various aspects.
https://rr-project.org/
rr "sells" as a "reversible debugger", but it obviously needs the determinism for its record and replay to work, and AFAIK it employs similar techniques regarding system call interception and serializing on a single CPU. The reversible debugger aspect is built on periodic snapshotting on top of it and replaying from those snapshots, AFAIK. They package it in a gdb compatible interface.
Hermit also lists record/replay as a motivation, although it doesn't list reversible debugging in general.
- Rr: Lightweight Recording and Deterministic Debugging
-
Deep Bug
Interesting. Perhaps you can inspect the disassembly of the function in question when using Graal and HotSpot. It is likely related to that.
Another debugging technique we use for heisenbugs is to see if `rr` [1] can reproduce it. If it can then that's great as it allows you to go back in time to debug what may have caused the bug. But `rr` is often not great for concurrency bugs since it emulates a single-core machine. Though debugging a VM is generally a nightmare. What we desperately need is a debugger that can debug both the VM and the language running on top of it. Usually it's one or the other.
> In general I’d argue you haven’t fixed a bug unless you understand why it happened and why your fix worked, which makes this frustrating, since every indication is that the bug exists within proprietary code that is out of my reach.
Were you using Oracle GraalVM? GraalVM community edition is open source, so maybe it's worth checking if it is reproducible in that.
[1]: https://github.com/rr-debugger/rr
-
So you think you want to write a deterministic hypervisor?
https://rr-project.org/ had the same problem. They use the retired conditional branch counter instead of instruction counter, and then instruction steeping until at the correct address.
-
Is Something Bugging You?
That'll work great for your Distributed QSort Incorporated startup, where the only product is a sorting algorithm.
Formal software verification is very useful. But what can be usefully formalized is rather limited, and what can be formalized correctly in practice is even more limited. That means you need to restrict your scope to something sane and useful. As a result, in the real world running thousands of tests is practically useful. (Well, it depends on what those tests are; it's easy to write 1000s of tests that either test the same thing, or only test the things that will pass and not the things that would fail.) They are especially useful if running in a mode where the unexpected happens often, as it sounds like this system can do. (It's reminiscent of rr's chaos mode -- https://rr-project.org/ linking to https://robert.ocallahan.org/2016/02/introducing-rr-chaos-mo... )
-
When "letting it crash" is not enough
The approach of check-pointing computation such that it is resumable and restartable sounds similar to a time-traveling debugger, like rr or WinDbg:
https://rr-project.org/
https://learn.microsoft.com/windows-hardware/drivers/debugge...
- When I got started I debugged using printf() today I debug with print()
- Rr: Record and Replay Debugger – Reverse Debugger
-
OpenBSD KDE Plasma Desktop
https://github.com/rr-debugger/rr?tab=readme-ov-file#system-...
What are some alternatives?
Protobuf - Protocol Buffers - Google's data interchange format
CodeLLDB - A native debugger extension for VSCode based on LLDB
FlatBuffers - FlatBuffers: Memory Efficient Serialization Library
rrweb - record and replay the web
Apache Avro - Apache Avro is a data serialization system.
gef - GEF (GDB Enhanced Features) - a modern experience for GDB with advanced debugging capabilities for exploit devs & reverse engineers on Linux
MessagePack - MessagePack implementation for C and C++ / msgpack.org[C/C++]
Module Linker - browse modules by clicking directly on "import" statements on GitHub
Boost.Serialization - Boost.org serialization module
nbdev - Create delightful software with Jupyter Notebooks
Cap'n Proto - Cap'n Proto serialization/RPC system - core tools and C++ library
clog-cli - Generate beautiful changelogs from your Git commit history