rr
FlatBuffers
Our great sponsors
rr | FlatBuffers | |
---|---|---|
102 | 48 | |
8,621 | 22,005 | |
1.1% | 0.9% | |
9.6 | 8.7 | |
5 days ago | 6 days ago | |
C++ | C++ | |
GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
rr
- rr: Lightweight Recording and Deterministic Debugging
-
Hermit is a hermetic and reproducible sandbox for running programs
I think this tool must share a lot techniques and use cases with rr. I wonder how it compares in various aspects.
https://rr-project.org/
rr "sells" as a "reversible debugger", but it obviously needs the determinism for its record and replay to work, and AFAIK it employs similar techniques regarding system call interception and serializing on a single CPU. The reversible debugger aspect is built on periodic snapshotting on top of it and replaying from those snapshots, AFAIK. They package it in a gdb compatible interface.
Hermit also lists record/replay as a motivation, although it doesn't list reversible debugging in general.
- Rr: Lightweight Recording and Deterministic Debugging
-
Deep Bug
Interesting. Perhaps you can inspect the disassembly of the function in question when using Graal and HotSpot. It is likely related to that.
Another debugging technique we use for heisenbugs is to see if `rr` [1] can reproduce it. If it can then that's great as it allows you to go back in time to debug what may have caused the bug. But `rr` is often not great for concurrency bugs since it emulates a single-core machine. Though debugging a VM is generally a nightmare. What we desperately need is a debugger that can debug both the VM and the language running on top of it. Usually it's one or the other.
> In general I’d argue you haven’t fixed a bug unless you understand why it happened and why your fix worked, which makes this frustrating, since every indication is that the bug exists within proprietary code that is out of my reach.
Were you using Oracle GraalVM? GraalVM community edition is open source, so maybe it's worth checking if it is reproducible in that.
[1]: https://github.com/rr-debugger/rr
-
So you think you want to write a deterministic hypervisor?
https://rr-project.org/ had the same problem. They use the retired conditional branch counter instead of instruction counter, and then instruction steeping until at the correct address.
-
Is Something Bugging You?
That'll work great for your Distributed QSort Incorporated startup, where the only product is a sorting algorithm.
Formal software verification is very useful. But what can be usefully formalized is rather limited, and what can be formalized correctly in practice is even more limited. That means you need to restrict your scope to something sane and useful. As a result, in the real world running thousands of tests is practically useful. (Well, it depends on what those tests are; it's easy to write 1000s of tests that either test the same thing, or only test the things that will pass and not the things that would fail.) They are especially useful if running in a mode where the unexpected happens often, as it sounds like this system can do. (It's reminiscent of rr's chaos mode -- https://rr-project.org/ linking to https://robert.ocallahan.org/2016/02/introducing-rr-chaos-mo... )
-
When "letting it crash" is not enough
The approach of check-pointing computation such that it is resumable and restartable sounds similar to a time-traveling debugger, like rr or WinDbg:
https://rr-project.org/
https://learn.microsoft.com/windows-hardware/drivers/debugge...
- When I got started I debugged using printf() today I debug with print()
- Rr: Record and Replay Debugger – Reverse Debugger
-
OpenBSD KDE Plasma Desktop
https://github.com/rr-debugger/rr?tab=readme-ov-file#system-...
FlatBuffers
- FlatBuffers – an efficient cross platform serialization library for many langs
-
Cap'n Proto 1.0
I don't work at Cloudflare but follow their work and occasionally work on performance sensitive projects.
If I had to guess, they looked at the landscape a bit like I do and regarded Cap'n Proto, flatbuffers, SBE, etc. as being in one category apart from other data formats like Avro, protobuf, and the like.
So once you're committed to record'ish shaped (rather than columnar like Parquet) data that has an upfront parse time of zero (nominally, there could be marshalling if you transmogrify the field values on read), the list gets pretty short.
https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-... goes into some of the trade-offs here.
Cap'n Proto was originally made for https://sandstorm.io/. That work (which Kenton has presumably done at Cloudflare since he's been employed there) eventually turned into Cloudflare workers.
Another consideration: https://github.com/google/flatbuffers/issues/2#issuecomment-...
-
Anyone has experience with reverse engineering flatbuffers?
Much more in the discussion of this particular issue onGitHub: flatbuffers:Reverse engineering #4258
-
Flatty - flat message buffers with direct mapping to Rust types without packing/unpacking
Related but not Rust-specific: FlatBuffers, Cap'n Proto.
- flatbuffers - FlatBuffers: Memory Efficient Serialization Library
-
How do AAA studios make update-compatible save systems?
If json files are a concern because of space, you can always look into something like protobuffers or flatbuffers. But whatever you use, you should try to find a solution where you don't have to think about the actual serialization/deserialization of your objects, and can just concentrate on the data.
- QuickBuffers 1.1 released
-
Choosing a protocol for communication between multiple microcontrollers
Or, as an alternative to protobuffers, there's also flatbuffers, which is lighter weight and needs less memory: https://google.github.io/flatbuffers/
- FlatBuffers: FlatBuffers
-
Is using Flatbuffers to parse sensor data a bad application of Flatbuffers?
As the title suggests, I am considering using Flatbuffers as a way to parse sensor data that has been stored in local datafiles. The project language is python.
What are some alternatives?
CodeLLDB - A native debugger extension for VSCode based on LLDB
Protobuf - Protocol Buffers - Google's data interchange format
rrweb - record and replay the web
MessagePack - MessagePack implementation for C and C++ / msgpack.org[C/C++]
gef - GEF (GDB Enhanced Features) - a modern experience for GDB with advanced debugging capabilities for exploit devs & reverse engineers on Linux
MessagePack - MessagePack serializer implementation for Java / msgpack.org[Java]
Module Linker - browse modules by clicking directly on "import" statements on GitHub
Cap'n Proto - Cap'n Proto serialization/RPC system - core tools and C++ library
nbdev - Create delightful software with Jupyter Notebooks
cereal - A C++11 library for serialization
clog-cli - Generate beautiful changelogs from your Git commit history
Kryo - Java binary serialization and cloning: fast, efficient, automatic