rr VS gdb-dashboard

I think this tool must share a lot techniques and use cases with rr. I wonder how it compares in various aspects.

https://rr-project.org/

rr "sells" as a "reversible debugger", but it obviously needs the determinism for its record and replay to work, and AFAIK it employs similar techniques regarding system call interception and serializing on a single CPU. The reversible debugger aspect is built on periodic snapshotting on top of it and replaying from those snapshots, AFAIK. They package it in a gdb compatible interface.

Hermit also lists record/replay as a motivation, although it doesn't list reversible debugging in general.

Interesting. Perhaps you can inspect the disassembly of the function in question when using Graal and HotSpot. It is likely related to that.

Another debugging technique we use for heisenbugs is to see if `rr` [1] can reproduce it. If it can then that's great as it allows you to go back in time to debug what may have caused the bug. But `rr` is often not great for concurrency bugs since it emulates a single-core machine. Though debugging a VM is generally a nightmare. What we desperately need is a debugger that can debug both the VM and the language running on top of it. Usually it's one or the other.

> In general I’d argue you haven’t fixed a bug unless you understand why it happened and why your fix worked, which makes this frustrating, since every indication is that the bug exists within proprietary code that is out of my reach.

Were you using Oracle GraalVM? GraalVM community edition is open source, so maybe it's worth checking if it is reproducible in that.

[1]: https://github.com/rr-debugger/rr

https://rr-project.org/ had the same problem. They use the retired conditional branch counter instead of instruction counter, and then instruction steeping until at the correct address.

That'll work great for your Distributed QSort Incorporated startup, where the only product is a sorting algorithm.

Formal software verification is very useful. But what can be usefully formalized is rather limited, and what can be formalized correctly in practice is even more limited. That means you need to restrict your scope to something sane and useful. As a result, in the real world running thousands of tests is practically useful. (Well, it depends on what those tests are; it's easy to write 1000s of tests that either test the same thing, or only test the things that will pass and not the things that would fail.) They are especially useful if running in a mode where the unexpected happens often, as it sounds like this system can do. (It's reminiscent of rr's chaos mode -- https://rr-project.org/ linking to https://robert.ocallahan.org/2016/02/introducing-rr-chaos-mo... )

The approach of check-pointing computation such that it is resumable and restartable sounds similar to a time-traveling debugger, like rr or WinDbg:

https://rr-project.org/

https://learn.microsoft.com/windows-hardware/drivers/debugge...

https://github.com/rr-debugger/rr?tab=readme-ov-file#system-...

Disclaimer: I'm making a competing build system.

I won't tell you specific build systems, but I will tell you what to look for.

Look for power. Unlimited power. [1]

Usually, this means a few things:

1. The build system uses a general-purpose language, even if the language needs features to be added.

2. The build system does not reduce the power of the general-purpose language. For example, say it starts with Python but prohibits recursion. In that case, you know it is not unlimited power. Looking at you, Starlark.

3. The build can be dynamically changed, i.e., the build is not statically determined before it even begins.

4. Each task has unlimited power. This means that the task can use a general-purpose language, not just run external processes.

5. And there has to be some thought put it in user experience.

Why are these important? Well, let's look at why with CMake, which fails all of them.

For #1, CMake's language started as a limited language for enumerating lists. (Hence, CMakeLists.txt is the file name.) And yet, it's grown to be as general-purpose as possible. Why? Because when you need an if statement, nothing else will do, and when you need a loop, nothing else will do.

And that brings us to #2: if CMake's language started limited, are there still places where it's limited? I argue yes, and I point to the article where it says that your couldn't dynamically call functions until recently. There are probably other places.

For #3, CMake's whole model precludes it. CMake generates the build upfront then expects another build system to actually execute it. There is no changing the build without regenerating it. (And even then, CMake did a poor job until the addition of `--fresh`.) A fully dynamic build should be able to add targets and make others targets depend on those new targets dynamically, among other things.

For #4, obviously CMake limits what tasks can do because Ninja and Make limit tasks to running commands.

As another example, to implement a LaTeX target, you technically need a while loop to iterate until a fixed point. To do that with Make and Ninja, you have to jump through hoops or use an external script that may not work on all platforms.

CMake obviously fails #5, and to see how much other build systems fail it, just look for comments pouring hate on those build systems. CMake fails the most, but I haven't seen one that passes yet.

As an example, CMake barely got a debugger. Wow! Cool! It's been 20 years! My build system will have a debugger in public release #2 (one after the MVP) that will be capable of outputting to multiple TTY's like gdb-dashboard. [2] They should have had this years ago!

Should other comments suggest specific build systems, like the one that suggested Bazel, judge them by this list. Some will be better than others. None will pass everything, IMO, which is why I'm making my own.

[1]: https://youtube.com/watch?v=Sg14jNbBb-8

[2]: https://github.com/cyrus-and/gdb-dashboard

To debug, termdebug is built in to vim and supports a front end to gdb. In combination with gdb-dashboard this gives a reasonably good debugging experience

https://github.com/cyrus-and/gdb-dashboard, https://github.com/cyrus-and/gdb-dashboard/wiki

You can use the Python API to GDB to get really fancy TUIs like this: https://github.com/cyrus-and/gdb-dashboard

Try GDB Dashboard, it makes gdb much easier to use:

https://github.com/cyrus-and/gdb-dashboard

There's also Voltron which works with both gdb and lldb (amongst others):

https://github.com/snare/voltron

I can't believe no one has mentioned `gdb-dashboard` [1] yet! I use it extensively. [2]

Beyond that, I have recently learned how to write custom pretty printers for GDB. This saves a lot of screen space. I should probably update [2] soon with those new techniques.

GDB is powerful, useful, and after getting my start in IDE debuggers, including Visual Studio, I struggle whenever I have to go back.

[1]: https://github.com/cyrus-and/gdb-dashboard

[2]: https://gavinhoward.com/2020/12/my-development-environment-a...

Debugging: Learn gdb. You can use it for assembly. You can use it for C on a MCU. You can use it for Go on a Linux system. It’s ubiquitous, versatile, and worth understanding. You can pretty-print the output with something like this to help you out at first. There are 2 things you need, however: on chip debugger and a debug probe (though you don’t need an expensive one). Effectively, you talk to GDB, GDB talks to the server exposed by OCD, OCD knows the debug probe protocol, and the debug probe can use the MCU debug peripheral via SWD or JTAG to get those details.

I use gdb because I'm usually in the terminal. I don't think terminal it's any faster though. This is a nice addition to boring old terminal gdb.