heaptrack VS leakdice-rust

If we are talking replacing the libc allocator, then something like heaptrack is worth mentioning.

https://github.com/KDE/heaptrack

For accurate memory usage I prefer a memory profiler that overrides malloc and friends instead of the ones that probe the OS at regular intervals. You won't find memory spikes with the latter. Try heaptrack on Linux. I haven't found a good one for Windows yet.

I know it is not a profiler, but it is so criminally underrated that I decided to share it: https://github.com/KDE/heaptrack

If none of the above helps - I recommend heaptrack as a tool for tracking down your memory usage.

> memory leaks. It's surprisingly hard to find an easy to use memory leak detection tool.

I can vouch for heaptrack[1] nowadays, although it's pretty much Linux only. It's under the umbrella of KDE, but a heaptrack trace only requires a CLI app, and there is a nice Qt viewer to analyse the memory consumption.

It tracks the memory utilization at the level of malloc'd/free'd bytes. It's fine if your memory leak or other memory utilization problem is on this level. Recently I dealt with an issue, where increasing memory utilization was caused by fragmentation within the allocator. This didn't show up in heaptrack as an increasing memory utilization, but heaptrack still pointed out where most of the temporary allocations happened, leading to the culprit of the fragmentation.

[1] https://github.com/KDE/heaptrack

This is probably what you are looking for https://github.com/KDE/heaptrack

When I had this kind problems (heap related) I always use heaptrack. Take a look here for the details: https://github.com/KDE/heaptrack

Maybe try Leakdice: https://github.com/tialaramex/leakdice in C or rewritten in Rust: https://github.com/tialaramex/leakdice-rust/

In my obviously biased opinion, very specialised, but sometimes exactly what you needed (I have used this in anger maybe 2-3 times in my career since then, which is why I wrote the C version):

https://github.com/tialaramex/leakdice (or https://github.com/tialaramex/leakdice-rust)

Leakdice implements some of Raymond Chen's "The poor man’s way of identifying memory leaks" for you. On Linux at least.

https://bytepointer.com/resources/old_new_thing/20050815_224...

All leakdice does is: You pick a running process which you own, leakdice picks a random heap page belonging to that process and shows you that page as hex + ASCII.

The Raymond Chen article explains why you might ever want to do this.

(or there's a Rust rewrite https://github.com/tialaramex/leakdice-rust because I was learning Rust)

leakdice is not a clever, sophisticated tool like valgrind, or eBPF programming, but that's fine because this isn't a subtle problem - it's very blatant - and running leakdice takes seconds so if it wasn't helpful you've lost very little time.

Here's what leakdice does: It picks a random heap page of a running process, which you suspect is leaking, and it displays that page as ASCII + hex.

That's all, and that might seem completely useless, unless you either read Raymond Chen's "The Old New Thing" or you paid attention in statistics class.

Because your program is leaking so badly the vast majority of heap pages (leakdice counts any pages which are writable and anonymous) are leaked. Any random heap page, therefore, is probably leaked. Now, if that page is full of zero bytes you don't learn very much, it's just leaking blank pages, hard to diagnose. But most often you're leaking (as was happening here) something with structure, and very often sort of engineer assigned investigating a leak can look at a 4kbyte page of structure and go "Oh, I know what that is" from staring at the output in hex + ASCII.

This isn't a silver bullet, but it's very easy and you can try it in like an hour (not days, or a week) including writing up something like "Alas the leaked pages are empty" which isn't a solution but certainly clarifies future results.