confluent-kafka-go
leakdice-rust
confluent-kafka-go | leakdice-rust | |
---|---|---|
1 | 4 | |
1 | 9 | |
- | - | |
0.0 | 0.0 | |
about 1 year ago | over 2 years ago | |
Go | Rust | |
Apache License 2.0 | GNU General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
confluent-kafka-go
-
Hunting down a C memory leak in a Go program
So, in the interests of full transparency - we at Zendesk are actually running a fork of confluent-kafka-go, which I forked to add, amongst other things, context support: https://github.com/confluentinc/confluent-kafka-go/pull/626
This bug actually happened because I mis-merged upstream into our fork and missed an important call to rd_kafka_poll_set_consumer: https://github.com/zendesk/confluent-kafka-go/commit/6e2d889...
leakdice-rust
-
My Rust program (Well, game) is leaking memory, 4MB/s.
Maybe try Leakdice: https://github.com/tialaramex/leakdice in C or rewritten in Rust: https://github.com/tialaramex/leakdice-rust/
-
Twenty Years of Valgrind
In my obviously biased opinion, very specialised, but sometimes exactly what you needed (I have used this in anger maybe 2-3 times in my career since then, which is why I wrote the C version):
https://github.com/tialaramex/leakdice (or https://github.com/tialaramex/leakdice-rust)
Leakdice implements some of Raymond Chen's "The poor man’s way of identifying memory leaks" for you. On Linux at least.
https://bytepointer.com/resources/old_new_thing/20050815_224...
All leakdice does is: You pick a running process which you own, leakdice picks a random heap page belonging to that process and shows you that page as hex + ASCII.
The Raymond Chen article explains why you might ever want to do this.
-
Hunting down a C memory leak in a Go program
(or there's a Rust rewrite https://github.com/tialaramex/leakdice-rust because I was learning Rust)
leakdice is not a clever, sophisticated tool like valgrind, or eBPF programming, but that's fine because this isn't a subtle problem - it's very blatant - and running leakdice takes seconds so if it wasn't helpful you've lost very little time.
Here's what leakdice does: It picks a random heap page of a running process, which you suspect is leaking, and it displays that page as ASCII + hex.
That's all, and that might seem completely useless, unless you either read Raymond Chen's "The Old New Thing" or you paid attention in statistics class.
Because your program is leaking so badly the vast majority of heap pages (leakdice counts any pages which are writable and anonymous) are leaked. Any random heap page, therefore, is probably leaked. Now, if that page is full of zero bytes you don't learn very much, it's just leaking blank pages, hard to diagnose. But most often you're leaking (as was happening here) something with structure, and very often sort of engineer assigned investigating a leak can look at a 4kbyte page of structure and go "Oh, I know what that is" from staring at the output in hex + ASCII.
This isn't a silver bullet, but it's very easy and you can try it in like an hour (not days, or a week) including writing up something like "Alas the leaked pages are empty" which isn't a solution but certainly clarifies future results.
- `Zig Cc`: A Powerful Drop-In Replacement for GCC/Clang
What are some alternatives?
librdkafka - The Apache Kafka C/C++ library
libclang_rt.builtins-wasm32.a - The missing libclang_rt.builtins-wasm32.a file to compile to WebAssembly.
kafka-go - Kafka library in Go
bytehound - A memory profiler for Linux.
leakdice - Monte Carlo leak diagnostic for Linux binaries
mevi - A memory visualizer in Rust (ptrace + userfaultfd)
Confluent Kafka Golang Client - Confluent's Apache Kafka Golang client
sanitizers - AddressSanitizer, ThreadSanitizer, MemorySanitizer
jemalloc
codechecker - CodeChecker is an analyzer tooling, defect database and viewer extension for the Clang Static Analyzer and Clang Tidy