libpcap VS zipkin

As long as your custom service used the rpcap protocol I'd expect it to be possible. You may find this GitHub page informative, since it includes some rpcap source code and one of the main Wireshark developers appears to be a contributor there.

mitmproxy can be used to find the HTTP request with the needed data in addition browser dev tools. At some point, I'll explore tcpdump and wireshark to reverse engineer websites for web scraping and share the learnings with you.

(Of course, this consideration should be appropriately downweighted by YAGNI, as threading memory management through prototype or internal utility code can by itself easily force it into very non-prototype amounts of effort.)

[1] https://github.com/the-tcpdump-group/libpcap/blob/2180b6e56a...

I am working with libpcap, a library that parses packets captured from tcpdump, which I have cIncluded from Zig.

Nmap and Wireshark, tcpdump tools can be used to scan networks and packets.

Also definitely start looking at tcpdump and wireshark. These are invaluable tools for the non-network engineer that is perhaps a sysadmin or analyst to use for network troubleshooting.

For future reference, libpcap is a library that Wireshark and many other network analysis tools use to record network traffic. It has its own file format that it can use to save the recorded network traffic to disk.

A lot is covered here and here, sar might be useful, getfacl and tcpdump and probably hundred more tools I am not even aware of. The easiest to learn those is to have a problem and identify and solve it with those tools, but to do that you need to know roughly what those tools can do.

For example, gopacket uses libpcap by default for capturing the traffic. Libpcap doesn't support network namespaces and we can't ask it to listen to traffic on a different namespace. However, we can change the network namespace of the calling thread and then start libpcap to see the traffic on a different namespace.

When choosing distributed tracing tools, considerations include your technology stack, business requirements, and monitoring complexity. Zipkin, SkyWalking, and OpenTelemetry are popular distributed tracing solutions, each with its unique features.

From the perspective of the realization of GraphQL infrastructure, the interesting direction is "Finding". How to find the problem? How to find the bottleneck of the system? Distributed Tracing System (DTS) will help answer this question. Distributed tracing is a method of observing requests as they propagate through distributed environments. In our scenario, we have dozens of subgraphs, gateway, and transport layer through which the request goes. We have several tools that can be used to detect the whole lifecycle of the request through the system, e.g. Jaeger, Zipkin or solutions that provided DTS as a part of the solution NewRelic.

Zipkin is a distributed tracing system used for tracking and analyzing how requests move through complex systems, especially in setups with many interconnected services, known as microservices.

Distributed Tracing: Middleware for distributed tracing like Jaeger and Zipkin helps monitor and trace requests as they flow through multiple microservices, aiding in debugging, performance optimization, and understanding the system's behavior.

For microservice tracing, you might want to look at Zipkin [0], or OpenTelemetry [1]

[0] https://zipkin.io/

I’ve not used a self-hosted solution before, but here’s one I found. https://zipkin.io/

> IMO the reason these vendors can and do charge so much is not because telemetry software is hard.

I always saw it as "they are charging for their polished UI/experience"

The UI of https://zipkin.io/ versus DataDog is kind of... not really in the same ballpark?

There are the zipkin https://zipkin.io/ and jaeger https://www.jaegertracing.io/ packages/components you can use both have quickstarts if you consider that to be a beginner's guide.

Zipkin, which was developed by Twitter, is an open source tool for distributed tracing that can also be used to troubleshoot latency issues in your application. While Zipkin is Java-based, py_zipkin is an implementation for Python.