loghub VS coroot-node-agent

https://github.com/logpai/loglizer has an MIT license. Seems like they've done some of the heavy lifting already. If you're just looking for logs, check out https://github.com/logpai/loghub.

Here is an example apache log fie: If you want to try and prove me wrong write a script to find and return line numbers which contain the string "Directory index forbidden" or some other substring which is better then O(N).

Any text? How about Loghub? Maybe you could visualize links in error messages for failure analysis, for example does a seemingly harmless warning seem to be linked to a much worse problem that occurs later?

Here is a summary for a log from the logpai/loghub dataset (kudos to the Logpai team for sharing this dataset):

Our product can do that. It's based on an eBPF Prometheus exporter and doesn't require manual integration.

Node-agent gathers per-TGID statistics from the kernel through the Netlink protocol and aggregates it to the per-container metrics:

While working on node-agent, we set out to measure network latency between containers and the services they communicate with. Since the agent has already discovered the endpoints that each container communicates with, we just need to measure network latency. We embedded "pinger" directly into the agent to measure end-to-end latency because the ICMP Echo requests should be sent from within the network namespace of each container.

To address said disadvantages, we implemented eBPF-based container tracing which is a part of our open source Prometheus exporter node-agent. It passively monitors all TCP connections on a node, associates every connection with the related container, and exports metrics in Prometheus format:

At Coroot, we implemented an automated log parsing in our open-source Prometheus exporter node-agent. To explain how it works, let's follow the same steps as I mentioned above.