HELK VS loghub

Utilizing that api and juniper notebooks is exactly why Hunting Elk is the way it from my understanding.

So you can actual do both defensive while practicing offensive. If you can set up a lab system with an attacker, for ease using kali, and defensive systems like a single windows box, or you can go balls to the wall if you have the resources and set up an AD environment and then ship all the logs to a SIEM system like Splunk or HELK (https://github.com/Cyb3rWard0g/HELK). Building off the environment you can also include Mordor (https://github.com/UraSecTeam/mordor)

HELK + Mordor combo https://github.com/Cyb3rWard0g/HELK

On a side note - I somehow have the feeling that you are trying to recreate https://github.com/Cyb3rWard0g/HELK

You can find tools that leverage ELK that aren't necessarily plugins. SIEM looks like it has some free component to it, too: https://github.com/Cyb3rWard0g/HELK https://www.elastic.co/blog/elastic-siem-free-open

HELK can help for the SIEM and detection part

MISP is a Threat Intelligence Platform (TIP), not a hunting platform. That would be something like HELK

https://github.com/logpai/loglizer has an MIT license. Seems like they've done some of the heavy lifting already. If you're just looking for logs, check out https://github.com/logpai/loghub.

Here is an example apache log fie: If you want to try and prove me wrong write a script to find and return line numbers which contain the string "Directory index forbidden" or some other substring which is better then O(N).

Any text? How about Loghub? Maybe you could visualize links in error messages for failure analysis, for example does a seemingly harmless warning seem to be linked to a much worse problem that occurs later?

Here is a summary for a log from the logpai/loghub dataset (kudos to the Logpai team for sharing this dataset):