Suricata VS hyperscan

Linux has (free) tools to improve security and detect/remove malware: Lynis,Chkrootkit,Rkhunter,ClamAV,Vuls,LMD,radare2,Yara,ntopng,maltrail,Snort,Suricata...

Monitoring & Active Measures - Exporting firewall events to an external time-series database like I describe above is good to see who is touching your firewall or accessing your web site. Using an Intrusion Detection System / Intrusion Prevention System (IDS/IPS) such as open-source Suricata, which is a free package on pfSense, and deploying file system integrity monitoring, such as the open-source Wazuh on the exposed server are also good approaches to protecting yourself.

You could try running Suricata

Check out https://suricata.io/

Suricata: https://suricata.io/ IDS/IPS

Active Measures - Includes (IDS/IPS) such as open-source Suricata or Snort on pfSense, and File Integrity Monitoring (FIM), such as the commercial Tripwire and dated, open-source Tripwire, or the open-source Wazuh installed on servers. These can be combined into a Security Information and Event Management (SIEM) system like the open-source solution, Security Onion. Wazuh itself has evolved into a SIEM.

Active measures may include an intrusion detection system / intrusion prevention systems (IDS/IPS) such as open-source Suricata on the firewall, and installing file system integrity monitoring, such as the open-source Wazuh on the exposed server. These are combined in one open-source solution, Security Onion

Maybe some other PCRE-compatible implementation offers streaming. For instance, https://www.intel.com/content/www/us/en/developer/articles/t... says it has this feature, but of course given who it's from it may be tied to a single brand of CPU.

github seems to be https://github.com/intel/hyperscan

I think of WAFs as an extra safety net. Defense in depth.

The author complained about the performance cost of WAFs in general, but not all WAFs have be structured like ModSecurity. They could for example be based on something like https://github.com/intel/hyperscan and perf is at a very different level.

Another example of old timey Unix code just breaking things in minor point releases. See https://abi-laboratory.pro/index.php?view=changelog&l=glibc&... and https://github.com/intel/hyperscan/issues/359.

It’s a bespoke scanning setup designed to deal with GitHub’s scale. Under the hood it’s using Intel’s hyperscan as the regex engine.

https://github.com/intel/hyperscan

Hyperscan is an Intel regular expression library.

We are using hyperscan [3] instead of grepping byte sequences with Python, which is orders of magnitudes faster. It can also handle 4Gb+ files because of this which binwalk cannot.

It's used for a year now in production and it's way more precise and faster than binwalk. We are getting less false-positives too, and even if unblob fails to extract everything, we still get meaningful information out of firmwares, where binwalk just failed with no output previously.

[1]: https://github.com/onekey-sec/unblob/blob/main/unblob/handle...

[2]: https://github.com/onekey-sec/unblob/blob/main/unblob/proces...

[3]: https://github.com/intel/hyperscan