structopt VS hyperscan

Hope you are aware that structopt is in maintenance mode and is merged into clap as of v3.

As I understood you need to implement a command line argument parser for that you can use clap https://github.com/clap-rs/clap or structopt https://github.com/TeXitoi/structopt.

I feel like discovering moves like this is a weakness in the ecosystem today. You can check out some of our discussion on raising visibility

Something I've been giving thought to is how to help structopt users discover that clap3 is their upgrade path. We've put notices in the structopt repo but cargo upgrade and docs.rs won't say anything. See https://github.com/TeXitoi/structopt/issues/525 for more ideas we're considering.

For myself, I have found serde.rs really useful for undertanding their derives while I've always been frustrated with finding anything in structopt's documentation, so I modeled it more off of serde. This ended up both being in structure and not being in docs.rs. I think it really was the structure that was the frustration point for me but there was interest elsewhere in moving stuff out of docs.rs.

I think it would be nice to have a comparison to clap-derive and/or structopt in the README, as that is what I expect most users would compare this to. The subcommand handling looks especially cumbersome compared to deriving on structs and enums.

However, the one place I'm a little curious to rewrite things is the CLI... every time we have to deal with cobra I long for Rust's structopt.

Have you tried https://github.com/TeXitoi/structopt ?

It is written in Rust using the structopt and swayipc crates. It is published on crates.io. The repository is hosted on GitLab. The repository is mirrored on GitHub.

I use structopt, which itself uses clap.

Maybe some other PCRE-compatible implementation offers streaming. For instance, https://www.intel.com/content/www/us/en/developer/articles/t... says it has this feature, but of course given who it's from it may be tied to a single brand of CPU.

github seems to be https://github.com/intel/hyperscan

I think of WAFs as an extra safety net. Defense in depth.

The author complained about the performance cost of WAFs in general, but not all WAFs have be structured like ModSecurity. They could for example be based on something like https://github.com/intel/hyperscan and perf is at a very different level.

Another example of old timey Unix code just breaking things in minor point releases. See https://abi-laboratory.pro/index.php?view=changelog&l=glibc&... and https://github.com/intel/hyperscan/issues/359.

It’s a bespoke scanning setup designed to deal with GitHub’s scale. Under the hood it’s using Intel’s hyperscan as the regex engine.

https://github.com/intel/hyperscan

Hyperscan is an Intel regular expression library.

We are using hyperscan [3] instead of grepping byte sequences with Python, which is orders of magnitudes faster. It can also handle 4Gb+ files because of this which binwalk cannot.

It's used for a year now in production and it's way more precise and faster than binwalk. We are getting less false-positives too, and even if unblob fails to extract everything, we still get meaningful information out of firmwares, where binwalk just failed with no output previously.

[1]: https://github.com/onekey-sec/unblob/blob/main/unblob/handle...

[2]: https://github.com/onekey-sec/unblob/blob/main/unblob/proces...

[3]: https://github.com/intel/hyperscan