fuzzcheck-rs VS structopt

Agreed. A while back I played around with fuzzcheck [1], which let's you write coverage-guided, structure-aware property tests, but the generation is smarter than just slamming a fuzzer's `&[u8]` input into `Arbitrary`. It also supports shrinking, which is nice. Don't know that I would recommend it though. It seemed difficult to write your own `Mutator`s. It also looks somewhat unmaintained nowadays, but I think the direction is worth exploring.

[1]: https://github.com/loiclec/fuzzcheck-rs/

Fuzzcheck is a structure-aware fuzzer for rust. "Fuzzing" means feeding large amounts of data into a program and checking for crashes (Fuzzcheck also checks to make sure that all the properties your program should uphold – e.g. a sorting algorithm applied to a list of n items should always return a list of n items – are upheld). Fuzzcheck is an "evolutionary" fuzzer – this means that it generates a set of random inputs, sees what percentage of the program is executed for each input, and keeps inputs which have high levels of percentage of program executed. It then "mutates" these inputs – whereas fuzzers such as AFL/Hongfuzz/etc mutate raw bytes in place (e.g. they swap bytes at different positions, or insert a random byte at a given position to generate inputs similar to the chosen "high coverage" inputs), Fuzzcheck works directly on the Rust types (so it might swap the order of two items in a vec, or randomly insert a new item). It's a really powerful tool for finding lots of bugs.

If you want help with Win support (issues/8) maybe post it here to get it added to TWIR.

I am working on a code coverage viewer for my fuzzer (fuzzcheck). I described what I've done so far in this issue and I am hoping to release the first version within two weeks.

The implications for my fuzzer, fuzzcheck, are huge! Compiling fuzz tests is a lot easier. There should be no more need to create a separate fuzz folder, fuzz tests can be regular #[test] functions, private implementation details can be fuzz-tested as well, rust-analyser works as expected, documentation can be easily generated, etc. I can also attach a human-readable coverage report to every test case :)

Since I graduated, I have had a lot more time to work on fuzzcheck. I am trying to flesh it out, test it, and document it for a new release. It has always felt a bit rushed/experimental and now I am hoping to make it into something solid. I have also played with an egui interface for it, to visualise the tested code coverage, understand how the fuzzer’s decisions are made, and also to interactively tweak the fuzzer’s behaviour. It's a lot of work but it's slowly all coming together! :)

fuzzcheck-rs is really cool. It combines property-testing with fuzzing, getting the nice, structured nature of the former, and the coverage-driven search of the latter, but it works by mutating the structure directly instead of going through a bit string. So if you have a binary tree, going from A(B, C) to A(C, B) can be a single mutation away if that makes sense in your use case, instead of being arbitrarily far away in the bitstring approach.

Hope you are aware that structopt is in maintenance mode and is merged into clap as of v3.

As I understood you need to implement a command line argument parser for that you can use clap https://github.com/clap-rs/clap or structopt https://github.com/TeXitoi/structopt.

I feel like discovering moves like this is a weakness in the ecosystem today. You can check out some of our discussion on raising visibility

Something I've been giving thought to is how to help structopt users discover that clap3 is their upgrade path. We've put notices in the structopt repo but cargo upgrade and docs.rs won't say anything. See https://github.com/TeXitoi/structopt/issues/525 for more ideas we're considering.

For myself, I have found serde.rs really useful for undertanding their derives while I've always been frustrated with finding anything in structopt's documentation, so I modeled it more off of serde. This ended up both being in structure and not being in docs.rs. I think it really was the structure that was the frustration point for me but there was interest elsewhere in moving stuff out of docs.rs.

I think it would be nice to have a comparison to clap-derive and/or structopt in the README, as that is what I expect most users would compare this to. The subcommand handling looks especially cumbersome compared to deriving on structs and enums.

However, the one place I'm a little curious to rewrite things is the CLI... every time we have to deal with cobra I long for Rust's structopt.

Have you tried https://github.com/TeXitoi/structopt ?

It is written in Rust using the structopt and swayipc crates. It is published on crates.io. The repository is hosted on GitLab. The repository is mirrored on GitHub.

I use structopt, which itself uses clap.