base32h-tests VS American Fuzzy Lop

I'm fond of taking this a step further and actually offloading the test cases to a separate language-agnostic file, then have implementations' test harnesses read cases from that file. The Ruby gem for Base32H¹ does exactly this to test the encoder/decoder logic, pulling the test cases from CSV files in a separate repo² included as a Git submodule. One of these days I need to port the other reference implementations over to using the tests repo, but once that's done I'll be able to update the test cases in one place and they'll automatically work everywhere.

----

¹ https://github.com/Base32H/base32h.rb/tree/master/spec

² https://github.com/Base32H/base32h-tests

There's some efforts to guide test generation for property based testing to make the instruction pointer explore as large a space as possible.

This effort is more mature in the fuzzing community. See eg American Fuzzy Lop https://github.com/google/AFL

What you're describing, just generating random input to test a program, is sometimes called "blind fuzzing" but the state-of-the-art is far beyond that. Maybe try reading through the documentation of e.g. https://github.com/google/AFL to see what a fuzzer does and why just producing random input isn't even scratching the surface.

for general riscv I used to use this https://github.com/google/AFL I dont know if it supports x64 tho.

Ex ( AFL, WinAFL, HonggFuzz, LibFuzzer, Jazzer )

I wrote this little fuzz test target in order to fuzz it with afl (under ASan and UBSan):

afl, which is trivial to apply to this program:

I made my own version of a TCL interpreter (well, a very TCL like langauge) derived from "picol" available at https://github.com/howerj/pickle. There are many different re-implementations and derivatives of this interpreter but they all seem very "crashy", this one has been significantly hardened by using a fuzzer on it which ran for months called American Fuzzy Lop https://lcamtuf.coredump.cx/afl/ . It is also more suitable for embedded use whilst still not having arbitrary restrictions like many other implementations.

On Linux afl is a very powerful bug-finding tool, and it's a great companion when doing code review. Composes well with ASan and UBSan.

b-, https://lcamtuf.coredump.cx/afl/