syzkaller VS Fuzzing101

https://arxiv.org/abs/2402.09171 :

> This paper describes Meta's TestGen-LLM tool, which uses LLMs to automatically improve existing human-written tests. TestGen-LLM verifies that its generated test classes successfully clear a set of filters that assure measurable improvement over the original test suite, thereby eliminating problems due to LLM hallucination. [...] We believe this is the first report on industrial scale deployment of LLM-generated code backed by such assurances of code improvement.

Coverage-guided unit test improvement might [with LLMs] be efficient too.

https://github.com/topics/coverage-guided-fuzzing :

- e.g. Google/syzkaller is a coverage-guided syscall fuzzer: https://github.com/google/syzkaller

- Gitlab CI supports coverage-guided fuzzing: https://docs.gitlab.com/ee/user/application_security/coverag...

- oss-fuzz, osv

Additional ways to improve tests:

Hypothesis and pynguin generate tests from type annotations.

There are various tools to generate type annotations for Python code;

> pytype (Google) [1], PyAnnotate (Dropbox) [2], and MonkeyType (Instagram) [3] all do dynamic / runtime PEP-484 type annotation type inference [4] to generate type annotations. https://news.ycombinator.com/item?id=39139198

icontract-hypothesis generates tests from icontract DbC Design by Contract type, value, and invariance constraints specified as precondition and postcondition @decorators:

https://google.github.io/clusterfuzz/setting-up-fuzzing/libf...

> OSS-Fuzz runs CloudFuzz[Lite?] for many open source repos and feeds OSV OpenSSF Vulnerability Format: https://github.com/google/osv#current-data-sources

.

Google/syzkaller https://github.com/google/syzkaller :

>> syzkaller is an unsupervised coverage-guided kernel fuzzer. Supported OSes: Akaros, FreeBSD, Fuchsia, gVisor, Linux, NetBSD, OpenBSD, Windows

.

ghidra-patchdiff-correlator:

The key to modern fuzzing is feedback, usually some kind of coverage testing of the program under test. This allows the fuzzer to be much smarter about how it finds new code paths, and makes fuzzing find bugs a lot quicker.

Google have a project to do fuzzing on Linux system calls using coverage feedback: https://github.com/google/syzkaller

Utilities that try to exercise ("fuzz") an interface with the intent of discovering bugs are called "fuzzers". The tool that comes to mind is syzkaller.

It's true that USB is probably a less desirable attack surface than modems, because it actually requires the user to physically connect their device to a malicious device, but I wouldn't discount it as impractical and unlikely to happen in the wild. There's a reason some of the more famous malware and spyware used to spread/attack over USB. Google actually does USB driver fuzzing and the amount of potentially devastating vulnerabilities is staggering.

This work comes from GitHub's Security Lab https://securitylab.github.com/

So I came back to those reports, and saw that new ones were disclosed. Octavio Gianatiempo had reported some issues in Vim, and I decided to contact him, in order to see if I could get some insights. Luckily, besides some really good materials which I am going to be putting in the end of the post, he had exactly what I was looking for, a repository with challenges and solutions involving fuzzing. This repository is called Fuzzing101.