oss-fuzz VS YCSB

> because the ifunc code was breaking with all sorts of build options and obviously caused many problems with various sanitizers

for example, https://github.com/google/oss-fuzz/pull/10667

Have you tried Googles grpc fuzzer?

https://github.com/google/oss-fuzz/blob/master/projects/grpc...

https://www.fda.gov/medical-devices/digital-health-center-ex...

oss-fuzz: https://github.com/google/oss-fuzz :

> We support the libFuzzer, AFL++, and Honggfuzz fuzzing engines in combination with Sanitizers, as well as ClusterFuzz, a distributed fuzzer execution environment and reporting tool.

> Currently, OSS-Fuzz supports C/C++, Rust, Go, Python, Java/JVM, and JavaScript code. Other languages supported by LLVM may work too. OSS-Fuzz supports fuzzing x86_64 and i386 builds.

I love fuzzing as a technique and use it quite regularly, but running AFL++ on even a single program occupies all threads of a high end AMD server for weeks. I'm running it locally so only paying for the electricity. If it was a cloud instance it would cost a small fortune. I think this is a reason it is not used more widely.

I will note that Google have a programme for doing fuzz testing on open source projects using computer from their cloud: https://google.github.io/oss-fuzz/

Yes, mosh has fuzz tests in oss-fuzz [1].

[1] https://github.com/google/oss-fuzz/tree/master/projects/mosh

We will explore how Jazzer is used to automatically generate malicious inputs for Java programs, and how it compares to Symflower, which can automatically generate unit tests to uncover bugs and errors in your code. With the help of Jazzer, many bugs - some of them even in the OpenJDK - were found already. Also, as of March 2021, Jazzer is officially part of OSS-Fuzz, Google's cloud fuzzing engine. It should be noted that Jazzer is a pure "bug detection" utility that finds reproducers for errors in user code. Symflower can do the same, but provides additional functionalities to boost developer productivity, like generating high coverage unit tests and providing test templates for the software developer or tester.

For performance testing, we also run common benchmarks: the popular YCSB (Yahoo! Cloud Serving Benchmark), NoSQLBench, LinkBench, SysBench, TPC-H, and TPC-C. We also run C Bench, our own Tarantool API benchmark. Its primitive operations are written in C, and scripts are described in Lua.

Most benchmarks of non-relational databases are done with either product-specific tooling (like cassandra-stress), or with YCSB, which gives you a lowest-common-denominator key-value workload across dozens of systems.