fuzzcheck-rs
hypothesis
Our great sponsors
fuzzcheck-rs | hypothesis | |
---|---|---|
8 | 20 | |
422 | 7,254 | |
- | 1.2% | |
5.5 | 9.9 | |
6 months ago | 11 days ago | |
Rust | Python | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
fuzzcheck-rs
-
Fuzzcheck (a structure-aware Rust fuzzer)
Fuzzcheck is a structure-aware fuzzer for rust. "Fuzzing" means feeding large amounts of data into a program and checking for crashes (Fuzzcheck also checks to make sure that all the properties your program should uphold – e.g. a sorting algorithm applied to a list of n items should always return a list of n items – are upheld). Fuzzcheck is an "evolutionary" fuzzer – this means that it generates a set of random inputs, sees what percentage of the program is executed for each input, and keeps inputs which have high levels of percentage of program executed. It then "mutates" these inputs – whereas fuzzers such as AFL/Hongfuzz/etc mutate raw bytes in place (e.g. they swap bytes at different positions, or insert a random byte at a given position to generate inputs similar to the chosen "high coverage" inputs), Fuzzcheck works directly on the Rust types (so it might swap the order of two items in a vec, or randomly insert a new item). It's a really powerful tool for finding lots of bugs.
-
fuzzcheck 0.9 release - run coverage-guided fuzz tests alongside your regular unit tests + code coverage visualiser + new online guide and improved documentation
If you want help with Win support (issues/8) maybe post it here to get it added to TWIR.
-
What's everyone working on this week (43/2021)?
I am working on a code coverage viewer for my fuzzer (fuzzcheck). I described what I've done so far in this issue and I am hoping to release the first version within two weeks.
-
What's everyone working on this week (31/2021)?
The implications for my fuzzer, fuzzcheck, are huge! Compiling fuzz tests is a lot easier. There should be no more need to create a separate fuzz folder, fuzz tests can be regular #[test] functions, private implementation details can be fuzz-tested as well, rust-analyser works as expected, documentation can be easily generated, etc. I can also attach a human-readable coverage report to every test case :)
-
What's everyone working on this week (30/2021)?
Since I graduated, I have had a lot more time to work on fuzzcheck. I am trying to flesh it out, test it, and document it for a new release. It has always felt a bit rushed/experimental and now I am hoping to make it into something solid. I have also played with an egui interface for it, to visualise the tested code coverage, understand how the fuzzer’s decisions are made, and also to interactively tweak the fuzzer’s behaviour. It's a lot of work but it's slowly all coming together! :)
-
What's your favourite under-rated Rust crate and why?
fuzzcheck-rs is really cool. It combines property-testing with fuzzing, getting the nice, structured nature of the former, and the coverage-driven search of the latter, but it works by mutating the structure directly instead of going through a bit string. So if you have a binary tree, going from A(B, C) to A(C, B) can be a single mutation away if that makes sense in your use case, instead of being arbitrarily far away in the bitstring approach.
- Fuzzcheck: Structure and coverage guided fuzzing for Rust
hypothesis
- Hypothesis
-
A Tale of Two Kitchens - Hypermodernizing Your Python Code Base
Hypothesis for Property-Based Testing: Hypothesis is a Python library facilitating property-based testing. It offers a distinct advantage by generating a wide array of input data based on specified properties or invariants within the code. The perks of Hypothesis include:
-
Pix2tex: Using a ViT to convert images of equations into LaTeX code
But then add tests! Tests for LaTeX equations that had never been executable as code.
https://github.com/HypothesisWorks/hypothesis :
> Hypothesis is a family of testing libraries which let you write tests parametrized by a source of examples. A Hypothesis implementation then generates simple and comprehensible examples that make your tests fail. This simplifies writing your tests and makes them more powerful at the same time, by letting software automate the boring bits and do them to a higher standard than a human would, freeing you to focus on the higher level test logic.
> This sort of testing is often called "property-based testing", and the most widely known implementation of the concept is the Haskell library QuickCheck, but Hypothesis differs significantly from QuickCheck and is designed to fit idiomatically and easily into existing styles of testing that you are used to, with absolutely no familiarity with Haskell or functional programming needed.
-
pgregory.net/rapid v1.0.0, modern Go property-based testing library
pgregory.net/rapid is a modern Go property-based testing library initially inspired by the power and convenience of Python's Hypothesis.
- Was muss man als nicht-technischer Quereinsteiger in Data Science *wirklich* können?
-
Python toolkits
Hypothesis to generate dummy data for test.
-
Best way to test GraphQL API using Python?
To create your own test cases, I recommend you use hypothesis-graphql in combination with hypothesis. hypothesis is a property-based testing library. Property-based testing is an approach to testing in which you make assertions about the result of a test given certain conditions and parameters. For example, if you have a mutation that requires a boolean parameter, you can assert that the client will receive an error if it sends a different type. hypothesis-graphql is a GraphQL testing library that knows how to use hypothesis strategies to generate query documents.
-
Fuzzcheck (a structure-aware Rust fuzzer)
The Hypothesis stateful testing code is somewhat self-contained, since it mostly builds on top of internal APIs that already existed.
-
Running C unit tests with pytest
We've had a lot of success combining that approach with property-based testing (https://github.com/HypothesisWorks/hypothesis) for the query engine at backtrace: https://engineering.backtrace.io/2020-03-11-how-hard-is-it-t... .
-
Machine Readable Specifications at Scale
Systems I've used for this include https://agda.readthedocs.io/en/v2.6.0.1/getting-started/what... https://coq.inria.fr https://www.idris-lang.org and https://isabelle.in.tum.de
An easier alternative is to try disproving the statement, by executing it on thousands of examples and seeing if any fail. That gives us less confidence than a full proof, but can still be better than traditional "there exists" tests. This is called property checking or property-based testing. Systems I've used for this include https://hypothesis.works https://hackage.haskell.org/package/QuickCheck https://scalacheck.org and https://jsverify.github.io
What are some alternatives?
openapi-fuzzer - Black-box fuzzer that fuzzes APIs based on OpenAPI specification. Find bugs for free!
pytest - The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
enum-map
Robot Framework - Generic automation framework for acceptance testing and RPA
phpass - PHPass, the WordPress password hasher, re-implemented in rust
Behave - BDD, Python style.
rs_pbrt - Rust crate to implement a counterpart to the PBRT book's (3rd edition) C++ code. See also https://www.rs-pbrt.org/about ...
nose2 - The successor to nose, based on unittest2
wg-allocators - Home of the Allocators working group: Paving a path for a standard set of allocator traits to be used in collections!
Schemathesis - Automate your API Testing: catch crashes, validate specs, and save time
structopt - Parse command line arguments by defining a struct.
nose - nose is nicer testing for python