alive2 VS CrossHair

Another problem with LLVM I’ve heard about is that it’s intermediate language or API or something is a moving, informally-specified target. People who know LLVM internals might weigh in on that claim. If true, it’s actually easier to target C or a subset of Rust just because it’s static and well-understood.

Two projects sought to mitigate these issues by going in different directions. One was a compiler backend that aimed to be easy to learn with well-specified IL. The other aimed to formalize LLVM’s IL.

http://c9x.me/compile/

https://github.com/AliveToolkit/alive2

There have also been typed, assembly languages to support verification from groups like FLINT. One can also compile language-specific analysis with a certified to LLVM IL compiler. Integrating pieces from different languages can have risks. That (IIRC) is being mitigated by people doing secure, abstract compilation.

You can use it to (mostly) validate small snippets are the same. See Alive2 for the application of Z3/formalization of programs as SMT for that [1]. As far as I'm aware there are some problems scaling up to arbitrarily sized programs due to a lack of formalization in higher level languages in addition to computational constraints. With a lot of time and effort it can be done though [2].

1. https://github.com/AliveToolkit/alive2

2. https://sel4.systems/

Yeah, this kind of thing is nice.

Alive had been used for years (almost a decade actually) by people to verify LLVM instcombine transforms.

Alive2 (https://github.com/AliveToolkit/alive2) makes it easier to do the same with most optimization passes.

Alive/Alive2 [1] is one of the most famous frameworks for compiler transformation verification using BitVec logic

[1] https://github.com/AliveToolkit/alive2

Writing some Python for Hacktoberfest? Try out CrossHair while you do that and get credit for a blog post too! https://github.com/pschanely/CrossHair/issues/173

CrossHair, Hypothesis, and Mutmut for advanced testing.

When you say "formal verification methods", what kind of techniques are you interested in? While using interactive theorem provers will most likely not become very widespread, there are plenty of tools that use formal techniques to give more correctness guarantees. These tools might give some guarantees, but do not guarantee complete functional correctness. WireGuard (VPN tunnel) is I think a very interesting application where they verified the protocol. There are also some tools in use, e.g. Mythril and CrossHair, that focus on detecting bugs using symbolic execution. There's also INFER from Facebook/Meta which tries to verify memory safety automatically. The following GitHub repo might also interest you, it lists some companies that use formal methods: practical-fm

The main difference that Klara bring to the table, compared to similar tool like pynguin and Crosshair is that the analysis is entirely static, meaning that no user code will be executed, and you can easily extend the test generation strategy via plugin loading (e.g. the options arg to the Component object returned from function above is not needed for test coverage).

Just in case you are looking for an alternative approach: if you write contracts in your code, you might also consider crosshair [1] or icontract-hypothesis [2]. If your function/method does not need any pre-conditions then the the type annotations can be directly used.

(I'm one of the authors of icontract-hypothesis.)

[1] https://github.com/pschanely/CrossHair

[2] https://github.com/mristin/icontract-hypothesis

There's a tool for verification of Python programs based on contracts which uses Z3: https://github.com/pschanely/CrossHair

You can use it as part of your CI or during the development (there's even a neat "watch" mode, akin to auto-correct).

Looking at some of your SMT-based projects, I'd love to compare your SMT solver notes with my mine from working on https://github.com/pschanely/CrossHair

Sadly, there aren't a lot of resources on how to use SMT solvers well.