vast
thorin2
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
vast
-
Print(“lol”) doubled the speed of my Go function
Most languages target C or LLVM, and C and LLVM have a fundamentally lossy compilation processes.
To get around this, you'd need a hodge podge of pre compiler directives, or take a completely different approach.
I found a cool project that uses a "Tower of IRs" that can restablish source to binary provenance, which, seems to me, to be on the right track:
https://github.com/trailofbits/vast
I'd definitely like to see the compilation processes be more transparent and easy to work with.
-
Compilers and IRS: LLVM IR, SPIR-V, and MLIR
At Trail of Bits, we are creating a new compiler front/middle end for Clang called VAST [1]. It consumes Clang ASTs and creates a high-level, information-rich MLIR dialect. Then, we progressively lower it through various other dialects, eventually down to the LLVM dialect in MLIR, which can be translated directly to MLIR.
Our goals with this pipeline are to enable static analyses that can choose the right abstraction level(s) for their goals, and using provenance, cross abstraction levels to relate results back to source code.
Neither Clang ASTs nor LLVM IR alone meet our needs for static analysis. Clang ASTs are too verbose and lack explicit representations for implicit behaviours in C++. LLVM IR isn't really "one IR," it's a two IRs (LLVM proper, and metadata), where LLVM proper is an unspecified family of dialects (-O0, -O1, -O2, -O3, then all the arch-specific stuff). LLVM IR also isn't easy to relate to source, even in the presence of maximal debug information. The Clang codegen process does ABI-specific lowering takes high-level types/values and transforms them to be more amenable to storing in target-cpu locations (e.g. registers). This actively works against relating information across levels; something that we want to solve with intermediate MLIR dialects.
Beyond our static analysis goals, I think an MLIR-based setup will be a key enabler of library-aware compiler optimizations. Right now, library-aware optimizations are challenging because Clang ASTs are hard to mutate, and by the time things are in LLVM IR, the abstraction boundaries provided by libraries are broken down by optimizations (e.g. inlining, specialization, folding), forcing optimization passes to reckon with the mechanics of how libraries are implemented.
We're very excited about MLIR, and we're pushing full steam ahead with VAST. MLIR is a technology that we can use to fix a lot of issues in Clang/LLVM that hinder really good static analysis.
[1] https://github.com/trailofbits/vast
thorin2
- Can one use lambda calculus as an IR?
-
Is continuation passing style conversion still used as an intermediate language?
There are intermediate representations in actively maintained compilers that rely on CPS at least partly. Examples are SML/NJ (as described here) and Flambda 2 for OCaml. Moreover, there are quite a few recently proposed IRs that are based on CPS. This not only includes IRs for functional languages, like those proposed in Farvadin and Reppy 2020 or Quiring et al. 2021 for example, but also for more imperative languages, see, e.g., Jung et al. 2018 or Leißa et al. 2015 (also see Thorin 2).
What are some alternatives?
clangir - A new (MLIR based) high-level IR for clang.
JuCC - JuCC - Jadavpur University Compiler Compiler
psychec - A compiler frontend for the C programming language
HVM - A massively parallel, optimal functional runtime in Rust
GrayC - GrayC: Greybox Fuzzing of Compilers and Analysers for C
honey-potion - Writing eBPF programs with Elixir!
dfir-orc - Forensics artefact collection tool for systems running Microsoft Windows
Kind - A next-gen functional language
FFMpeg-Online - This repository catalogs a list of FFMpeg commands for different situations. By https://hotpot.ai.
mewa - Compiler-compiler for writing compiler frontends with Lua
exo - A process manager & log viewer for dev
Penguin-Subtitle-Player - An open-source, cross-platform standalone subtitle player