awesome-jit
Cwerg
Our great sponsors
awesome-jit | Cwerg | |
---|---|---|
5 | 59 | |
387 | 398 | |
- | - | |
3.1 | 9.6 | |
9 months ago | 3 days ago | |
Python | ||
- | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
awesome-jit
-
Good high-level JIT libraries for an easy project?
https://github.com/wdv4758h/awesome-jit. Take a look on MIR jit.
-
MiniVM: A minimal cross-language runtime that beats C/luajit on some benchmarks
Out of curiosity, are you planning on (progressively, slowly) rolling your own JIT, or using something like DynASM (https://luajit.org/dynasm_features.html), libFirm (https://pp.ipd.kit.edu/firm/Features.html), or some other preexisting thing (eg https://github.com/wdv4758h/awesome-jit) in the space?
FWIW, I understand that LuaJIT gets some of its insane real-world performance from a JIT and VM design that's effectively shrink-wrapped around Lua semantics/intrinsics - it's not general-purpose. I've read (unfortunately don't remember exactly where) that people have tried to run off with the JIT and VM and use it in other projects, but never succeeded because of the tight coupling.
In the same way, while a bespoke 64/32-bit x86+ARM JIT would be a reasonable undertaking, it could make for a pretty interesting target with a potentially wide-ranging set of uses.
For example, it could be the VM+JIT combination that all those people dissecting LuaJIT were looking for :).
I could see something like this becoming an attractive option for games that want an exceptionally simple runtime. Sort of like a scaled-down NekoVM (nee Haxe).
Broadly speaking, I get the (potentially incorrect) impression (from a very naive/inexperienced POV) that MiniVM+JIT would be looking to close a more widely-scoped, higher-level loop than something like libFirm would be. So it'd be closer to Cling (https://root.cern/cling/) than raw LLVM, perhaps (albeit with 0.01% of the code size :D). It is for this reason that I kind of pause for a minute and ponder that a fully integrated JIT could be a pretty good idea. It would absolutely make the irreducible complexity of the project balloon, with reasonable motivation likely necessary to maintain cohesion.
If I were to backseat-drive for a minute :) the first thing I'd rant about is how attractive modern JITs need trivial ways to verify code correctness, both online (how exactly was *this* specific generated code constructed - so, straightforward logging) but also (and particularly) offline (humans staring at the JIT source code and mentally stepping through its behavior - and succeeding (miracles!!) because the code is small and well-written). If the JIT implemented in such a straightforward manner, end users wanting to run potentially malicious user-supplied code with high performance in potentially security-sensitive settings might be attracted to the project. (Mike Pall made bank for a while while CloudFlare was using LuaJIT for its WAF... ahem...)
I came across this reference of how to break out of LuaJIT 2.1 (2015) a while back: https://www.corsix.org/content/malicious-luajit-bytecode - and every time I take a look at the code I switch away from the tab :) (and sometimes step away from the computer for a minute :D). It's solely a demonstration of "this is how it would work", and clarifies that LuaJIT makes no sandbox guarantees about the code it executes, but reading through it, the amount of Stuff™ going on represents a surface area that to me (naively) seems just... like LuaJIT as a whole is generally too large to easily reason about from a security standpoint (oh yeah, besides being written in assembly language...). This might be inexperience speaking, but I can't help but wonder whether a smaller, simpler implementation might be able to implement a secure JIT; for all I know this might be an impossible P=NP pipe dream I haven't fully grasped yet, I guess what I'm trying to figure out is whether "small enough to mentally reason through" and "large enough to do a few things quickly" have practical overlap?
---
On an unrelated note, something I discovered recently and which I thought I'd idly mention is that JITs might seem to have a bit of a hard time on Android. I can't (yet) tell if this is LuaJIT-specific or "anything that's a JIT"-specific: KOReader (Android eBook reader, implemented entirely using LuaJIT) has a bunch of very scary magic Things™ it seems to need to do to make LuaJIT even work at all on Android (https://github.com/koreader/android-luajit-launcher/blob/mas...), due to a apparently-current issue causing issues across different domains (https://github.com/LuaJIT/LuaJIT/issues/285), which has been apparently cropping up going back years (https://www.freelists.org/post/luajit/Android-performance-dr... (2013)). KOReader has even nontrivially patched LuaJIT's C code in places (https://github.com/koreader/android-luajit-launcher/blob/bb0...) with purposes I am yet to fully understand (it might just be for debugging). I happened to be considering idly playing around with Lua on Android (currently fascinated with interpreted/JITed runtimes) and after stumbling on this I'm debating whether to use Lua instead, ha. I've been meaning to ask around on the LuaJIT list and wherever KOReader discusses stuff to learn more, after focusing on actually getting LuaJIT linked into an Android project and poking around. Haven't got to it yet. This could be an absolutely massive red herring that I'm going deer-in-headlights about because it just looks off-in-the-weeds, or potentially significant. It might also be absolutely irrelevant, but as I noted I'm not (yet) sure how to tell.
-
GNU JIT bytecode VM?
Here is a good overview: https://github.com/wdv4758h/awesome-jit
- Awesome jit
- Awesome-JIT – A curated list of JIT frameworks, libraries, software, resources
Cwerg
-
Cwerg: C-like language that can be implemented in 10kLOC
Perhaps these have already been dealt with and I'm missing critical information. If so, my apologies. Great work, in any case.
[1] https://github.com/robertmuth/Cwerg/tree/master/FrontEnd#dis...
-
Where can I find resources and guides on how to build compiler backends?
Cwerg has backend that can be used as JIT and is written with readability in mind. Additional documentation can be found here: https://github.com/robertmuth/Cwerg/tree/master/Docs
- Most important language features not touched in the book "Crafting Interpreters"?
- Lack of resources in creating Assemblers from scratch.
-
Minimum ISA Capabilities to Support Most (Non-Interactive) Programs?
I defined a basic ISA-like IR for Cwerg. It has unlimited registers and no constraints on immediates.
-
How do you design a compiler and a language?
entire compiler front end ast nodes
-
Syntax Design
I was also going down the path of bike shedding concrete syntax for my language Cwerg before pulling the plug on that effort and just using s-exprs. I managed to make the s-expr quite succinct by carefully choosing the order of arguments so I can omit optional ones. Also very helpful was to use square brackets for list, e.g. (call fun-name [arg1 arg2]). This simplifies parsing a little bit and is easier on the eye. Here are some Code Examples
-
November 2022 monthly "What are you working on?" thread
I am iterating over the languages features for Cwerg's Frontend which aims to be a low level language with about the complexity of C but with some of the comforts of modern languages. I am especially happy with the choice of adding sum types. Relative to C the current feature set looks like this: Removed: * arrays decay to pointers * bitfields * separate compilation (more of a backend issue) * pre-processor * varargs * implcit type conversions * (untagged) unions * ++/-- * comma operator * implicitly nullable pointers * goto
- typed asts and codegen
- Features Compendium
What are some alternatives?
android-luajit-launcher - Android NativeActivity based launcher for LuaJIT, implementing the main loop within Lua land via FFI
mir - A lightweight JIT compiler based on MIR (Medium Internal Representation) and C11 JIT compiler and interpreter based on MIR
LuaJIT - Mirror of the LuaJIT git repository
tinycc - Unofficial mirror of mob development branch
paka - Paka language
asmjit - Low-latency machine code generation
minivm - A VM That is Dynamic and Fast
bluebird - A work-in-progess programming language modeled after Ada and C++
Papers
konna - A fast functional language based on two level type theory
os49 - basically minivm os
boring-lang - A very boring programming language