LjTools VS PeLib

I dug a little more and came across this tool which does seem to have the capability to view all LuaJIT Bytecode. https://github.com/rochus-keller/LjTools

> a large lua game code base, over 4000 files, 1.5 million lines of code

Interesting; how do you manage to keep consistency? Do you have special tools to e.g. detect inadvertent global variables? I once wrote a Smalltalk VM in Lua (https://github.com/rochus-keller/Smalltalk/blob/master/Inter...) which is a much smaller code base but even with this size I quickly would have lost track of e.g. scopes and names without tools I had to write myself (https://github.com/rochus-keller/LJTools).

Most of my languages have VM backends; see e.g. https://github.com/rochus-keller/Oberon; I implemented different backends generating LuaJIT bytecode; a year ago I switched to Mono which is based on ECMA-335; here is a discussion why I switched: https://github.com/rochus-keller/Oberon/releases/tag/IDEv0.9.0; I implemented utility libraries for both LuaJIT and CIL bytecode; see https://github.com/rochus-keller/LjTools/, https://github.com/rochus-keller/Pelib/ and https://github.com/rochus-keller/MonoTools/. I evaluated many VMs and think the mentioned ones are best suited. There were a lot of challenges with both technologies, what is to be expected, and too much to describe here.

LuaJIT is well suited as a backend/runtime environment for custom languages; I did it several times (see e.g. https://github.com/rochus-keller/Smalltalk, https://github.com/rochus-keller/Som/, https://github.com/rochus-keller/Oberon/). I also implemented a bit of infrastructure to ease the reuse: https://github.com/rochus-keller/LjTools. LuaJIT has some limitations though; if you require closures you have to know that the corresponding LuaJIT FNEW bytecode is not yet supported by the JIT, i.e. switches to the interpreter; as a work-around I implemented my own closures; LuaJIT also doesn't support multi-threading, but co-routines; and there is no debugger, and the infrastructure to implement one has limitations (i.e. performance is low when running to breakpoints). For most of my projects this was no issue. Recently I switched to CIL/Mono for my Oberon+ implementation which was a good move. But still I consider LuaJIT a good choice if you can cope with the mentioned limitations. The major advantage of LuaJIT is the small footprint and impressive performance for dynamic languages.

Implementing a VM is certainly interesting, but if you just need a fast backend you could generate LuaJIT bytecode (see e.g. https://github.com/rochus-keller/ljtools/ LuaJitComposer.h/cpp).

If you want to re-use LuaJIT as a backend, have e.g. a look at https://github.com/rochus-keller/ljtools

Recently I mostly use https://github.com/rochus-keller/LjTools#lua-parser-and-ide-features

I'm usually working with the third edition of the ISO 23271 standard, which corresponds to ECMA 335 5th Edition. There is also the book ".NET IL Assembler" (2014) by Serge Lidin which gives some additional context and a good structure to learn the details. My compiler is written in C++, not depending on the .Net framework; I don't use the reflection and emit features of .Net, but https://github.com/rochus-keller/Pelib/ to generate assemblies; also the generated code only requires minimal mscorlib.dll features, so I can use a minimal CLR.

Most of my languages have VM backends; see e.g. https://github.com/rochus-keller/Oberon; I implemented different backends generating LuaJIT bytecode; a year ago I switched to Mono which is based on ECMA-335; here is a discussion why I switched: https://github.com/rochus-keller/Oberon/releases/tag/IDEv0.9.0; I implemented utility libraries for both LuaJIT and CIL bytecode; see https://github.com/rochus-keller/LjTools/, https://github.com/rochus-keller/Pelib/ and https://github.com/rochus-keller/MonoTools/. I evaluated many VMs and think the mentioned ones are best suited. There were a lot of challenges with both technologies, what is to be expected, and too much to describe here.

And as I understand now achieving optimal performance with CoreCLR requires inside know-how of RyuJIT and Roslyn (which I don't have anyway), i.e. ECMA-335 alone is not sufficient; the actual compilation target to achieve optimal performance with CoreCLR is therefore Roslyn, not ECMA-335; of course I could generate C# instead of IL, but then I would depend on the framework and slow down the compiler. My current code generator based on https://github.com/rochus-keller/PeLib is about 30 times faster than ILASM.