LjTools VS tex-rs

I dug a little more and came across this tool which does seem to have the capability to view all LuaJIT Bytecode. https://github.com/rochus-keller/LjTools

> a large lua game code base, over 4000 files, 1.5 million lines of code

Interesting; how do you manage to keep consistency? Do you have special tools to e.g. detect inadvertent global variables? I once wrote a Smalltalk VM in Lua (https://github.com/rochus-keller/Smalltalk/blob/master/Inter...) which is a much smaller code base but even with this size I quickly would have lost track of e.g. scopes and names without tools I had to write myself (https://github.com/rochus-keller/LJTools).

Most of my languages have VM backends; see e.g. https://github.com/rochus-keller/Oberon; I implemented different backends generating LuaJIT bytecode; a year ago I switched to Mono which is based on ECMA-335; here is a discussion why I switched: https://github.com/rochus-keller/Oberon/releases/tag/IDEv0.9.0; I implemented utility libraries for both LuaJIT and CIL bytecode; see https://github.com/rochus-keller/LjTools/, https://github.com/rochus-keller/Pelib/ and https://github.com/rochus-keller/MonoTools/. I evaluated many VMs and think the mentioned ones are best suited. There were a lot of challenges with both technologies, what is to be expected, and too much to describe here.

LuaJIT is well suited as a backend/runtime environment for custom languages; I did it several times (see e.g. https://github.com/rochus-keller/Smalltalk, https://github.com/rochus-keller/Som/, https://github.com/rochus-keller/Oberon/). I also implemented a bit of infrastructure to ease the reuse: https://github.com/rochus-keller/LjTools. LuaJIT has some limitations though; if you require closures you have to know that the corresponding LuaJIT FNEW bytecode is not yet supported by the JIT, i.e. switches to the interpreter; as a work-around I implemented my own closures; LuaJIT also doesn't support multi-threading, but co-routines; and there is no debugger, and the infrastructure to implement one has limitations (i.e. performance is low when running to breakpoints). For most of my projects this was no issue. Recently I switched to CIL/Mono for my Oberon+ implementation which was a good move. But still I consider LuaJIT a good choice if you can cope with the mentioned limitations. The major advantage of LuaJIT is the small footprint and impressive performance for dynamic languages.

Implementing a VM is certainly interesting, but if you just need a fast backend you could generate LuaJIT bytecode (see e.g. https://github.com/rochus-keller/ljtools/ LuaJitComposer.h/cpp).

If you want to re-use LuaJIT as a backend, have e.g. a look at https://github.com/rochus-keller/ljtools

Recently I mostly use https://github.com/rochus-keller/LjTools#lua-parser-and-ide-features

I have an rust implementation at https://github.com/crlf0710/tex-rs , though it's the plain old TeX without eTeX extensions. Feel free to do any hacking on top of it any one wants to!

I too have recently been interested in TeX and Rust. Apart from Tectonic (wrapper around XeTeX and dvipdfmx) and the attempt to rewrite it in Rust (your first link), I also found [1], an attempt to rewrite TeX itself in Rust. I also understand that you are a supporter of the Tectonic in Rust effort, so hopefully you can fill me in on the current progress.

At first I though that just rewriting XeTeX/dvipdfmx in Rust just for sake of being written in Rust was foolish, because of TeX's untypical memory model. Also because of the manual translation using c2rust (also on C code generated from Pascal code that uses a lot of macros).

But after lookin at it now it seems that the rewrite progressed and the result are parts looking very Rusty, which is nice.

I too had ideas about TeX in Rust, but I think that starting with LuaTeX would be much more beneficial. Apart from LuaTeX having obvious support of scripting in Lua and being very extensible, XeTeX has other disadvantages. See for example [2], which still in my opinion misses many internal differences, where LuaTeX is much superior.

Do you have any tips on how to join the Tectonic / TeX in Rust community? Where can I potentially discuss my foolish TeX/Rust ideas, being very new to Rust?

[1] https://github.com/crlf0710/tex-rs