Building the fastest Lua interpreter.. automatically

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • luajit-remake

    An ongoing attempt to re-engineer LuaJIT from scratch

  • This seems like an awesome way of writing faster interpreters – i.e. not in assembly, but in C++ snippets you stitch together with a tool.

    I did peek at the deegen tool a bit, and it seems quite large? https://github.com/luajit-remake/luajit-remake/tree/master/d...

    I would be interested in an overview of all the analysis it has to do, which as I understand is basically “automated Mike Pall”

    FWIW I think this is the hand-written equivalent with LuaJIT’s dynasm tool: https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/vm_x64.dasc (just under 5000 lines)

    Also there are several of these files with no apparent sharing, as you would get with deegen:

    https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/vm_x86.dasc

    https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/vm_ppc.dasc

  • qbe-rs

    QBE IR in natural Rust data structures

  • GCC is written in C++ these days, so something like QBE(https://c9x.me/compile/) would be needed.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
  • idel

    A low-level virtual machine for mobile code

  • I don't know -- there were at least a couple papers about it, but I haven't been following. At the time it was first announced I used some of the ideas in one of my projects (https://github.com/darius/idel/blob/master/src/opcodes.awk) but unfortunately didn't develop it further.

  • llvm-project

    The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.

  • Yeah, I did a quick check in LLVM at some point to see what it does (query I relied on: https://github.com/llvm/llvm-project/search?q=getPredictable...) and all the results seemed to be exclusively code motion or deciding how to lower a branch. Similarly cold path outlining seemed to just want to split the function in a fairly simple manner rather than doing anything beyond that. Perhaps I missed something, but I think the current hints are just to help the branch predictor or instruction cache rather than significantly alter codegen.

    Unfortunately, I don't have much to share at the moment besides my thoughts; I've done a few small tests but haven't been able to really do a full implementation yet. The primary consumer of this work would be iSH (https://github.com/ish-app/ish), which has a need for a fast interpreter, so you can at least take a look at the current implementation to see what we'd like to replace. The nature of the project means that most of my time has been tied up in things like making sure that keyboard avoidance is set up correctly and that users can customize the background color of their terminal :/

    With that said, I'd be happy to chat more if you'd like–feel free to send me an email or whatever. Not sure I can say I'm at the state of the art yet, but perhaps we can get there :)

  • ish

    Linux shell for iOS

  • Yeah, I did a quick check in LLVM at some point to see what it does (query I relied on: https://github.com/llvm/llvm-project/search?q=getPredictable...) and all the results seemed to be exclusively code motion or deciding how to lower a branch. Similarly cold path outlining seemed to just want to split the function in a fairly simple manner rather than doing anything beyond that. Perhaps I missed something, but I think the current hints are just to help the branch predictor or instruction cache rather than significantly alter codegen.

    Unfortunately, I don't have much to share at the moment besides my thoughts; I've done a few small tests but haven't been able to really do a full implementation yet. The primary consumer of this work would be iSH (https://github.com/ish-app/ish), which has a need for a fast interpreter, so you can at least take a look at the current implementation to see what we'd like to replace. The nature of the project means that most of my time has been tied up in things like making sure that keyboard avoidance is set up correctly and that users can customize the background color of their terminal :/

    With that said, I'd be happy to chat more if you'd like–feel free to send me an email or whatever. Not sure I can say I'm at the state of the art yet, but perhaps we can get there :)

  • llvm-project

    The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. Note: the repository does not accept github pull requests at this moment. Please submit your patches at http://reviews.llvm.org. (by haberman)

  • Yes, I've already experimented with doing this, see: https://github.com/haberman/llvm-project/commit/e8d9c75bb35c...

    But when I tried it on my actual code, the results weren't quite as good as I hoped, due to sub-optimal register allocation.

  • LuaJIT

    Mirror of the LuaJIT git repository

  • This seems like an awesome way of writing faster interpreters – i.e. not in assembly, but in C++ snippets you stitch together with a tool.

    I did peek at the deegen tool a bit, and it seems quite large? https://github.com/luajit-remake/luajit-remake/tree/master/d...

    I would be interested in an overview of all the analysis it has to do, which as I understand is basically “automated Mike Pall”

    FWIW I think this is the hand-written equivalent with LuaJIT’s dynasm tool: https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/vm_x64.dasc (just under 5000 lines)

    Also there are several of these files with no apparent sharing, as you would get with deegen:

    https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/vm_x86.dasc

    https://github.com/LuaJIT/LuaJIT/blob/v2.1/src/vm_ppc.dasc

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts