proposal-temporal VS Graal

Temporal is the new JavaScript standard for date manipulations. It has a long, difficult journey that I want to share with you. Let's explore whether it's ready for production.

The Temporal API, which is currently in Stage 3, is being developed to improve the current Date object, which is mostly known for its unexpected behavior. Today there are lots of date-time libraries for JavaScript, such as date-fns, moment, js-joda and a huge number of others. They all try to help with unpredictable and unexpected behavior of JavaScript Date object by adding features such as timezones, date parsing and almost everything else.

I'm developing vremel, an utility library for Temporal API (similar to date-fns for Date). It's a pure ESM package1 and written in TypeScript.

Even with the initial pitfalls of JavaScript dates, the ecosystem managed to tackle those challenges with good libraries. Some use the native JavaScript date while others have instead created their own robust tools to make up for the quirks JavaScript dates might have. With the JavaScript Temporal API about to roll out, working with dates and time in JavaScript will only improve.

Temporal API

> TruffleRuby even had the extremely big brain idea of running a Truffle interpreter for C for native extensions […]

TruffleC was a research project and the first attempt of running C code on Truffle that I'm aware of. It directly interpreted C source code and while that works for small self-contained programs, you quickly run into a lot of problems as soon as you want to run larger real world programs. You need everything including the C library available as pure C code and you have to deal with the fact that a lot of C code uses some UB/IB. In addition, your C parser has to fully adhere to the C standard and once you want to support C++ too because a lot of code is written in C++, you have to re-start from scratch. I don't know if TruffleC was ever released as open source.

The next / current attempt is Sulong which uses LLVM to compile C/C++/Rust/… to LLVM IR ("bitcode") and then directly interprets that bitcode. It's a lot better, because you don't have to write your own complete C/C++/… parser/compiler, but bitcode still has various limitations. Essentially as soon as the program uses handwritten assembler code somewhere, or if it does some low level things like setjmp/longjmp, things get hairy pretty quickly. Bitcode itself is also platform dependent (think of constants/macros/… that get expanded during compilation), you still need all code / libraries in bitcode, every language uses a just so slightly different set of IR nodes and requires a different runtime library so you have to explicitly support them, and even then you can't make it fully memory safe because typical programs will just break. In addition, the optimization level you choose when compiling the source program can result in very different bitcode with very different IR nodes, some of which were not supported for a long time (e.g., everything related to vectorization). Sulong can load libraries and expose them via the Truffle FFI, and it can be used for C extensions in GraalPython and TruffleRuby AFAIK. It's open source [1] and part of GraalVM, so you can play around with it.

Another research project was then to directly interpret AMD64 machine code and emulate a Linux userspace environment, because that would solve all the problems with inline assembly and language compatibility. Although that works, it has an entirely different set of problems: Graal/Truffle is simply not made for this type of code and as a result the performance is significantly worse than Sulong. You also end up re-implementing the Linux syscall interface in your interpreter, you have to deal with all the low level memory features that are available on Linux like mmap/mprotect/... and they have to behave exactly as on a real Linux system, and you can't easily export subroutines via Truffle FFI in a way that they also work with foreign language objects. It does work with various guest languages like C/C++/Rust/Go/… without modifying the interpreter, as long as the program is available as native Linux/AMD64 executable and doesn't use any of the unimplemented features. This project is also available as open source [2], but its focus somewhat shifted to using the interpreter for execution trace based program analysis.

Things that aren't supported by any of these projects AFAIK are full support for multithreading and multiprocessing, full support for IPC, and so on. Sulong partially solves it by calling into the native C library loaded in the VM for subroutines that aren't available as bitcode and aborting on certain unsupported calls like fork/clone, but then you obviously lose the advantage of having everything in the interpreter.

The conclusion is, whatever you try to interpret C/C++/… code, get ready for a world of pain and incompatibilities if you intend to run real world programs.

[1] https://github.com/oracle/graal/tree/master/sulong

[2] https://github.com/pekd/tracer/tree/master/vmx86

They are indeed scary, but one thing that can make them less so is writing them in a high level language. A nice code base to study if you're new to GCs is the GenScavange module of SubstrateVM, a JVM written entirely in Java. Start here:

https://github.com/oracle/graal/blob/master/substratevm/src/...

It gets compiled to native code for use. It's actually a bit of a dialect of Java because of course, you need low level memory and stack access. So you can see at parts there is use of special APIs that get compiled to things like stack allocations, so it can avoid allocating on the heap whilst working with it. Even so the "business logic" of a GC can be easily seen here, especially if you don't know C++.

Contrary to what vocal Kotlin advocates might believe, Kotlin only matters on Android, and that is thanks to Google pushing it no matter what.

https://spectrum.ieee.org/the-top-programming-languages-2023

https://snyk.io/reports/jvm-ecosystem-report-2021/

And even so, they had to conceed Android and Kotlin on their own, without the Java ecosystem aren't really much useful, thus ART is now updatable via Play Store, and currently supports OpenJDK 17 LTS on Android 12 and later devices.

As for your question regarding numbers, mostly Java 74.6%, C++ 13.7%, on the OpenJDK, other JVM implementations differ, e.g. GraalVM is mostly Java 91.8%, C 3.6%.

https://github.com/openjdk/jdk

https://github.com/oracle/graal

Two examples from many others, https://en.wikipedia.org/wiki/List_of_Java_virtual_machines

Pkl was built using the GraalVM Truffle framework. So it supports runtime compilation using Futurama Projections. We have been working with Apple on this for a while, and I am quite happy that we can finally read the sources!

https://github.com/oracle/graal/tree/master/truffle

Disclaimer: graalvm dev here.

That's pretty interesting. It's not as aggressive as Bee sounds, but the Espresso JVM is somewhat similar in concept. It's a full blown JVM written in Java with all the mod cons, which can either be compiled ahead of time down to memory-efficient native code giving something similar to a JVM written in C++, or run itself as a Java application on top of another JVM. In the latter mode it obviously doesn't achieve top-tier performance, but the advantage is you can easily hack on it using all the regular Java tools, including hotswapping using the debugger.

When run like this, the bytecode interpreter, runtime system and JIT compiler are all regular Java that can be debugged, edited, explored in the IDE, recompiled quickly and so on. Only the GC is provided by the host system. If you compile it to native code, the GC is also written in Java (with some special conventions to allow for convenient direct memory access).

What's most interesting is that Espresso isn't a direct translation of what a classical C++ VM would look like. It's built on the Truffle framework, so the code is extremely high level compared to traditional VM code. Details like how exactly transitions between the interpreter/compiled code happen, how you communicate pointer maps to the GC and so on are all abstracted away. You don't even have to invoke the JIT compiler manually, that's done for you too. The only code Espresso really needs is that which defines the semantics of the Java bytecode language and associated tools like the JDWP debugger protocol.

https://github.com/oracle/graal/tree/master/espresso

This design makes it easy to experiment with new VM features that would be too difficult or expensive to implement otherwise. For example it implements full hotswap capability that lets you arbitrarily redefine code and data on the fly. Espresso can also fully self-host recursively without limit, meaning you can achieve something like what's described in the paper by running Espresso on top of Espresso.

I'm also using GraalVM if that's of any help.

Quarkus is one of Java frameworks for microservices development and cloud-native deployment. It is developed as container-first stack and working with GraalVM and HotSpot virtual machines (VM).

Apologies, I didn't mean to imply DCEVM went poof, just that I was sad it didn't make it into OpenJDK so one need not do JDK silliness between the production one and the "debugging one" since my experience is that's an absolutely stellar way to produce Heisenbugs

And I'll be straight: Graal scares me 'cause Oracle but I just checked and it looks to the casual observer that it's straight-up GPLv2 now so maybe my fears need revisiting: https://github.com/oracle/graal/blob/vm-23.1.0/LICENSE

> to be compiled to a single executable is a strength that Java does not have

I think this is very outdated claim: https://www.graalvm.org/