Rust Is the Future of JavaScript Infrastructure

• markdoc

  21 6,975 6.1 TypeScript

  A powerful, flexible, Markdown-based authoring framework.

  

I'm bullish on Rust, but there's a long way still to go. The overhead of passing values across the boundary between JavaScript and Rust is quite high. There are a lot of cases where you want to be able to provide a dynamic configuration to Rust, ideally in JavaScript, and that's still pretty costly from a performance perspective.

One of my projects (https://markdoc.dev/) is a Markdown dialect that supports custom tags and a React renderer. I recently experimented with implementing a parser for it in Rust in order to increase performance. My Rust-based parser is significantly faster than my existing JavaScript parser, but then I have to serialize the AST in order to move it from Rust to JavaScript. I'd like to implement the entire processor in Rust, but I need to let users define custom tags in JavaScript, and the overhead of going back and forth is far from ideal.

I'm hopeful that the recently-ratified Wasm GC proposal—which introduces managed structs and arrays that don't cost anything to pass between the Wasm environment and JavaScript—will help a lot. But it's going to take awhile for Wasm GC features to land in LLVM and be properly supported in Rust.

• oil

  234 2,720 9.9 Python

  Oils is our upgrade path from bash to a better language and runtime. It's also for Python and JavaScript users who avoid shell!

  

Yup, more generally, I'd say that statically typed languages that compile to native code (Rust, Go, C++, etc.) are necessary to achieve good performance when writing linters / formatters, compilers, and interpreters.

I think of these "AST- or graph-based workloads", as opposed to numeric workloads. (I'm looking for references/measurements specifically on these types of workloads.)

You could frame it as a "failure of JITs in the 2010's". JavaScript isn't a good language to write the TypeScript compiler or a linter, because v8 isn't fast enough.

The semantics of JavaScript do not allow v8 to be fast enough.

IIRC this was precisely why the Dart project was started more than 10 years ago by the v8 authors at Google. They were spending a lot of time looking into why web page performance was falling off various cliffs in v8. They realized they needed to change the LANGUAGE in order to be able to write fast programs, like writing Google Docs and GMail in the browser.

JITs are fast in common cases, but they not only have big costs in terms of memory (code storage) and startup time, but they're hard to engineer with!

---

Emery Berger has memorable framing of this -- "Perl, Python, PHP, and JS are the irrational exuberance" languages.

https://www.sigarch.org/from-heavy-metal-to-irrational-exube...

That is, he says that for awhile we thought that clock speeds would increase, and JITs would get better, and so we could design languages with semantics without regard to performance -- language that almost require slow implementations.

(I think his take is about 50% true. The other 50% is that dynamic languages simply allowed people to produce popular and useful software at a greater rate, especially for the web, so we ended up with a lot of software written in dynamic languages! Doing web apps in Java vs. Ruby/Python/JS is a huge difference.)

---

This also tracks with my experience with https://www.oilshell.org, where we reverse-engineered the shell in an experimental fashion with Python, and then evolved that implementation into a statically typed language that generates C++ (using MyPy, ASDL, and algebraic data types). This core of the program is the elaborate and strongly typed "lossless syntax tree", which is basically what's used in linters and formatters.

Even though I've been using both C++ and Python for >20 years, it is a little shocking how much worse Python is for AST- and graph-based workloads.

I'd say that Python and JS are 10x as slow as native code for "business" and "web app" workloads, but they're closer to 100x too slow for ASTs and graphs. This is because of all the allocation and GC overhead, in both time and space.

So the funny thing is that Oil is the most statically-typed shell implementation, even though it's nominally written in Python :) It uses fine-grained static types, where as shells written in C use a homogeneous "WORD*" representation in C, and strings with control codes embedded in them. I should probably write a blog post about that ...

• SurveyJS

  surveyjs.io sponsored

  Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.

  

• Next.js

  2,039 120,313 10.0 JavaScript

  The React Framework

  

FWIW, that doesn't take into account that Next actually has its own wrapper around SWC, and also vendors it. I think it's probably the largest consumer of SWC:

- https://github.com/vercel/next.js/tree/canary/packages/next-...

Those add up pretty fast:

- https://npmtrends.com/@next/swc-linux-x64-gnu-vs-@next/swc-w...

Suggest a related project