bincode VS swc

Ermm... actually I meant something like this: playground, but then I realized it's basically (de)serialization, and I just found that we already have a crate for that: bincode.

One, figure out the bincode format (documented here: https://github.com/bincode-org/bincode/blob/trunk/docs/spec.md) and write your own parser. Maybe a one-off that specifically only handles this one data structure would be fairly straightforward.

Step 1: The Deserialize type requests data from the Deserializer with one of the deserialize_type methods. This gives it an opportunity to provide certain metadata about the type: structs provide a list of fields, enums provide a list of variants, tuples provide a length, etc. Some data formats (notably bincode) require this metadata to drive deserializing, as the wire format is not self-describing. Crucially, the Deserialize type also provides a visitor that is capable of receiving the requested data from the Deserializer.

Alternatively, give Bincode a try.

Like separate instructions? I was thinking if a instruction have unknown length I make sure I have some kind of header field that tells the data length of the instruction so receiver knows when next instruction starts. And I was planning on using Bincode with serde to serialize and dezerialize like structs and stuff.

Isn't this essentially bincode?

This is similar in practice to using abi_stable, and end-users will still receive compiled files, but your plugins will be sandboxed and a single build will work on all platforms. The downside is that it's a bit more work because WebAssembly's support for passing complex data types between the host and the WebAssembly code is in the preliminary stages, so you need to do something like using Serde to encode your data into something like Bincode or MessagePack (or JSON and friends) to hand it off between the host and the plugin.

What kind of access do you need to the data ? You should be able to make a safe api to the Vec class by iterating on in in chunks, and using a closure to translate data between u8 and other representations. ( f32, u32 has the fomr_ne_bytes() / to_ne_bytes() methods ) You could make a helper function that takes a format description ( i.e. "fffuucc" , and calculates the size of the chunk, and generates a closure for reading accessing the data, of the layout is completely dynamic. This closure could use an enum to wrap the different primitive types. ) Or if the layouts are known at compile time , you could use procedural macros to generate code for serializaion / deserialization inot the the [u8] , though https://crates.io/crates/bincode may already do that for you )

Apparently there's a lot of discussion going on about that (3 of the 4 open tickets on the bincode implementation are about it), for example this one.

First, we switched the default compiler for new projects from Babel to SWC (Speedy Web Compiler). SWC is dramatically faster than Babel and requires zero configuration. We’ll continue to support Babel in any project currently using it.

SWC

As the reference explains “**SWC** (Speedy Web Compiler) is an extensible Rust-based platform that can be used for both compilation and bundling. Using SWC with Nest CLI is a great and simple way to significantly speed up your development process.”

This is specifically about breaking the myth that performing expensive self-contained operations (e.g, parsing GraphQL) in a native extension (C, Rust, etc.) is always faster than the interpreted language.

The JS ecosystem has the same problem, people think rewriting everything in Rust will be a magic fix. In practice, there's always the problem highlighted in the post (transitioning is expensive, causes optimization bailouts), as well as the cost of actually getting the results back into Node-land. This is why SWC abandoned the JS API for writing plugins - constantly bouncing back and forth while traversing AST nodes was even slower than Babel (e.g https://github.com/swc-project/swc/issues/1392#issuecomment-...)

Why Speedy Web Compiler ?

FTA is a TypeScript static analysis tool built on the speedy foundations of swc. FTA is fast; capable of analyzing more than 150 files per second on typical hardware, it offers a powerful addition to your code quality toolkit.

Very cool! I'm curious, is this intended for dev tooling?

For example, I could see this (or something similar) being useful as the engine for a typescript language server that would be faster than the standard one

But if it's not aimed at 1:1 with tsc, would it be intended more for something like swc[1]?

Or what would you expect people to use this for, besides just being a cool project to learn from?

[1] https://github.com/swc-project/swc

This is... good news, but I still cannot fathom using the default Typescript compiler for regular development. Seriously, leave the type-checking to your IDE and CICD chain, and switch to using tsx (https://www.npmjs.com/package/tsx) or swc (https://swc.rs/) and you will _immediately_ notice the difference in speed and productivity.