PyO3 VS wasmtime

The interop shop for Rust and Python is Pyo3. As the only game in town, Pyo3 uses the Foreign Function Interface (FFI) to allow your Rust code to make calls into the CPython binary. This works by agreeing on the Application Binary Interface (ABI), a concept I used during my career at AMD. Core software ftw!

Sorry, "FFI" was a shorthand for "mixing and matching two languages GC expectations, memory layouts, ..." and all the overhead associated with merging something opinionated, like Rust, with something dynamic, like Python. You almost certainly _can_ reduce that overhead further, but unless somebody has gone out of their way to do so, the default expectation for cross-language calls like that should be that somebody opted for maintainable code that has actually shipped instead of shaving off every last theoretical bit of overhead.

It's been a few years, so I really can't tell you exactly what the problem was (other than the general observation that you should try to do nontrivial amounts of work in your python extensions rather than trivial amounts), but PyO3 agrees with the general sentiment [0] [1], or at least did at roughly the same time I was working there.

[0] https://github.com/PyO3/pyo3/issues/679

[1] https://github.com/PyO3/pyo3/issues/1470

This blog post is meant to be an introduction to PyO3 by walking the reader through the build process of a non-trivial extension module in Rust using PyO3. Some familiarity with Python and Rust is recommended to get the most out of this post, basic understanding of Bitcoin concepts may be required to fully grasp the code samples.

Integrating Rust into Python, Edward Wright, 2021-04-12 Examples for making rustpython run actual python code Calling Rust from Python using PyO3 Writing Python inside your Rust code — Part 1, 2020-04-17 RustPython, RustPython Rust for Python developers: Using Rust to optimize your Python code PyO3 (Rust bindings for Python) Musing About Pythonic Design Patterns In Rust, Teddy Rendahl, 2023-07-14

Standards help in a completely different way, too: since all of the HTTP support is now built using wasi-http, applications built with the new SDK that don’t make use of the Spin-specific APIs we also support can run in any environment that supports wasi-http, such as Wasmtime and Node.js (via JCO).

If you’re on Windows, check out the precompiled packages: https://github.com/bytecodealliance/wasmtime/releases

You should look at embedding Wasmtime into your C.

https://github.com/bytecodealliance/wasmtime/tree/main/examp...

And a special thank you to everyone who has been contributing and continues contribute to the WebAssembly ecosystem particularly to the maintainers of the Bytecode Alliance projects, the Wasmtime project and the developers working on WASI and the WebAssembly component model. Their work is instrumental in supporting Spin.

For the audience that would be looking to use the WASM Component Model, and not be an infrastructure implementer of it, whether or not they meet some definition of a method, the component model does define things called resources [1] that have "methods". You'll hold a "handle" to it like you would in your own programming language with the expected drop and/or GC semantics (once implemented [2]) because code is generated to access it like any other FFI like C/C++.

With that in mind, the other confusing thing one may come across is composition vs linking within your WASM runtime that supports the Component Model. When you hear "composition" think of compile-time merging of libraries such that the bundle may have less unresolved dependencies of WASM code/implemented component interfaces. Anything unresolved needs to be linked at runtime with your WASM runtime of choice, like wasmtime [3]. Pretty interesting reading/potential after reading if you ask me -- sounds like you could implement something like a custom Java classloader hierarchy.

But I'd agree with a statement saying it is still a long way for general usage.

[1] https://github.com/WebAssembly/component-model/blob/5a34794d...

[2] https://github.com/WebAssembly/gc

[3] https://github.com/bytecodealliance/wasmtime/blob/ba8131c6bf...

[4] https://www.digitalocean.com/community/tutorials/java-classl...

Since you're using Rust, the Cranelift JIT compiler implements something like this[0] to construct an e-graph for its expression rewriting subsystem called ISLE, however if I'm not mistaken it is for disjoint sets (not intervals), and therefore it does not deal with ordering.

Maybe you can adapt it for your use case and add those new constraints in?

Keep in mind though that this was not written to be in the hot-path itself, you could probably do significantly better by pouring your soul into some SIMD adventure (though SIMD in Rust is usually very annoying to write)

Best of luck, hope this helps!

[0] https://github.com/bytecodealliance/wasmtime/blob/7dcb9bd6ea...

Just a documentation change, fortunately:

https://github.com/bytecodealliance/wasmtime/commits?author=...

They've submitted little documentation tweaks to other projects, too, for example:

https://learn.microsoft.com/en-us/cpp/overview/whats-new-cpp...

I don't know whether this is a formerly-legitimate open source contributor who went rogue, or a deep-cover persona spreading innocuous-looking documentation changes around to other projects as a smokescreen.

WebAssembly is extremely portable. WebAssembly runs on: all major web browsers, V8 runtimes like Node.js, and independent Wasm runtimes like Wasmtime, Lucet, and Wasmer.