tch-rs VS rustc_codegen_cranelift

I wanted to do something like this but then I would miss on proper CUDA acceleration and lose performance compared to using torchlib.

I wrote a forgettable llama implementation for https://github.com/LaurentMazare/tch-rs (pytorch's torchlib rust binding).

I first encountered OCaml's PyTorch bindings because apparently they generate a C wrapper around PyTorch's C++ API, and Rust's PyTorch bindings use OCaml's C wrapper. See: https://github.com/LaurentMazare/tch-rs

You could try looking at the min-GPT example of tch-rs. I'd also strongly suggest watching Karpathy's video to understand what's going on.

If you pefer to see it in code there's a succint gpt implementation here https://github.com/LaurentMazare/tch-rs/blob/main/examples/m...

Rust is fully compatible w/ C bindings, so even Python libraries written in C can be easily set up to work in Rust (and have been). For example, see PyTorch Rust bindings, which actually works faster than in Python because all of the glue code around the C++ API is in Rust instead of Python.

Pytorch

I have looked into this a bit and found some crates which help in loading torchscript models called tch-rs

Windows is supported. See https://github.com/rust-lang/rustc_codegen_cranelift/issues/....

> When this happens, it seems like it'll be possible to get the LLVM bits out of the bootstrap process and lead to a fully self-hosted Rust.

What do you mean by "when this happens"? GP's point is that this has already happened: the Cranelift backend is feature-complete from the perspective of the language [0], except for inline assembly and unwinding on panic. It was merged into the upstream compiler in 2020 [1], and a compiler built with only the Cranelift backend is perfectly capable of building another compiler. LLVM hasn't been a necessary component of the Rust compiler for quite some time.

[0] https://github.com/bjorn3/rustc_codegen_cranelift

[1] https://github.com/rust-lang/rust/pull/77975

Note that the Cranelift codegen will eventually become standard for debug builds to speed them up.

Maybe one day we'll have rustc b3 backend like what they did with Cranelift

Additionally, there is gcc codegen for rustc (https://github.com/rust-lang/rustc_codegen_gcc), which is not a compiler per se, but an alternative code generator, with more architectures supported and other nice things. It's also coming along, but there's still a lot of work to do there too. There's also Cranelift codegen (https://github.com/bjorn3/rustc_codegen_cranelift), which is designed to make debug builds faster, but this is not as exciting/useful as the other 2.

For the instant reloading, that's in Sycamore, so you should speak to its devs, but as for the alternative compiler backend, it's not my project, but it uses Cranelift and works pretty well! See https://github.com/bjorn3/rustc_codegen_cranelift for details.

The more immediate goal of "distribute the cranelift backend as a rustup component" has been making good progress and seems like it might happen relatively soon https://github.com/bjorn3/rustc_codegen_cranelift/milestone/...