burn VS tch-rs

I intend to grow this into an open-source project because deep inside, this is ideally how I would like ComfyUI to be. There's still a long journey ahead for building all the custom nodes, which is especially challenging given that the majority of code for AI workflows is written in Python. However, with my hands-on experience with Candle and Burn libraries, I may be able to get pretty close!

The need to build CubeCL came from the Burn deep learning framework (https://github.com/tracel-ai/burn), where we want to easily build algorithms like in CUDA with a real programming language, while also being able to integrate those algorithms inside a compiler at runtime to fuse dynamic graphs.

Since we don't want to rewrite everything multiple times, it also has to be multi-platform and optimal, so the feature set must be per-device, not per-language. I'm not aware of a tool that does that, especially in Rust (which Burn is written in).

Burn is an emerging deep learning framework written in pure Rust that aims to provide a flexible, efficient, and safe environment for building and training neural networks. With its modular design and strong type system, Burn represents a significant step forward in bringing deep learning to the Rust ecosystem.

7. Burn Burn is a dynamic deep-learning framework built with flexibility and efficiency in mind. If you're into AI or machine learning, this framework offers the ability to explore how Rust can power complex neural networks.

You can use libtorch directly via `tch-rs`, and at present I'm porting over to Burn (see https://burn.dev) which appears incredibly promising. My impression is it's in a good place, if of course not close to the ecosystem of Python/C++. At very least I've gotten my nn models training and running without too much difficulty. (I'm moving to Burn for the thread safety - their `Tensor` impl is `Sync` - libtorch doesn't have such a guarantee.)

Burn has Candle as one of its backends, which I understand is also quite popular.

[package] name = "resnet_burn" version = "0.1.0" edition = "2021" [dependencies] burn = { git = "https://github.com/tracel-ai/burn.git", rev = "75cb5b6d5633c1c6092cf5046419da75e7f74b11", features = ["ndarray"] } burn-import = { git = "https://github.com/tracel-ai/burn.git", rev = "75cb5b6d5633c1c6092cf5046419da75e7f74b11" } image = { version = "0.24.7", features = ["png", "jpeg"] }

Great!

For Burn project, we have WebGPU example and I was looking into how we could add automated tests in the browser. Now it seems possible.

Here is the image classification example if you'd like to check out:

https://github.com/tracel-ai/burn/tree/main/examples/image-c...

I wanted to do something like this but then I would miss on proper CUDA acceleration and lose performance compared to using torchlib.

I wrote a forgettable llama implementation for https://github.com/LaurentMazare/tch-rs (pytorch's torchlib rust binding).

I first encountered OCaml's PyTorch bindings because apparently they generate a C wrapper around PyTorch's C++ API, and Rust's PyTorch bindings use OCaml's C wrapper. See: https://github.com/LaurentMazare/tch-rs

You could try looking at the min-GPT example of tch-rs. I'd also strongly suggest watching Karpathy's video to understand what's going on.

If you pefer to see it in code there's a succint gpt implementation here https://github.com/LaurentMazare/tch-rs/blob/main/examples/m...

Rust is fully compatible w/ C bindings, so even Python libraries written in C can be easily set up to work in Rust (and have been). For example, see PyTorch Rust bindings, which actually works faster than in Python because all of the glue code around the C++ API is in Rust instead of Python.

Pytorch

I have looked into this a bit and found some crates which help in loading torchscript models called tch-rs