burn VS candle

Release Note: https://github.com/burn-rs/burn/releases/tag/v0.10.0

The comprehensive, open-source deep learning framework in Rust, Burn, has recently undergone significant advancements in its latest release, highlighted by the addition of The Burn Book 🔥. There has never been a better moment to embark on your deep learning journey with Rust, as this book will guide you through your initial project, providing extensive explanations and links to relevant resources.

Burn (deep learning framework in rust) has WGPU backend (WebGPU) already. Check it out https://github.com/burn-rs/burn. It was released recently.

For more details about the latest release see the release notes: https://github.com/burn-rs/burn/releases/tag/v0.8.0.

Tract is the most well known ML crate in Rust, which I believe can compile to WASM - https://github.com/sonos/tract/. Burn may also be useful - https://github.com/burn-rs/burn.

We, the burn team, are working on the wgpu backend (WebGPU) for Burn deep learning framework. You can check out the current state: https://github.com/burn-rs/burn/tree/main/burn-wgpu

Here is the project: https://github.com/burn-rs/burn

Disclaimer, I'm the main author of Burn https://burn-rs.github.io.

The Candle framework from Hugging Face became our foundation. Unlike PyTorch, Candle runs inference without Python overhead. Combined with the ONNX Runtime bindings for Rust, we could execute our Gemma-3 embedding model directly on CPU with SIMD optimisations that Python simply cannot match.

Was getting annoyed having to work out which video call platform to use for intro meetings, so I’m currently working on https://simplevc.blazingbanana.com.

It’s a private, simple browser video chat. No accounts. You can create a new chat or join via link/code. It also has in browser transcription using Whisper (using https://github.com/huggingface/candle/).

Currently fighting getting background blur to work smoothly on Chrome on Android, which is proving surprisingly harder than getting browser transcription working...

Also still working on small updates to Whistle, my completely free offline transcription app, available on Android, macOS, Windows and Linux (with CUDA builds for Windows/Linux) - https://blazingbanana.com/work/whistle

If you need WASM, I think Candle is your current best bet: https://github.com/huggingface/candle

[package] name = "rust-phi3" version = "0.1.0" edition = "2021" [dependencies] anyhow = "1.0" tokenizers = "0.19.1" clap = { version = "4.4", features = ["derive"] } candle-core = { git = "https://github.com/huggingface/candle.git", branch = "main" } candle-transformers = { git = "https://github.com/huggingface/candle.git", branch = "main" } candle-nn = { git = "https://github.com/huggingface/candle.git", branch = "main" }

The Rust ML ecosystem is still young. There are great projects like burn and candle, but there's room for more exploration and different approaches. Even if CorrosiveNet never becomes production-ready, understanding how ML frameworks work under the hood is valuable.

I intend to grow this into an open-source project because deep inside, this is ideally how I would like ComfyUI to be. There's still a long journey ahead for building all the custom nodes, which is especially challenging given that the majority of code for AI workflows is written in Python. However, with my hands-on experience with Candle and Burn libraries, I may be able to get pretty close!

You are correct. This project is "on the CPU", so it will not utilize your GPU for computation. If you would like to try out a Rust framework that does support GPUs, [Candle](https://github.com/huggingface/candle/tree/main) may be worth exploring

While looking into how to create text embeddings quickly and directly, we discovered a few helpful tools that allowed us to achieve our goal. Consequently, we created an easy-to-use PHP extension that can generate text embeddings. This extension lets you pick any model from Sentence Transformers on HuggingFace. It is built on the CandleML framework, which is written in Rust and is a part of the well-known HuggingFace ecosystem. The PHP extension itself is also crafted in Rust using the php-ext-rs library. This approach ensures the extension runs fast while still being easy to develop.