burn VS candle

View the Project on GitHub

I intend to grow this into an open-source project because deep inside, this is ideally how I would like ComfyUI to be. There's still a long journey ahead for building all the custom nodes, which is especially challenging given that the majority of code for AI workflows is written in Python. However, with my hands-on experience with Candle and Burn libraries, I may be able to get pretty close!

The need to build CubeCL came from the Burn deep learning framework (https://github.com/tracel-ai/burn), where we want to easily build algorithms like in CUDA with a real programming language, while also being able to integrate those algorithms inside a compiler at runtime to fuse dynamic graphs.

Since we don't want to rewrite everything multiple times, it also has to be multi-platform and optimal, so the feature set must be per-device, not per-language. I'm not aware of a tool that does that, especially in Rust (which Burn is written in).

Burn is an emerging deep learning framework written in pure Rust that aims to provide a flexible, efficient, and safe environment for building and training neural networks. With its modular design and strong type system, Burn represents a significant step forward in bringing deep learning to the Rust ecosystem.

7. Burn Burn is a dynamic deep-learning framework built with flexibility and efficiency in mind. If you're into AI or machine learning, this framework offers the ability to explore how Rust can power complex neural networks.

You can use libtorch directly via `tch-rs`, and at present I'm porting over to Burn (see https://burn.dev) which appears incredibly promising. My impression is it's in a good place, if of course not close to the ecosystem of Python/C++. At very least I've gotten my nn models training and running without too much difficulty. (I'm moving to Burn for the thread safety - their `Tensor` impl is `Sync` - libtorch doesn't have such a guarantee.)

Burn has Candle as one of its backends, which I understand is also quite popular.

[package] name = "resnet_burn" version = "0.1.0" edition = "2021" [dependencies] burn = { git = "https://github.com/tracel-ai/burn.git", rev = "75cb5b6d5633c1c6092cf5046419da75e7f74b11", features = ["ndarray"] } burn-import = { git = "https://github.com/tracel-ai/burn.git", rev = "75cb5b6d5633c1c6092cf5046419da75e7f74b11" } image = { version = "0.24.7", features = ["png", "jpeg"] }

I intend to grow this into an open-source project because deep inside, this is ideally how I would like ComfyUI to be. There's still a long journey ahead for building all the custom nodes, which is especially challenging given that the majority of code for AI workflows is written in Python. However, with my hands-on experience with Candle and Burn libraries, I may be able to get pretty close!

You are correct. This project is "on the CPU", so it will not utilize your GPU for computation. If you would like to try out a Rust framework that does support GPUs, [Candle](https://github.com/huggingface/candle/tree/main) may be worth exploring

While looking into how to create text embeddings quickly and directly, we discovered a few helpful tools that allowed us to achieve our goal. Consequently, we created an easy-to-use PHP extension that can generate text embeddings. This extension lets you pick any model from Sentence Transformers on HuggingFace. It is built on the CandleML framework, which is written in Rust and is a part of the well-known HuggingFace ecosystem. The PHP extension itself is also crafted in Rust using the php-ext-rs library. This approach ensures the extension runs fast while still being easy to develop.

Candle already exists[1], and it runs pretty well. Can use both CUDA and Metal backends (or just plain-old CPU).

[1] https://github.com/huggingface/candle

Check out Candle! It's a Deep Learning framework for Rust. You can run LLMs in binaries.

Kalosm is a meta-framework for AI written in Rust using candle. Kalosm supports local quantized large language models like Llama, Mistral, Phi-1.5, and Zephyr. It also supports other quantized models like Wuerstchen, Segment Anything, and Whisper. In addition to local models, Kalosm supports remote models like GPT-4 and ada embeddings.

I have been working on a machine learning library called candle-lora for Candle. It implementes a technique called LoRA (low rank adaptation), which allows you to reduce a model's trainable parameter count by wrapping and freezing old layers.

[2] https://github.com/huggingface/candle/issues/313