tch-rs VS cbindgen

I wanted to do something like this but then I would miss on proper CUDA acceleration and lose performance compared to using torchlib.

I wrote a forgettable llama implementation for https://github.com/LaurentMazare/tch-rs (pytorch's torchlib rust binding).

I first encountered OCaml's PyTorch bindings because apparently they generate a C wrapper around PyTorch's C++ API, and Rust's PyTorch bindings use OCaml's C wrapper. See: https://github.com/LaurentMazare/tch-rs

You could try looking at the min-GPT example of tch-rs. I'd also strongly suggest watching Karpathy's video to understand what's going on.

Pytorch

I have looked into this a bit and found some crates which help in loading torchscript models called tch-rs

PyTorch has libtorch as its purely native library. There are also Rust bindings for libtorch:

https://github.com/LaurentMazare/tch-rs

I used this in the past to make a transformer-based syntax annotator. Fully in Rust, no Python required:

https://github.com/tensordot/syntaxdot

The game is in Rust, and so I have been working at using the pytorch Rust bindings, which have an A2C example, so that's what I've been going with. Example here: https://github.com/LaurentMazare/tch-rs/blob/main/examples/reinforcement-learning/a2c.rs

Burn is different: it is built around the Backend trait which encapsulates tensor primitives. Even the reverse mode automatic differentiation is just a backend that wraps another one using the decorator pattern. The goal is to make it very easy to create optimized backends and support different devices and use cases. For now, there are only 3 backends: NdArray (https://github.com/rust-ndarray/ndarray) for a pure rust solution, Tch (https://github.com/LaurentMazare/tch-rs) for an easy access to CUDA and cuDNN optimized operations and the ADBackendDecorator making any backend differentiable. I am now refactoring the internal backend API to make it as easy as possible to plug in new ones.

Golang uses cgo to communicate with external C libraries and Rust uses FFI (Foreign function interfaces) to export C functions. Additionally, a library for automatically generating bindings can be used to generate the header files (.h). In our case, we'll only be exporting two C functions, so using cbindgen is overkill but we'll use it regardless because why not?  Firstly, we'll write the rust functions we want to export. We'll start by creating a new rust library:

As a 2-step approach, you could also probably use https://github.com/mozilla/cbindgen and then Nim's native C FFI.

The tooling for the first kind -- calling Rust from another language -- is a bit less developed, and tends to rely on code generation that doesn't necessarily produce a natural C API. cbindgen, uniffi, cxx, and Diplomat all take this course.

There's also cbindgen for automating the generation of C headers once you've got your code in the right shape and you'll also want to read the Rustonomicon.

You can backport Rust standard library to C using https://github.com/eqrion/cbindgen .

Later we'll see what cbindgen essentially does is receive a configuration and a Rust library and then spit out a C header (.h) file. One might think that what cbindgen is doing might not be that special and can be done by hand. In which some cases that might be true if the project is simple enough. Though additionally as the cbindgen documentation states:

Fuck it, I'll take the bait. Rust allows for amazing C interop by allowing for exporting of C types as a dynamic library, like C. Unlike C, this still allows for Rust's advanced checking at compile time. librsvg is one such library that is being used in the GTK ecosystem that is also almost completely Rust. You can use tools like cbindgen to generate C bindings.

Or alternatively cbindgen to create the header file from the Rust source.

The easiest way to obtain the headers for the rust code is to use the cbingen library.

cbindgen is a good place to start