Llama2.rs: One-file Rust implementation of Llama2

• llama2.rs

  3 982 8.9 Rust

  A fast llama2 decoder in pure Rust.

• tch-rs

  37 3,843 7.5 Rust

  Rust bindings for the C++ api of PyTorch.

I wanted to do something like this but then I would miss on proper CUDA acceleration and lose performance compared to using torchlib.

I wrote a forgettable llama implementation for https://github.com/LaurentMazare/tch-rs (pytorch's torchlib rust binding).

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• candle

  17 13,376 9.9 Rust

  Minimalist ML framework for Rust

  

Still not ideal but at least you get the same GPU performance you would get on pytorch.

...And then I spotted Candle, a new ML framework by the same author: https://github.com/huggingface/candle

It's all in Rust, self contained, a huge undertaking, but it looks very promising.

Suggest a related project