Llama2.rs: One-file Rust implementation of Llama2

1. llama2.rs

   1 3 1,034 8.9 Rust

   A fast llama2 decoder in pure Rust.

   

2. InfluxDB

   www.influxdata.com featured

   InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

   

3. tch-rs

   2 37 4,767 7.5 Rust

   Rust bindings for the C++ api of PyTorch.

   

   I wanted to do something like this but then I would miss on proper CUDA acceleration and lose performance compared to using torchlib.

   I wrote a forgettable llama implementation for https://github.com/LaurentMazare/tch-rs (pytorch's torchlib rust binding).

4. candle

   3 20 17,166 9.5 Rust

   Minimalist ML framework for Rust

   

   Still not ideal but at least you get the same GPU performance you would get on pytorch.

   ...And then I spotted Candle, a new ML framework by the same author: https://github.com/huggingface/candle

   It's all in Rust, self contained, a huge undertaking, but it looks very promising.

Suggest a related project