A fast llama2 decoder in pure Rust.
Rust bindings for the C++ api of PyTorch.
I wanted to do something like this but then I would miss on proper CUDA acceleration and lose performance compared to using torchlib.
I wrote a forgettable llama implementation for https://github.com/LaurentMazare/tch-rs (pytorch's torchlib rust binding).
Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.
Minimalist ML framework for Rust
Still not ideal but at least you get the same GPU performance you would get on pytorch.
...And then I spotted Candle, a new ML framework by the same author: https://github.com/huggingface/candle
It's all in Rust, self contained, a huge undertaking, but it looks very promising.
1 project | news.ycombinator.com | 7 Dec 2023
llm: a Rust crate/CLI for CPU inference of LLMs, including LLaMA, GPT-NeoX, GPT-J and more
7 projects | /r/rust | 9 May 2023
Simply explained: How does GPT work?
1 project | news.ycombinator.com | 6 Apr 2023
Will I ever need python again if I learn rust other than for AI stuff?
1 project | /r/Python | 4 Feb 2023
[D] HuggingFace in Julia or Rust ?
3 projects | /r/MachineLearning | 11 Jan 2023