Llama2.rs: One-file Rust implementation of Llama2

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com
#Pytorch #Rust #Machine Learning #neural-network #Deep Learning

Our great sponsors
  • InfluxDB - Collect and Analyze Billions of Data Points in Real Time
  • Onboard AI - Learn any GitHub repo in 59 seconds
  • SaaSHub - Software Alternatives and Reviews
Our great sponsors

  • llama2.rs

    A fast llama2 decoder in pure Rust.

  • tch-rs

    Rust bindings for the C++ api of PyTorch.

    I wanted to do something like this but then I would miss on proper CUDA acceleration and lose performance compared to using torchlib.

    I wrote a forgettable llama implementation for https://github.com/LaurentMazare/tch-rs (pytorch's torchlib rust binding).

  • InfluxDB

    Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  • candle

    Minimalist ML framework for Rust

    Still not ideal but at least you get the same GPU performance you would get on pytorch.

    ...And then I spotted Candle, a new ML framework by the same author: https://github.com/huggingface/candle

    It's all in Rust, self contained, a huge undertaking, but it looks very promising.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts