Llama2.rs: One-file Rust implementation of Llama2

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  1. llama2.rs

    A fast llama2 decoder in pure Rust.

  2. InfluxDB

    InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.

    InfluxDB logo
  3. tch-rs

    Rust bindings for the C++ api of PyTorch.

    I wanted to do something like this but then I would miss on proper CUDA acceleration and lose performance compared to using torchlib.

    I wrote a forgettable llama implementation for https://github.com/LaurentMazare/tch-rs (pytorch's torchlib rust binding).

  4. candle

    Minimalist ML framework for Rust

    Still not ideal but at least you get the same GPU performance you would get on pytorch.

    ...And then I spotted Candle, a new ML framework by the same author: https://github.com/huggingface/candle

    It's all in Rust, self contained, a huge undertaking, but it looks very promising.

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Tch-Rs

    1 project | news.ycombinator.com | 7 Dec 2023
  • llm: a Rust crate/CLI for CPU inference of LLMs, including LLaMA, GPT-NeoX, GPT-J and more

    7 projects | /r/rust | 9 May 2023
  • Simply explained: How does GPT work?

    1 project | news.ycombinator.com | 6 Apr 2023
  • Will I ever need python again if I learn rust other than for AI stuff?

    1 project | /r/Python | 4 Feb 2023
  • [D] HuggingFace in Julia or Rust ?

    3 projects | /r/MachineLearning | 11 Jan 2023