Rust llm

Open-source Rust projects categorized as llm
Edit details
Topics: Rust AI openai chatgpt Gpt
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
Sponsored

Top 23 Rust llm Projects

  • burn

    Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.

    • Project mention: 3 years of fulltime Rust game development, and why we're leaving Rust behind | news.ycombinator.com | 2024-04-26

    You can use libtorch directly via `tch-rs`, and at present I'm porting over to Burn (see https://burn.dev) which appears incredibly promising. My impression is it's in a good place, if of course not close to the ecosystem of Python/C++. At very least I've gotten my nn models training and running without too much difficulty. (I'm moving to Burn for the thread safety - their `Tensor` impl is `Sync` - libtorch doesn't have such a guarantee.)

    Burn has Candle as one of its backends, which I understand is also quite popular.

  • llm

    An ecosystem of Rust libraries for working with large language models

    • Project mention: Open-sourcing a simple automation/agent workflow builder | /r/ChatGPTPro | 2023-10-07

    We're open-sourcing a project that lets you build simple automations/agent workflows that use LLMs for different tasks. Kinda like Zapier or IFTTT but focused on using natural language to accomplish your tasks.It's super early but we'd love to start getting feedback to steer it in the right direction. It currently supports OpenAI and local models through llm.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • aichat

    All-in-one AI-Powered CLI Chat & Copilot that integrates 10+ AI platforms, including OpenAI, Azure-OpenAI, Gemini, VertexAI, Claude, Mistral, Cohere, Ollama, Ernie, Qianwen...

    • Project mention: Show HN: A shell CLI tool to predict your next command enhanced by LLM and RAG | news.ycombinator.com | 2024-04-16

    thanks for sharing this, I have been using aichat (https://github.com/sigoden/aichat) and shell_gpt for a while. Let's see how it works.

  • text-embeddings-inference

    A blazing fast inference solution for text embeddings models

    • Project mention: HuggingFace text-generation-inference is reverting to Apache 2.0 License | news.ycombinator.com | 2024-04-08

    Worth noting that this also impacts the great https://github.com/huggingface/text-embeddings-inference, which allows anyone to run state of the art embeddings with great performance.

  • smartgpt

    A program that provides LLMs with the ability to complete complex tasks using plugins.

    • Project mention: Smartgpt: A program that provides LLMs ability with complex tasks using plugins | news.ycombinator.com | 2023-08-28

  • aici

    AICI: Prompts as (Wasm) Programs

    • Project mention: HonoJS: Small, simple, and ultrafast web framework for the Edges | news.ycombinator.com | 2024-04-15

    Have you looked at AICI by Microsoft yet?

    https://github.com/microsoft/aici/

  • pgvecto.rs

    Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database.

    • Project mention: My binary vector search is better than your FP32 vectors | dev.to | 2024-03-25

    To evaluate the performance metrics in comparison to the original vector approach, we conducted benchmarking using the dbpedia-entities-openai3-text-embedding-3-large-3072-1M dataset. The benchmark was performed on a Google Cloud virtual machine (VM) with specifications of n2-standard-8, which includes 8 virtual CPUs and 32GB of memory. We used pgvecto.rs v0.2.1 as the vector database.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo

  • llm-chain

    `llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks

    • Project mention: are there anytools or frameworks similar to "langchain" or "llamaindexbut implemented or designed in a language other than python? | /r/LocalLLaMA | 2023-06-28

  • floneum

    A toolkit for controllable, private AI on consumer hardware in rust

    • Project mention: Dioxus 0.5: Web, Desktop, Mobile Apps in Rust | news.ycombinator.com | 2024-03-28

    It is pretty good. I am working on an application that uses SVGs as a way to draw a workflow editor UI with Dioxus: https://github.com/floneum/floneum

  • llama-node

    Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on your laptop CPU. support llama/alpaca/gpt4all/vicuna/rwkv model.

    • Project mention: Tell HN: Rust Is the Superglue | news.ycombinator.com | 2023-07-15

    You can practice your Rust skills by writing performant and/or gluey extensions for higher-level language such as NodeJS (checkout napi-rs) and Python or complementing JS in the browser if you target Webassembly.

    For instance, checkout Llama-node https://github.com/Atome-FE/llama-node for an involved Rust-based NodeJS extension. Python has PyO3, a Rust-Python extension toolset: https://github.com/PyO3/pyo3.

    They can help you leverage your Rust for writing cool new stuff.

  • plock

    From anywhere you can type, query and stream the output of an LLM or any other script

    • Project mention: Show HN: NotesOllama – I added local LLM support to Apple Notes (through Ollama) | news.ycombinator.com | 2024-02-21

    I'm a big fan of this space and have been hacking on it too. With a few tricks you can use LLMs or anything else you can call from a script, via input capture and simulation, and clipboard. And it can be cross platform!

    Here's the project, if it might interest anyone.

    https://github.com/jasonjmcghee/plock

  • ai00_server

    A localized open-source AI server that is better than ChatGPT.

    • Project mention: People who've used RWKV, whats your wishlist for it? | /r/LocalLLaMA | 2023-12-09

  • oatmeal

    Terminal UI to chat with large language models (LLM) using different model backends, and integrations with your favourite editors!

    • Project mention: FLaNK Stack 26 February 2024 | dev.to | 2024-02-26

  • fltr

    Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

    • Project mention: Show HN: Like grep but for natural questions. Mixtral 8x7B – 28 tok/s on 8GB GPU | news.ycombinator.com | 2024-03-13

  • indexify

    A scalable realtime and continuous indexing and structured extraction engine for Unstructured Data to build Generative AI Applications

    • Project mention: I accidentally built a meme search engine | news.ycombinator.com | 2024-04-13

    These hacks/side projects are amazing! I feel we will see a lot of creativity as tools to build data intensive AI applications become easier.

    We built and open sourced Indexify https://github.com/tensorlakeai/indexify to make it easy to build resilient pipelines to combine data with many different models and transformations to build applications that relies on embedding or any other metadata extracted by models from Videos, Photos and any documents!

    I didn’t know about SigClip, the author mentioned on the blog, need to add this to our library :) I also found it incredible that he generated the crawler with Claude! This is the type of boilerplate I hope we don’t have to write in the future

  • tenere

    🔥 TUI interface for LLMs written in Rust

    • Project mention: Tenere supports llamacpp and ollama as back ends now | news.ycombinator.com | 2024-02-09

  • mistral.rs

    Blazingly fast LLM inference.

    • Project mention: Mistral.rs: Fast LLM Engine | news.ycombinator.com | 2024-04-25

  • smolrsrwkv

    A relatively basic implementation of RWKV in Rust written by someone with very little math and ML knowledge. Supports 32, 8 and 4 bit evaluation. It can also directly load PyTorch RWKV models.

  • opentau

    Using Large Language Models for Gradual Type Inference

    • Project mention: Opentau – Using Large Language Models for Gradual Type Inference | news.ycombinator.com | 2023-08-13

  • signway

    Bring the power of pre-signed URLs to your apps. Signway is a gateway for redirecting ephimeral signed URLs to the requested API

    • Project mention: Show HN: Signed URLs Gateway – The Missing Infra Piece for Building Modern Apps | news.ycombinator.com | 2023-11-20

  • Orphic

    A natural language shell interface for *nix systems

  • memex

    Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc. (by spyglass-search)

    • Project mention: Playing around w/ a more structured LLM workflow | /r/LocalLLaMA | 2023-08-21

    It's currently working with OpenAI (which you can play with here: https://cloud.spyglass.fyi/playground). We're still ironing out some bugs before we can release an open-source version that works the llama models (via our other utility: https://github.com/spyglass-search/memex).

  • cai

    The fastest CLI tool for prompting LLMs. Including support for prompting several LLMs at once!

    • Project mention: Show HN: Cai – The fastest CLI tool for prompting LLMs | news.ycombinator.com | 2024-03-28

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Rust llm related posts

Index

What are some of the best open-source llm projects in Rust? This list will help you:

Project Stars
1 burn 7,020
2 llm 5,885
3 aichat 2,804
4 text-embeddings-inference 2,004
5 smartgpt 1,726
6 aici 1,723
7 pgvecto.rs 1,375
8 llm-chain 1,156
9 floneum 959
10 llama-node 847
11 plock 421
12 ai00_server 377
13 oatmeal 364
14 fltr 337
15 indexify 238
16 tenere 226
17 mistral.rs 105
18 smolrsrwkv 91
19 opentau 90
20 signway 78
21 Orphic 52
22 memex 44
23 cai 28

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com