Could I get a suggestion for a simple HTTP API with no GUI for llama.cpp?

• llama.cpp-dotnet

  1 48 9.5 C#

  Minimal C# bindings for llama.cpp + .NET core library with API host/client.

• llama-cpp-python

  54 6,378 9.9 Python

  Python bindings for llama.cpp

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• go-llama.cpp

  4 554 8.4 C++

  LLama.cpp golang bindings

  

Go: go-skynet/go-llama.cpp

• llama-node

  2 847 8.6 Rust

  Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on your laptop CPU. support llama/alpaca/gpt4all/vicuna/rwkv model.

  

Node.js: hlhr202/llama-node

• llama_cpp.rb

  2 125 9.6 C++

  llama_cpp provides Ruby bindings for llama.cpp

Ruby: yoshoku/llama_cpp.rb

• LLamaSharp

  3 1,839 9.8 C#

  A cross-platform library to run 🦙LLaMA/LLaVA model (and others) on your local device efficiently.

  

C#/.NET: SciSharp/LLamaSharp

• FastChat

  82 33,668 9.7 Python

  An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

  

I used the FastChat API to load two quantized Vicuna-13 models locally so I could repeatedly query them for the modern translation of a given paragraph from the complete works of Jonathan Swift. Then I LoRa+PEFTed Llama-7b to convert from modern English to Swift. Works great: https://huggingface.co/pcalhoun/LLaMA-7b-JonathanSwift

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• LocalAI

  82 19,033 9.9 C++

  :robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.

  

Suggest a related project