Top 7 C++ llamacpp Projects
-
cortex
Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM). Powers 👋 Jan (by janhq)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
llama-server-chat-terminal-client
Lightweight chat terminal-interface for llama.cpp server compilable for windows and linux.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Jan incorporates a lightweight, built-in inference server called Nitro. Nitro supports both llama.cpp and NVIDIA's TensorRT-LLM engines. This means many open LLMs in the GGUF format are supported. Jan's Model Hub is designed for easy installation of pre-configured models but it also allows you to install virtually any model from Hugging Face or even your own.
Project mention: I've open sourced my Flutter plugin to run on-device LLMs on any platform. TestFlight builds available now. | /r/FlutterDev | 2023-12-08And more stuff I’m often checking back on: - https://github.com/staghado/vit.cpp - https://github.com/serp-ai/bark-with-voice-clone - https://github.com/leejet/stable-diffusion.cpp (generate images) - etc … there’s too much fun stuff out there. Wish I had more free time haha.
Project mention: Show HN: Collider – the platform for local LLM debug and inference at warp speed | news.ycombinator.com | 2023-11-30
I put Mistral-7B-Instruct in a Godot poc game the other day, and the simulated conversations it generates is funny as heck:
https://github.com/opyate/godot-llm-experiment
C++ llamacpp related posts
Index
What are some of the best open-source llamacpp projects in C++? This list will help you:
Project | Stars | |
---|---|---|
1 | koboldcpp | 4,078 |
2 | cortex | 1,661 |
3 | vit.cpp | 184 |
4 | booster | 126 |
5 | pyllamacpp | 59 |
6 | godot-llm-experiment | 33 |
7 | llama-server-chat-terminal-client | 10 |
Sponsored