C++ llamacpp

Open-source C++ projects categorized as llamacpp
Edit details
Topics: llama llm llms AI ggml
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

Top 7 C++ llamacpp Projects

  • koboldcpp

    A simple one-file way to run various GGML and GGUF models with KoboldAI's UI

    • Project mention: Any Online Communities on Local/Home AI? | news.ycombinator.com | 2024-04-24

  • cortex

    Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM). Powers 👋 Jan (by janhq)

    • Project mention: Ollama Python and JavaScript Libraries | news.ycombinator.com | 2024-01-24

    I'd like to see a comparison to nitro https://github.com/janhq/nitro which has been fantastic for running a local LLM.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • vit.cpp

    Inference Vision Transformer (ViT) in plain C/C++ with ggml

    • Project mention: I've open sourced my Flutter plugin to run on-device LLMs on any platform. TestFlight builds available now. | /r/FlutterDev | 2023-12-08

    And more stuff I’m often checking back on: - https://github.com/staghado/vit.cpp - https://github.com/serp-ai/bark-with-voice-clone - https://github.com/leejet/stable-diffusion.cpp (generate images) - etc … there’s too much fun stuff out there. Wish I had more free time haha.

  • collider

    Large Model Collider - The Platform for serving LLM models

    • Project mention: Show HN: Collider – the platform for local LLM debug and inference at warp speed | news.ycombinator.com | 2023-11-30

  • pyllamacpp

    Python bindings for llama.cpp

    • Project mention: How to - Local LLMs ? | /r/artificial | 2023-05-10

    For me, the easiest way was to start was to use https://github.com/abdeladim-s/pyllamacpp. If you have Python installed, do `pip install numpy pyllamacpp`, download compatible model file (links are in README.md) and simply run with `pyllamacpp model.file`.

  • godot-llm-experiment

    Getting an LLM to work with Godot.

    • Project mention: Mistral 7B Paper on ArXiv | news.ycombinator.com | 2023-10-11

    I put Mistral-7B-Instruct in a Godot poc game the other day, and the simulated conversations it generates is funny as heck:

    https://github.com/opyate/godot-llm-experiment

  • llama-server-chat-terminal-client

    Lightweight chat terminal-interface for llama.cpp server compilable for windows and linux.

    • Project mention: Terminal client chat for llama.cpp server. | /r/LocalLLaMA | 2023-12-05

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

C++ llamacpp related posts

  • Any Online Communities on Local/Home AI?

    1 project | news.ycombinator.com | 24 Apr 2024

  • PyLLaMACpp: A simple way to run llama.cpp + Python bindings

    1 project | /r/Python | 21 Mar 2023

  • PyLLaMACpp: A simple way to run llama.cpp + Python bindings

    1 project | /r/LocalLLaMA | 21 Mar 2023

Index

What are some of the best open-source llamacpp projects in C++? This list will help you:

Project Stars
1 koboldcpp 3,817
2 cortex 1,600
3 vit.cpp 174
4 collider 117
5 pyllamacpp 59
6 godot-llm-experiment 31
7 llama-server-chat-terminal-client 10

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com