Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression. Learn more →
Similar projects and alternatives to llama.cpp
A gradio web UI for running Large Language Models like GPT-J 6B, OPT, GALACTICA, LLaMA, and Pygmalion.
Inference code for LLaMA models
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
Locally run an Instruction-Tuned Chat-Style LLM
4 bits quantization of LLaMa using GPTQ
Instruct-tune LLaMA on consumer hardware
Tensor library for machine learning
Code and documentation to train Stanford's Alpaca models, and generate the data.
Clean code begins in your IDE with SonarLint. Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.
Run LLaMA inference on CPU, with Rust 🦀🚀🦙
OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
ChatGLM-6B：开源双语对话语言模型 | An Open Bilingual Dialogue Language Model
High-speed download of LLaMA, Facebook's 65B parameter GPT model
Rust+OpenCL+AVX2 implementation of LLaMA inference code
🐉 Making Rust a first-class language and ecosystem for GPU shaders 🚧
The simplest way to run LLaMA on your local machine
Port of OpenAI's Whisper model in C/C++
You like pytorch? You like micrograd? You love tinygrad! ❤️
Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.
Command Line Interface for OpenAi ChatGPT (by yudax42)
Yet Another LLaMA Discord Bot
Inference code for LLaMA models (by gmorenz)
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
llama.cpp reviews and mentions
Introducing llamacpp-for-kobold, run llama.cpp locally with a fancy web UI, persistent stories, editing tools, save formats, memory, world info, author's note, characters, scenarios and more with minimal setup.
5 projects | reddit.com/r/KoboldAI | 23 Mar 2023
I cannot fix it myself - please raise awareness to it here: https://github.com/ggerganov/llama.cpp/discussions/2295 projects | reddit.com/r/KoboldAI | 23 Mar 2023
There's an important fix for 65B models upstream: https://github.com/ggerganov/llama.cpp/pull/438/files. I've verified it works on my local copy. Can your fork be updated from upstream? Without it llama will segfault to an under-estimate of the memory required.
Cformers 🚀 - "Transformers with a C-backend for lightning-fast CPU inference". | Nolano
2 projects | reddit.com/r/LocalLLaMA | 22 Mar 2023
How does the inference speed compare to GGML based approaches like llama.cpp? Repo link: https://github.com/ggerganov/llama.cpp
GitHub announces a bunch of new GPT-4 powered coding assistants. What should and could Emacs and open-source community do?
5 projects | reddit.com/r/emacs | 22 Mar 2023
There are some "open-source" alternatives for the open-AI generative LLMs there, like LLAMA https://github.com/ggerganov/llama.cpp trained and leaked by Meta, and made available in c++ and actually runnable even on a MacBook Air. Sure it's not as great as Codex, GPT-4 and such, but it's a start (there are also open datasets with lots of code https://huggingface.co/datasets/bigcode/the-stack, so maybe in the future the gap will close).
[P] fastLLaMa, A python wrapper to run llama.cpp
4 projects | reddit.com/r/MachineLearning | 21 Mar 2023
llama.cpp as of very recently support `--perplexity`, which calculates the perplexity over the prompt (and you can just pass wikitext2 test set as the prompt for example). See https://github.com/ggerganov/llama.cpp/pull/270
Bill Gates just published a 7-page letter about AI and his predictions for its future
3 projects | reddit.com/r/Futurology | 21 Mar 2023
https://github.com/ggerganov/llama.cpp [CPU loading with comparatively low memory requirements (7b running on phones and Raspberry Pi 4) - no fancy front end yet]
[Project] Alpaca-30B: Facebook's 30b parameter LLaMa fine-tuned on the Alpaca dataset
If my understanding is correct, Alpaca.cpp or https://github.com/ggerganov/llama.cpp are a sort of 'front-end' for these model.
LLaMa/Alpaca work just fine on CPU with llama.cpp/alpaca.cpp. Not very snappy but fast enough for me.
You can run llama-30B on a CPU using llama.cpp, it's just slow. The alpaca models I've seen are the same size as the llama model they are trained on, so I would expect running the alpaca-30B models will be possible on any system capable of running llama-30B.
LLaMa-cpp: RISC-V (TH1520&D1) benchmark and hack for <1GB DDR device
2 projects | reddit.com/r/RISCV | 20 Mar 2023
A note from our sponsor - InfluxDB
www.influxdata.com | 25 Mar 2023
ggerganov/llama.cpp is an open source project licensed under MIT License which is an OSI approved license.