rknn-multi-threaded VS llama.cpp

Compare rknn-multi-threaded vs llama.cpp and see what are their differences.


A simple demo of yolov5s running on rk3588/3588s using Python (about 72 frames). / 一个使用Python在rk3588/3588s上运行的yolov5s简单demo(大约72帧/s)。 (by leafqycc)


Port of Facebook's LLaMA model in C/C++ (by ggerganov)
Our great sponsors
  • LearnThisRepo.com - Learn 300+ open source libraries for free using AI.
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern API for authentication & user identity.
rknn-multi-threaded llama.cpp
1 734
131 51,425
- -
4.5 9.9
10 months ago 2 days ago
Python C++
Apache License 2.0 MIT License
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.


Posts with mentions or reviews of rknn-multi-threaded. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-06-01.
  • OPI 5B deep learning
    5 projects | /r/OrangePI | 1 Jun 2023
    This is an example: https://github.com/leafqycc/rknn-multi-threaded If you want to run the example you need to install rknn-toolkit2-lite:


Posts with mentions or reviews of llama.cpp. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2024-02-21.

What are some alternatives?

When comparing rknn-multi-threaded and llama.cpp you can also consider the following projects:

ollama - Get up and running with Llama 2, Mistral, and other large language models.

gpt4all - gpt4all: run open-source LLMs anywhere

text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

GPTQ-for-LLaMa - 4 bits quantization of LLaMA using GPTQ

ggml - Tensor library for machine learning

alpaca.cpp - Locally run an Instruction-Tuned Chat-Style LLM

FastChat - An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

rust-gpu - 🐉 Making Rust a first-class language and ecosystem for GPU shaders 🚧

ChatGLM-6B - ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

AutoGPT - AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

safetensors - Simple, safe way to store and distribute tensors

alpaca-lora - Instruct-tune LLaMA on consumer hardware