SaaSHub helps you find the best software and product alternatives Learn more →
Top 5 C++ inference-engine Projects
-
k=3 ssh_k # building this on the third machine git clone https://github.com/PygmalionAI/aphrodite-engine.git ~/shared/aphrodite-engine cd ~/shared/aphrodite-engine pip install protobuf==3.20.2 ninja msgspec coloredlogs portalocker pytimeparse -r requirements-common.txt python setup.py bdist_wheel pip install --no-deps dist/*.whl
-
Nutrient
Nutrient - The #1 PDF SDK Library. Bad PDFs = bad UX. Slow load times, broken annotations, clunky UX frustrates users. Nutrient’s PDF SDKs gives seamless document experiences, fast rendering, annotations, real-time collaboration, 100+ features. Used by 10K+ devs, serving ~half a billion users worldwide. Explore the SDK for free.
-
Oops, you're right and it's a difference between my blog post and source code. It should be __shfl_down_sync as seen [here](https://github.com/andrewkchan/yalm/blob/8c908f23f5d8cc3f14c...)
-
Daisykit
Daisykit is an easy AI toolkit with face mask detection, pose detection, background matting, barcode detection, and more. With Daisykit, you don't need AI knowledge to build AI software.
-
-
nnl
a low-latency and high-performance inference engine for large models on low-memory GPU platform.
-
CodeRabbit
CodeRabbit: AI Code Reviews for Developers. Revolutionize your code reviews with AI. CodeRabbit offers PR summaries, code walkthroughs, 1-click suggestions, and AST-based analysis. Boost productivity and code quality across all major languages with each PR.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
C++ inference-engine discussion
C++ inference-engine related posts
-
Fast LLM Inference From Scratch (using CUDA)
-
QUIK is a method for quantizing LLM post-training weights to 4 bit precision
-
Intel OpenVINO 2023.1.0 released
-
Intel OpenVINO 2023.1.0 released, open-source toolkit for optimizing and deploying AI inference
-
OpenVINO 2023.1.0 released
-
[N] Intel OpenVINO 2023.1.0 released, open-source toolkit for optimizing and deploying AI inference
-
Intel OpenVINO 2023.1.0 released, open-source toolkit for optimizing and deploying AI inference
-
A note from our sponsor - SaaSHub
www.saashub.com | 16 Feb 2025
Index
What are some of the best open-source inference-engine projects in C++? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | aphrodite-engine | 1,287 |
| 2 | yalm | 233 |
| 3 | Daisykit | 104 |
| 4 | EasyOCR-cpp | 51 |
| 5 | nnl | 5 |
Sponsored
Nutrient - The #1 PDF SDK Library
Bad PDFs = bad UX. Slow load times, broken annotations, clunky UX frustrates users. Nutrient’s PDF SDKs gives seamless document experiences, fast rendering, annotations, real-time collaboration, 100+ features. Used by 10K+ devs, serving ~half a billion users worldwide. Explore the SDK for free.
nutrient.io