Top 5 C++ Inference Projects
-
Project mention: Why don't most 3d vtubers have mouth tracking like this? | reddit.com/r/vtubertech | 2021-12-03
I believe something like that is possible with just a regular RGB webcam (excluding the tongue tracking) using Face Mesh from mediapipe
-
ncnn
ncnn is a high-performance neural network inference framework optimized for the mobile platform
ncnn install ncnn
-
-
jetson-inference
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
Project mention: Jetson Nano 2GB Issues During Training (Out Of Memory / Process Killed) & Other Questions! | reddit.com/r/JetsonNano | 2021-11-05I’m trying to do the tutorial, where they retrain the neural network to detect fruits (jetson-inference/pytorch-ssd.md at master · dusty-nv/jetson-inference · GitHub 1)
-
server
The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)
Tools for developing and deploying large language models: NVIDIA NeMo Megatron, for training models with trillions of parameters; the Megatron 530B customizable LLM that can be trained for new domains and languages; and NVIDIA Triton Inference Server™ with multi-GPU, multinode distributed inference functionality.
-
I toyed around with this idea a while back but I never got around to finishing the implementation. If all you need is inference with no training and you are relatively familiar with c++ you could look into creating a module for Godot that interfaces with the Tensorflow C API. Something like cppflow would provide an even easier API to work with. Looking into that project could also explain how they interface with the Tensorflow C API if you'd rather cut out the middle man. A module like this would let you train your model in Python and then load it and perform inference in Godot natively.
C++ Inference related posts
- [P] Python library to optimize Hugging Face transformer for inference: < 0.5 ms latency / 2850 infer/sec
- Nvidia Fiscal Q3 2022 Financial Result
- Jetson Nano 2GB Issues During Training (Out Of Memory / Process Killed) & Other Questions!
- [Python] Importing a TensorFlow AI?
- Simplest way to deploy Keras NN model into C++?
- Triton: Open-Source GPU Programming for Neural Networks
- Jetson Nano
Index
What are some of the best open-source Inference projects in C++? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | mediapipe | 14,817 |
| 2 | ncnn | 13,219 |
| 3 | jetson-inference | 5,132 |
| 4 | server | 2,899 |
| 5 | cppflow | 518 |
Are you hiring? Post a new remote job listing for free.
