Top 5 C++ Inference Projects
Cross-platform, customizable ML solutions for live and streaming media.Project mention: Why don't most 3d vtubers have mouth tracking like this? | reddit.com/r/vtubertech | 2021-12-03
I believe something like that is possible with just a regular RGB webcam (excluding the tongue tracking) using Face Mesh from mediapipe
ncnn is a high-performance neural network inference framework optimized for the mobile platformProject mention: ncnn convert | dev.to | 2021-09-29
ncnn install ncnn
Run Linux Software Faster and Safer than Linux with Unikernels.
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.Project mention: Jetson Nano 2GB Issues During Training (Out Of Memory / Process Killed) & Other Questions! | reddit.com/r/JetsonNano | 2021-11-05
I’m trying to do the tutorial, where they retrain the neural network to detect fruits (jetson-inference/pytorch-ssd.md at master · dusty-nv/jetson-inference · GitHub 1)
The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)Project mention: Nvidia Fiscal Q3 2022 Financial Result | reddit.com/r/nvidia | 2021-11-17
Tools for developing and deploying large language models: NVIDIA NeMo Megatron, for training models with trillions of parameters; the Megatron 530B customizable LLM that can be trained for new domains and languages; and NVIDIA Triton Inference Server™ with multi-GPU, multinode distributed inference functionality.
Run TensorFlow models in C++ without installation and without BazelProject mention: [Python] Importing a TensorFlow AI? | reddit.com/r/godot | 2021-10-12
I toyed around with this idea a while back but I never got around to finishing the implementation. If all you need is inference with no training and you are relatively familiar with c++ you could look into creating a module for Godot that interfaces with the Tensorflow C API. Something like cppflow would provide an even easier API to work with. Looking into that project could also explain how they interface with the Tensorflow C API if you'd rather cut out the middle man. A module like this would let you train your model in Python and then load it and perform inference in Godot natively.
C++ Inference related posts
[P] Python library to optimize Hugging Face transformer for inference: < 0.5 ms latency / 2850 infer/sec
4 projects | reddit.com/r/MachineLearning | 23 Nov 2021
Nvidia Fiscal Q3 2022 Financial Result
4 projects | reddit.com/r/nvidia | 17 Nov 2021
Jetson Nano 2GB Issues During Training (Out Of Memory / Process Killed) & Other Questions!
1 project | reddit.com/r/JetsonNano | 5 Nov 2021
[Python] Importing a TensorFlow AI?
1 project | reddit.com/r/godot | 12 Oct 2021
Simplest way to deploy Keras NN model into C++?
2 projects | reddit.com/r/learnmachinelearning | 29 Aug 2021
Triton: Open-Source GPU Programming for Neural Networks
3 projects | news.ycombinator.com | 28 Jul 2021
1 project | reddit.com/r/JetsonNano | 18 Jul 2021
What are some of the best open-source Inference projects in C++? This list will help you:
Are you hiring? Post a new remote job listing for free.