C++ Inference

Open-source C++ projects categorized as Inference | Edit details

Top 5 C++ Inference Projects

  • GitHub repo mediapipe

    Cross-platform, customizable ML solutions for live and streaming media.

    Project mention: Why don't most 3d vtubers have mouth tracking like this? | reddit.com/r/vtubertech | 2021-12-03

    I believe something like that is possible with just a regular RGB webcam (excluding the tongue tracking) using Face Mesh from mediapipe

  • GitHub repo ncnn

    ncnn is a high-performance neural network inference framework optimized for the mobile platform

    Project mention: ncnn convert | dev.to | 2021-09-29

    ncnn install ncnn

  • Nanos

    Run Linux Software Faster and Safer than Linux with Unikernels.

  • GitHub repo jetson-inference

    Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

    Project mention: Jetson Nano 2GB Issues During Training (Out Of Memory / Process Killed) & Other Questions! | reddit.com/r/JetsonNano | 2021-11-05

    I’m trying to do the tutorial, where they retrain the neural network to detect fruits (jetson-inference/pytorch-ssd.md at master · dusty-nv/jetson-inference · GitHub 1)

  • GitHub repo server

    The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)

    Project mention: Nvidia Fiscal Q3 2022 Financial Result | reddit.com/r/nvidia | 2021-11-17

    Tools for developing and deploying large language models: NVIDIA NeMo Megatron, for training models with trillions of parameters; the Megatron 530B customizable LLM that can be trained for new domains and languages; and NVIDIA Triton Inference Server™ with multi-GPU, multinode distributed inference functionality.

  • GitHub repo cppflow

    Run TensorFlow models in C++ without installation and without Bazel

    Project mention: [Python] Importing a TensorFlow AI? | reddit.com/r/godot | 2021-10-12

    I toyed around with this idea a while back but I never got around to finishing the implementation. If all you need is inference with no training and you are relatively familiar with c++ you could look into creating a module for Godot that interfaces with the Tensorflow C API. Something like cppflow would provide an even easier API to work with. Looking into that project could also explain how they interface with the Tensorflow C API if you'd rather cut out the middle man. A module like this would let you train your model in Python and then load it and perform inference in Godot natively.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-12-03.

C++ Inference related posts


What are some of the best open-source Inference projects in C++? This list will help you:

Project Stars
1 mediapipe 14,817
2 ncnn 13,219
3 jetson-inference 5,132
4 server 2,899
5 cppflow 518
Find remote jobs at our new job board 99remotejobs.com. There are 33 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives