C++ Inference

Open-source C++ projects categorized as Inference | Edit details
Related topics: #Deep Learning #Machine Learning #Tensorflow #Computer Vision #Android

Top 5 C++ Inference Projects

  • GitHub repo mediapipe

    Cross-platform, customizable ML solutions for live and streaming media.

    Project mention: First try to replicate mediapipe codepen hand detection. | reddit.com/r/cpp | 2021-12-28

    I have used mediapipie library to make this. https://github.com/google/mediapipe

  • GitHub repo ncnn

    ncnn is a high-performance neural network inference framework optimized for the mobile platform

    Project mention: ncnn convert | dev.to | 2021-09-29

    ncnn install ncnn

  • OPS

    OPS - Build and Run Open Source Unikernels. Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.

  • GitHub repo jetson-inference

    Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

    Project mention: Jetson Nano 2GB Issues During Training (Out Of Memory / Process Killed) & Other Questions! | reddit.com/r/JetsonNano | 2021-11-05

    I’m trying to do the tutorial, where they retrain the neural network to detect fruits (jetson-inference/pytorch-ssd.md at master · dusty-nv/jetson-inference · GitHub 1)

  • GitHub repo server

    The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)

    Project mention: [D] patterns for scaling video inference service? | reddit.com/r/MachineLearning | 2021-12-22

    If you are working at an infrastructure level I would use ECS and utilize the NVIDIA Triton Inference Server. It can handle the multimodel paradigm through their ensemble method (bit of a misnomer since its really just a DAG of data flow through your models though you can add an ensembling method at the end of desired). Also provides a nice HTTP or GRPC interface. With ECS you can also use Application Load Balancer to further scale but how you set that up will also heavily depend on if you are using stateful models or not.

  • GitHub repo cppflow

    Run TensorFlow models in C++ without installation and without Bazel

    Project mention: [Python] Importing a TensorFlow AI? | reddit.com/r/godot | 2021-10-12

    I toyed around with this idea a while back but I never got around to finishing the implementation. If all you need is inference with no training and you are relatively familiar with c++ you could look into creating a module for Godot that interfaces with the Tensorflow C API. Something like cppflow would provide an even easier API to work with. Looking into that project could also explain how they interface with the Tensorflow C API if you'd rather cut out the middle man. A module like this would let you train your model in Python and then load it and perform inference in Godot natively.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-12-28.

C++ Inference related posts

Index

What are some of the best open-source Inference projects in C++? This list will help you:

Project Stars
1 mediapipe 15,775
2 ncnn 13,576
3 jetson-inference 5,268
4 server 3,123
5 cppflow 538
Find remote jobs at our new job board 99remotejobs.com. There are 29 new remote jobs listed recently.
Are you hiring? Post a new remote job listing for free.

Sponsored
Less time debugging, more time building
Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.
scoutapm.com