C++ serving

Open-source C++ projects categorized as serving
Edit details
Topics: Deep Learning Machine Learning ML Python CPP
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

C++ serving Projects

  • serving

    A flexible, high-performance serving system for machine learning models

    • Project mention: Llama.cpp: Full CUDA GPU Acceleration | news.ycombinator.com | 2023-06-12

    Yet another TEDIOUS BATTLE: Python vs. C++/C stack.

    This project gained popularity due to the HIGH DEMAND for running large models with 1B+ parameters, like `llama`. Python dominates the interface and training ecosystem, but prior to llama.cpp, non-ML professionals showed little interest in a fast C++ interface library. While existing solutions like tensorflow-serving [1] in C++ were sufficiently fast with GPU support, llama.cpp took the initiative to optimize for CPU and trim unnecessary code, essentially code-golfing and sacrificing some algorithm correctness for improved performance, which isn't favored by "ML research".

    NOTE: In my opinion, a true pioneer was DarkNet, which implemented the YOLO model series and significantly outperformed others [2]. Same trick basically like llama.cpp

    [1] https://github.com/tensorflow/serving

  • FastDeploy

    ⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end optimization, multi-platform and multi-framework support.

    • Project mention: Testing YOLO on Orange Pi 5 | /r/OrangePI | 2023-07-09

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

C++ serving related posts

  • Quickly develop risk control algorithms in business scenarios based on MetaSpore

    1 project | /r/learnmachinelearning | 15 Jun 2022

  • Quickly develop risk control algorithms in business scenarios based on MetaSpore

    1 project | dev.to | 15 Jun 2022

  • Usage Guide：Quickly deploy an intelligent data platform with the One-stop AI development and production platform, AlphaIDE

    1 project | dev.to | 14 Jun 2022

  • [P]MMML | Deploy HuggingFace training model rapidly based on MetaSpore

    1 project | /r/MachineLearning | 1 Jun 2022

  • MMML | Deploy HuggingFace training model rapidly based on MetaSpore

    1 project | /r/learnmachinelearning | 1 Jun 2022

  • The design concept of an almighty Opensource project about machine learning platform

    1 project | dev.to | 30 Apr 2022

  • Almighty Opensource project about machine learning you should try out

    1 project | dev.to | 12 Apr 2022
  • A note from our sponsor - InfluxDB
    www.influxdata.com | 10 May 2024
    Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

Project Stars
1 serving 6,085
2 FastDeploy 2,724

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com