Python inference-api

Open-source Python projects categorized as inference-api
Edit details
Topics: Machine Learning inference-server Onnx Inference model-serving
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

Top 5 Python inference-api Projects

  • BentoML

    The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!

    • Project mention: Who's hiring developer advocates? (December 2023) | dev.to | 2023-12-04

    Link to GitHub -->

  • inference

    A fast, easy-to-use, production-ready inference server for computer vision supporting deployment of many popular model architectures and fine-tuned models. (by roboflow)

    • Project mention: Supervision: Reusable Computer Vision | news.ycombinator.com | 2024-03-24

    Yeah, inference[1] is our open source package for running locally (either directly in Python or via a Docker container). It works with all the models on Universe, models you train yourself (assuming we support the architecture; we have a bunch of notebooks available[2]), or train in our platform, plus several more general foundation models[3] (for things like embeddings, zero-shot detection, question answering, OCR, etc).

    We also have a hosted API[4] you can hit for most models we support (except some of the large vision models that are really GPU-heavy) if you prefer.

    [1] https://github.com/roboflow/inference

    [2] https://github.com/roboflow/notebooks

    [3] https://inference.roboflow.com/foundation/about/

    [4] https://docs.roboflow.com/deploy/hosted-api

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • truss

    The simplest way to serve AI/ML models in production (by basetenlabs)

  • ai-hub-models

    The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

    • Project mention: Qualcomm has open sourced more than 80 AI models | news.ycombinator.com | 2024-02-28

    Hi HN! I'm a member of the team that worked hard on AI Hub and AI Hub Models for the last few years. Excited to see our work show up here!!

    I also encourage you take a look at our GitHub repository: https://github.com/quic/ai-hub-models

    If you have questions or feature requests, you can reach out to us on Slack (https://join.slack.com/t/qualcomm-ai-hub/shared_invite/zt-2d...) or file an issue on GitHub / Huggingface. We are pretty responsive!

  • textsum

    CLI & Python API to easily summarize text-based files with transformers

    • Project mention: Training on documents to summarize them. | /r/LocalLLaMA | 2023-06-16

    See https://github.com/pszemraj/textsum. He's the guy that trained most of the popular long finetuned long models. He created a pip package to make life easier (which uses Huggingface under the hood, just pre-selects good models and obfuscates boilerplate).

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Index

What are some of the best open-source inference-api projects in Python? This list will help you:

Project Stars
1 BentoML 6,537
2 inference 1,031
3 truss 833
4 ai-hub-models 212
5 textsum 110

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com