Top 7 Python inference-engine Projects

FedML

6 4,052 9.9 Python

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, FEDML Nexus AI (https://fedml.ai) is your generative AI platform at scale.

Project mention: [Experiment] The future of AI is open-source, and here is the plan | /r/samkoesnadi | 2023-06-05

FedML https://github.com/FedML-AI/FedML might already provide a lot of tools to do the job

onediff

2 1,094 9.7 Python

OneDiff: An out-of-the-box acceleration library for diffusion models.

Project mention: Accelerating Stable Video Diffusion 3x Faster with OneDiff DeepCache and Int8 | news.ycombinator.com | 2024-01-29

--output-video path/to/output_image.mp4
Run with ComfyUI
Run with OneDiff workflow: https://github.com/siliconflow/onediff/blob/main/onediff_com...
Run with OneDiff + DeepCache workflow: https://github.com/siliconflow/onediff/blob/main/onediff_com...
The use of Int8 can be referenced in the workflow: https://github.com/siliconflow/onediff/blob/main/onediff_com...

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
astroid

2 510 9.2 Python

A common base representation of python source code for pylint and other projects (by pylint-dev)
aphrodite-engine

2 487 9.7 Python

PygmalionAI's large-scale inference engine

Project mention: Using vLLM for Home Assistant. I need help getting any model to work. Can't get anything over 7B to run on a 3090. | /r/LocalLLaMA | 2023-11-22

Try out https://github.com/PygmalionAI/aphrodite-engine

Savant

54 476 9.4 Python

Python Computer Vision & Video Analytics Framework With Batteries Included

Project mention: 🚀 Savant 0.2.6 is Out: Release Notes | /r/savant_video | 2023-11-09

Website: https://savant-ai.io/

ai-hub-models

1 195 7.1 Python

The Qualcomm® AI Hub Models are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

Project mention: Qualcomm has open sourced more than 80 AI models | news.ycombinator.com | 2024-02-28

Hi HN! I'm a member of the team that worked hard on AI Hub and AI Hub Models for the last few years. Excited to see our work show up here!!
I also encourage you take a look at our GitHub repository: https://github.com/quic/ai-hub-models
If you have questions or feature requests, you can reach out to us on Slack (https://join.slack.com/t/qualcomm-ai-hub/shared_invite/zt-2d...) or file an issue on GitHub / Huggingface. We are pretty responsive!

experta

0 136 0.0 Python

Expert Systems for Python
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2024-02-28.