truss
inference-benchmark
| truss | inference-benchmark | |
|---|---|---|
| 3 | 1 | |
| 837 | 26 | |
| 2.3% | - | |
| 9.6 | 6.4 | |
| 5 days ago | 11 months ago | |
| Python | Python | |
| MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
truss
- Show HN: Truss – serve any ML model, anywhere, without boilerplate code
-
[P] Truss, a new open-source library for model packaging and deployment
At work, I just helped launch Truss, our company’s first open-source project, and I wanted to tell you a bit about it in case it can help you serve and deploy your models.
inference-benchmark
-
[D] Handling Concurrent Request for ML Model API
I have done some benchmarks before: https://github.com/tensorchord/inference-benchmark
What are some alternatives?
inference - Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
agentchain - Chain together LLMs for reasoning & orchestrate multiple large models for accomplishing complex tasks
BentoML - The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!
ChatFred - Alfred workflow using ChatGPT, DALL·E 2 and other models for chatting, image generation and more.
Ray - Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
mosec - A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine
data-science-ipython-notebooks - Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
text-generation-inference - Large Language Model Text Generation Inference
scikit-learn - scikit-learn: machine learning in Python
pipeless - An open-source computer vision framework to build and deploy apps in minutes without worrying about multimedia pipelines [Moved to: https://github.com/pipeless-ai/pipeless]
Keras - Deep Learning for humans