truss
inference
| truss | inference | |
|---|---|---|
| 3 | 2 | |
| 837 | 2,629 | |
| 2.3% | 19.7% | |
| 9.6 | 9.8 | |
| 5 days ago | 7 days ago | |
| Python | Python | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
truss
- Show HN: Truss – serve any ML model, anywhere, without boilerplate code
-
[P] Truss, a new open-source library for model packaging and deployment
At work, I just helped launch Truss, our company’s first open-source project, and I wanted to tell you a bit about it in case it can help you serve and deploy your models.
inference
-
GreptimeAI + Xinference - Efficient Deployment and Monitoring of Your LLM Applications
Xorbits Inference (Xinference) is an open-source platform to streamline the operation and integration of a wide array of AI models. With Xinference, you’re empowered to run inference using any open-source LLMs, embedding models, and multimodal models either in the cloud or on your own premises, and create robust AI-driven applications. It provides a RESTful API compatible with OpenAI API, Python SDK, CLI, and WebUI. Furthermore, it integrates third-party developer tools like LangChain, LlamaIndex, and Dify, facilitating model integration and development.
-
🤖 AI Podcast - Voice Conversations🎙 with Local LLMs on M2 Max
Code: https://github.com/xorbitsai/inference/blob/main/examples/AI_podcast.py
What are some alternatives?
BentoML - The most flexible way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Inference Graph/Pipelines, Compound AI systems, Multi-Modal, RAG as a Service, and more!
agentchain - Chain together LLMs for reasoning & orchestrate multiple large models for accomplishing complex tasks
Ray - Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
ChatGLM2-6B - ChatGLM2-6B: An Open Bilingual Chat LLM | 开源双语对话语言模型
data-science-ipython-notebooks - Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
h2o-wizardlm - Open-Source Implementation of WizardLM to turn documents into Q:A pairs for LLM fine-tuning
inference-benchmark - Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
mpt-30B-inference - Run inference on MPT-30B using CPU
scikit-learn - scikit-learn: machine learning in Python
aihandler - A simple engine to help run diffusers and transformers models
pipeless - An open-source computer vision framework to build and deploy apps in minutes without worrying about multimedia pipelines [Moved to: https://github.com/pipeless-ai/pipeless]
rwkv.cpp - INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model