serve
polyaxon
Our great sponsors
serve | polyaxon | |
---|---|---|
11 | 9 | |
3,924 | 3,465 | |
2.0% | 0.8% | |
9.6 | 8.8 | |
5 days ago | 10 days ago | |
Java | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
serve
-
Show Show HN: Llama2 Embeddings FastAPI Server
What's wrong with just using Torchserve[1]? We've been using it to serve embedding models in production.
-
BetterTransformer: PyTorch-native free-lunch speedups for Transformer-based models
I did a Space to showcase a bit the speedups we can have in a end-to-end case with TorchServe to deploy the model on a cloud instance (AWS EC2 g4dn, using one T4 GPU): https://huggingface.co/spaces/fxmarty/bettertransformer-demo
-
[D] How to get the fastest PyTorch inference and what is the "best" model serving framework?
For 2), I am aware of a few options. Triton inference server is an obvious one as is the ‘transformer-deploy’ version from LDS. My only reservation here is that they require the model compilation or are architecture specific. I am aware of others like Bento, Ray serving and TorchServe. Ideally I would have something that allows any (PyTorch model) to be used without the extra compilation effort (or at least optionally) and has some convenience things like ease of use, easy to deploy, easy to host multiple models and can perform some dynamic batching. Anyway, I am really interested to hear people's experience here as I know there are now quite a few options! Any help is appreciated! Disclaimer - I have no affiliation or are connected in any way with the libraries or companies listed here. These are just the ones I know of. Thanks in advance.
- Choose JavaScript 🧠
-
Popular Machine Learning Deployment Tools
GitHub
polyaxon
-
Any MLOps platform you use?
If you're not concerned about self-hosting, WandB is one of the more fully featured training monitoring tools (I've used it in the past without any issues but the lack of data and training privacy and lack of self-hosting possibilities makes it a hard no for anything that isn't scholastic). Polyaxon is an alternative but rewriting all your variable logging to conform to their requirements makes it very difficult to switch to it in the middle of a project so you have to commit to it from the get-go.
-
[D] Kubernetes for ML - how are y'all doing it?
[4]: https://github.com/polyaxon/polyaxon
We use Polyaxon and it’s pretty good
-
[D] What MLOps platform do you use, and how helpful are they?
Disclosure - I'm the author of Polyaxon.
-
[D] Productionalizing machine learning pipelines for small teams
For running experiments, http://polyaxon.com/ is a really good free open-source package that has lots of nice integrations so you can quickly run experiments in k8s but it might be overkill in some cases.
-
Top 5 tools to get started with MLOps !
Polyaxon : https://polyaxon.com
What are some alternatives?
MLflow - Open source platform for the machine learning lifecycle
kubeflow - Machine Learning Toolkit for Kubernetes
flyte - Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
dvc - 🦉 ML Experiments and Data Management with Git
server - The Triton Inference Server provides an optimized cloud and edge inferencing solution.
neptune-client - :ledger: The MLOps stack component for experiment tracking
onepanel - The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.
mmlspark - Simple and Distributed Machine Learning [Moved to: https://github.com/microsoft/SynapseML]
deepchecks - Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.
mpi-operator - Kubernetes Operator for MPI-based applications (distributed training, HPC, etc.)
intelligent-trading-bot - Intelligent Trading Bot: Automatically generating signals and trading based on machine learning and feature engineering
pinferencia - Python + Inference - Model Deployment library in Python. Simplest model inference server ever.