[D] Huggingface finally get a stable inference serving env for their 10k+ models

This page summarizes the projects mentioned and recommended in the original post on /r/MachineLearning

Our great sponsors
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • SaaSHub - Software Alternatives and Reviews
  • onnxruntime

    ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

  • Models larger than 2 GB are still not officially supported in ONNX. So if you want to get your large model hosted on AWS efficiently (CPU) you'll still have to do all the quantization (employing the workaround discussed in that issue) and AWS deployment yourself.

  • WorkOS

    The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

    WorkOS logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts