Onnxruntime Alternatives
Similar projects and alternatives to onnxruntime
-
-
-
OPS
OPS - Build and Run Open Source Unikernels. Quickly and easily build and deploy open source unikernels in tens of seconds. Deploy in any language to any cloud.
-
-
-
-
-
-
SonarLint
Deliver Cleaner and Safer Code - Right in Your IDE of Choice!. SonarLint is a free and open source IDE extension that identifies and catches bugs and vulnerabilities as you code, directly in the IDE. Install from your favorite IDE marketplace today.
-
-
AppleNeuralHash2ONNX
Convert Apple NeuralHash model for CSAM Detection to ONNX.
-
-
Torch-TensorRT
PyTorch/TorchScript compiler for NVIDIA GPUs using TensorRT
-
transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
-
-
-
-
-
nn
🧑🏫 50! Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
-
tokenizers
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
-
f-stack
F-Stack is an user space network development kit with high performance based on DPDK, FreeBSD TCP/IP stack and coroutine API.
-
Reviews and mentions
-
💡 What's new in txtai 4.0
txtai supports generating vectors with Hugging Face Transformers, PyTorch, ONNX and Word Vector models.
-
Inference machine learning models in the browser with JavaScript and ONNX Runtime Web
ONNX Runtime GitHub
-
PyTorch 1.10
Any recommendations about how to use a PyTorch trained model for inference? Is it best to load it up with PyTorch directly, or convert to ONNX and use ONNX-runtime [1] instead? This seems to be the required way at least if you want to TensorRT the model. I appreciate this is a very general question.
-
Export and run other machine learning models
txtai primarily has support for Hugging Face Transformers and ONNX models. This enables txtai to hook into the rich model framework available in Python, export this functionality via the API to other languages (JavaScript, Java, Go, Rust) and even export and natively load models with ONNX.
-
[D] ONNX ecosystem
The ONNX runtime provides a common serialization format for machine learning models. ONNX supports a number of different platforms/languages and has features built in to help reduce inference time.
-
Portability of Rust in 2021
Also briefly tried https://github.com/microsoft/onnxruntime
-
To all C++ professionals, can you state what field you're working in? Is it a niche?
Artificial intelligence applied to food processing and packaging. I am currently working on real-time automatic quality controls. Practically speaking, this involves high-performance image grabbing + computer vision (mostly with OnnxRuntime).
-
AppleNeuralHash2ONNX: Convert Apple NeuralHash Model for CSAM Detection to ONNX
ONNX != onnxjs
ONNX is a representation format for ML models (mostly neural networks). onnxjs is a just a browser runtime for ONNX models.
While it may be true that onnxjs is neglected, please note that the 'main' runtime, onnxruntime, is under heavy active development[1].
Moreover, Microsoft is not the sole steward of the ONNX ecosystem. They are one of the many contributors, alongside companies like Facebook, Amazon, Nvidia, and many others [2].
I don't think ONNX is going away anytime soon. Not so sure about the TF ecosystem though.
-
[D] Huggingface finally get a stable inference serving env for their 10k+ models
Models larger than 2 GB are still not officially supported in ONNX. So if you want to get your large model hosted on AWS efficiently (CPU) you'll still have to do all the quantization (employing the workaround discussed in that issue) and AWS deployment yourself.
-
John Snow Labs Spark-NLP 3.1.0: Over 2600+ new models and pipelines in 200+ languages, new DistilBERT, RoBERTa, and XLM-RoBERTa transformers, support for external Transformers, and lots more!
It might be worth having a look at the ONNX Runtime Java API in addition to TF-Java, it'll let you deploy the rest of the HuggingFace pytorch models that don't have TF equivalents. I built the Java API a few years ago, and it's now a supported part of the ONNX Runtime project. We use it in Tribuo to provide one of our text feature embedding classes (BERTFeatureExtractor).
-
[D] PyTorch to Desktop App Deployment
LibTorch is pretty heavyweight to ship with desktop application and not so fast for CPU inference, but I have positive experience with onnxruntime:
-
Show HN: Deploy ML Models on a Budget
And ONNX helps too - https://github.com/microsoft/onnxruntime
Stats
microsoft/onnxruntime is an open source project licensed under MIT License which is an OSI approved license.
Popular Comparisons
Are you hiring? Post a new remote job listing for free.