DirectML VS onnx

Only reason I am using the DirectML fork of Automatic1111 is because I am on Windows and pytorch hasn't caught up to RocM 6.

DirectML is fully supported path on Windows and is support by Microsoft et al. (https://github.com/microsoft/DirectML).

Everyone is moving off Cuda as quickly as possible not because the other are better, per se, but because it is easier and cheaper.

See: https://github.com/microsoft/DirectML/issues/400

ONNX (https://onnx.ai/ https://github.com/onnx/onnx) is an alternative to the basic CUDA model, using Direct-ML ( https://learn.microsoft.com/en-us/windows/ai/directml/dml-intro https://github.com/microsoft/DirectML), which is a microsoft-backed open approach. That is what has allowed AMD cards, even slightly older ones, to join in on the machine learning fun.

It's really shocking that AMD fails to extend support natively.

Workarounds such as DirectML claim to be the answer in unifying people with NVIDIA or AMD GPUs, but thus far it hasn't, with issues such as [this](https://github.com/microsoft/DirectML/issues/58) constantly popping up.

As nicolaslem points out, Arch does have community packages for ROCm, but that, unsurprisingly fails to lend support to many consumer GPUs. The best community support I have come across are [rocm-opencl](https://copr.fedorainfracloud.org/coprs/mystro256/rocm-openc... [rocm-hip](https://copr.fedorainfracloud.org/coprs/mystro256/rocm-hip/) for Fedora maintained by [mystro256](https://github.com/Mystro256), who is a single AMD employee.Thanks to him, my AMD GPU (Radeon 6800XT) hasn't completely gone to waste, and I was able to tinker with some things (Gaming isn't really up my alley).

Lately however, after beginning to work on DGX V100s and A100s, and using my older laptop with a GTX 1650, it was apparent how simple setting up CUDA was, and how easily I could experiment with it on my consumer card. Many have spoken about similar stories, and here's mine. Really hope AMD does a whole lot more, and doesn't exclusively keep their powerful GPUs for gaming.

You can use a NVidia gpu or the cpu to do the training but the cpu training is very very slow. For AMD graphics cards like the AMD Radeon VII the only solution is pytorch_directml but unfortunately there appears to be a bug that stops it working nn-morse and torch-directml memory leak? · Issue #355

Github: https://github.com/microsoft/DirectML

Once your model has been trained and validated using Anomalib, the next step is to prepare it for real-time implementation. This is where ONNX (Open Neural Network Exchange) or OpenVINO (Open Visual Inference and Neural network Optimization) comes into play.

ONNX is an open format from the Linux Foundation to represent machine learning models. It is becoming extensively adopted by the Machine Learning community and is compatible with most of the machine learning frameworks like PyTorch, TensorFlow, etc. Converting a model between any of those formats and ONNX is really simple and can be done in most cases with a single command.

ONNX[0], model-as-protosbufs, continuing to gain adoption will hopefully solve this issue.

[0] https://github.com/onnx/onnx

Running inference for large models on CPU is not a new problem and fortunately there has been great development in many different optimization frameworks for speeding up matrix and tensor computations on CPU. We explored multiple optimization frameworks and methods to improve latency, namely TorchScript, BetterTransformer and ONNX.

ONNX is a format for representing machine learning models in a portable way. Additionally, ONNX models can be easily optimized and thus become smaller and faster.

I would say onnx.ai [0] provides more information about ONNX for those who aren’t working with ML/DL.

[0] https://onnx.ai

It's not clear why you thing this sub is appropriate for some third party system with a Python interface. Why don't you try their discussion group: https://github.com/onnx/onnx/discussions

This kind of indirection through standardisation is pretty common to make compatibility between different kinds of software components easier. Some other good examples are the LSP project from Microsoft and ONNX to represent machine learning models. The first provides a standard so that IDEs don't have to re-invent the weel for every programming language. The latter decouples training frameworks from inference frameworks. Going back to WSGI, you can find a pretty extensive rationale for the WSGI standard here if interested.