XNNPACK VS Awesome-Rust-MachineLearning

Why are you writing your own inference code in C++ or Rust instead of using some kind of established framework like XNNPACK?

XNNPACK - https://github.com/google/XNNPACK

Ok, concrete example time! At work, we're going to be using some software which includes XNNPACK, which is a library of highly-optimised operations for doing neural-network inference. This is the sort of thing where people have gone in and specifically tuned for performance, and nope, there's no attempt at all made to have code which is different for Intel/AMD or Apple/Other ARM. What they target is elements of the ISA, like NEON (i.e. ARM SIMD) and SSE, AVX etc. on x86(-64). And Wasm SIMD for Wasm.

It's quite simple. https://github.com/google/XNNPACK for example.

There’s an awesome-git list for a bunch of ML rust stuff not sure how up to date it is as well https://github.com/vaaaaanquish/Awesome-Rust-MachineLearning … not mine

I am looking for something like [Triton Inference Server](https://github.com/triton-inference-server/server) or [TFX Serving](https://www.tensorflow.org/tfx/guide/serving), but in Rust. I came across [Orkon](https://github.com/vertexclique/orkhon) which seems to be dormant and a bunch of examples off of the [Awesome-Rust-MachineLearning](https://github.com/vaaaaanquish/Awesome-Rust-MachineLearning)

There's also https://github.com/vaaaaanquish/Awesome-Rust-MachineLearning

I saw Awesome-Rust-MachineLearning. It have something replacements from Python.