XNNPACK
AITemplate
| XNNPACK | AITemplate | |
|---|---|---|
| 8 | 37 | |
| 1,700 | 4,455 | |
| 1.6% | 0.7% | |
| 9.9 | 8.7 | |
| 6 days ago | 4 days ago | |
| C | Python | |
| GNU General Public License v3.0 or later | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
XNNPACK
- Xnnpack: High-efficiency floating-point neural network inference operators
- Can a NPU be used for vectors?
-
Performance critical ML: How viable is Rust as an alternative to C++
Why are you writing your own inference code in C++ or Rust instead of using some kind of established framework like XNNPACK?
- [P] Pure C/C++ port of OpenAI's Whisper
-
[Discussion] Is XNNPACK a part of mediapipe? or should be additionally configured with mediapipe?
XNNPACK - https://github.com/google/XNNPACK
- WebAssembly Techniques to Speed Up Matrix Multiplication by 120x
-
Prediction: Macs won't see many new games, no matter how powerful their hardware is
Ok, concrete example time! At work, we're going to be using some software which includes XNNPACK, which is a library of highly-optimised operations for doing neural-network inference. This is the sort of thing where people have gone in and specifically tuned for performance, and nope, there's no attempt at all made to have code which is different for Intel/AMD or Apple/Other ARM. What they target is elements of the ISA, like NEON (i.e. ARM SIMD) and SSE, AVX etc. on x86(-64). And Wasm SIMD for Wasm.
-
Where are Nvidia's DLSS models stored and how big are they?
It's quite simple. https://github.com/google/XNNPACK for example.
AITemplate
-
Show HN: Shortbread, a web app that helps you create AI comics in minutes
VoltaML is a relatively vanilla diffusers-based backend, so its not a hairy monster to hack like you may have seen with SAI-based UIs.
The AITTemplate code is a lightly modified version of Facebook's example, code, to get rid of small issues like VRAM spikes: https://github.com/facebookincubator/AITemplate/tree/main/ex...
InvokeAI is also diffusers based, but they seem to mess with the pipeline a bit more.
And anyway, all that may be a better reference for interesting features rather than a backend to try and adopt.
-
List of all the ways to improve performance for stable diffusion.
let me know if you discover any more ways to improve SD. I am currently looking into facebooks AITemplate : https://github.com/facebookincubator/AITemplate
- [R] AITemplate Python to AMD compiler {META}
-
Nearly 2x speedup for SD rendering using AITemplate
Link to AITemplate itself: https://github.com/facebookincubator/AITemplate
- Render a neural network into CUDA/HIP code
- Render neural network into CUDA/HIP code
- AITemplate: a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.
- A1111 vs Olive vs AITemplate.
What are some alternatives?
ncnn - ncnn is a high-performance neural network inference framework optimized for the mobile platform
stable-diffusion-webui - Stable Diffusion web UI
gemm-benchmark - Simple [sd]gemm benchmark, similar to ACES dgemm
nebuly - The user analytics platform for LLMs
cpuid2cpuflags - Tool to generate CPU_FLAGS_* for your CPU
xformers - Hackable and optimized Transformers building blocks, supporting a composable construction.
wasmblr - C++ WebAssembly assembler in a single header file
voltaML - ⚡VoltaML is a lightweight library to convert and run your ML/DL deep learning models in high performance inference runtimes like TensorRT, TorchScript, ONNX and TVM.
Genann - simple neural network library in ANSI C
stable-diffusion-tensorflow - Stable Diffusion in TensorFlow / Keras
ruby-fann - Ruby library for interfacing with FANN (Fast Artificial Neural Network)
rocm-gfx803