XNNPACK
HIP-CPU
| XNNPACK | HIP-CPU | |
|---|---|---|
| 8 | 5 | |
| 1,700 | 105 | |
| 1.6% | 2.9% | |
| 9.9 | 7.2 | |
| 6 days ago | about 2 months ago | |
| C | C++ | |
| GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
XNNPACK
- Xnnpack: High-efficiency floating-point neural network inference operators
- Can a NPU be used for vectors?
-
Performance critical ML: How viable is Rust as an alternative to C++
Why are you writing your own inference code in C++ or Rust instead of using some kind of established framework like XNNPACK?
- [P] Pure C/C++ port of OpenAI's Whisper
-
[Discussion] Is XNNPACK a part of mediapipe? or should be additionally configured with mediapipe?
XNNPACK - https://github.com/google/XNNPACK
- WebAssembly Techniques to Speed Up Matrix Multiplication by 120x
-
Prediction: Macs won't see many new games, no matter how powerful their hardware is
Ok, concrete example time! At work, we're going to be using some software which includes XNNPACK, which is a library of highly-optimised operations for doing neural-network inference. This is the sort of thing where people have gone in and specifically tuned for performance, and nope, there's no attempt at all made to have code which is different for Intel/AMD or Apple/Other ARM. What they target is elements of the ISA, like NEON (i.e. ARM SIMD) and SSE, AVX etc. on x86(-64). And Wasm SIMD for Wasm.
-
Where are Nvidia's DLSS models stored and how big are they?
It's quite simple. https://github.com/google/XNNPACK for example.
HIP-CPU
- HIP CPU
- [P] Pure C/C++ port of OpenAI's Whisper
-
AMD publishes GPUFORT as Open Source to address CUDA’s dominance
If I'm reading this right, this is Fortran's equivalent of HIP, i.e. a way to (semi-)automatically convert CUDA-based solution to a more backend-independent one so that the same source can be run both on CUDA and ROCm GPUs (and potentially more; e.g. they also have an experimental CPU backend).
-
Test Coverage with CUDA
So, I know that you asked about cuda, but this might actually be possible in hip, and you can convert your code to hip relatively easily. The path would be to use the CPU implementation (https://github.com/ROCm-Developer-Tools/HIP-CPU) and then run your code coverage on that.
What are some alternatives?
ncnn - ncnn is a high-performance neural network inference framework optimized for the mobile platform
AdaptiveCpp - Implementation of SYCL and C++ standard parallelism for CPUs and GPUs from all vendors: The independent, community-driven compiler for C++-based heterogeneous programming models. Lets applications adapt themselves to all the hardware in the system - even at runtime!
gemm-benchmark - Simple [sd]gemm benchmark, similar to ACES dgemm
libcudacxx - [ARCHIVED] The C++ Standard Library for your entire system. See https://github.com/NVIDIA/cccl
cpuid2cpuflags - Tool to generate CPU_FLAGS_* for your CPU
rocFFT - Next generation FFT implementation for ROCm
wasmblr - C++ WebAssembly assembler in a single header file
HIP - HIP: C++ Heterogeneous-Compute Interface for Portability
Genann - simple neural network library in ANSI C
stdgpu - stdgpu: Efficient STL-like Data Structures on the GPU
ruby-fann - Ruby library for interfacing with FANN (Fast Artificial Neural Network)
AITemplate - AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.