HIPIFY VS intel-extension-for-pytorch

Right. I can't speak to its correctness/completeness as I've only done a quick installation and smoke test of the ROCm/HIP/MIOpen stack, but there's even a tool that automates the translation [1].

[1] https://github.com/ROCm-Developer-Tools/HIPIFY

From https://news.ycombinator.com/item?id=32904285 re: AMD Rocm, HIPIFY, :

>> ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY :

>> hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced.

> ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY :

>> hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced.

> AMD ROcm supports Pytorch, TensorFlow, MlOpen, rocBLAS on NVIDIA and AMD GPUs: https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni...

> Thus, the idea is that through typically negligible effort porting to HiP, your code becomes vendor-independent.

Here, the big AMD mistake was to rename those function prefixes in the first place. It's a mistake that they could have avoided...

What a lot of SW codebases did to support AMD (see PyTorch code notably): codebase is still CUDA, have the conversion pass to HIP done at build time.

See https://github.com/ROCm-Developer-Tools/HIPIFY/blob/amd-stag... for the Perl script to do it.

Then comes the problem of AMD not supporting ROCm HIP on most of their hardware or user base.

On Windows, the ROCm HIP SDK is private and only available under NDA. This means that while you can use Blender w/ HIP on Windows, the Blender builds that you compile yourself will not be able to use ROCm HIP.

On Linux, the supported GPUs are few and far between, Vega20 onwards are supported today. APUs, RDNA1, and lower end RDNA2 w/o unsupported hacks (6700 XT and below) are excluded.

https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni...

RadeonOpenCompute/ROCm_Documentation: https://github.com/RadeonOpenCompute/ROCm_Documentation

ROCm-Developer-Tools/HIPIFYhttps://github.com/ROCm-Developer-Tools/HIPIFY :

> hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced.

ROCmSoftwarePlatform/gpufort: https://github.com/ROCmSoftwarePlatform/gpufort :

> GPUFORT: S2S translation tool for CUDA Fortran and Fortran+X in the spirit of hipify

ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP:

> HIP is a C++ Runtime API and Kernel Language that allows developers to create portable applications for AMD and NVIDIA GPUs from single source code. [...] Key features include:

> - HIP is very thin and has little or no performance impact over coding directly in CUDA mode.

> - HIP allows coding in a single-source C++ programming language including features such as templates, C++11 lambdas, classes, namespaces, and more.

> - HIP allows developers to use the "best" development environment and tools on each target platform.

> - The [HIPIFY] tools automatically convert source from CUDA to HIP.

> - * Developers can specialize for the platform (CUDA or AMD) to tune for performance or handle tricky cases.*

并且有一个自动转换工具 https://github.com/ROCm-Developer-Tools/HIPIFY https://rocmdocs.amd.com/en/latest/Programming_Guides/HIP-porting-guide.html

It might be worth checking out HIPIFY, which lets you automatically convert CUDA code to vendor neutral code that can be run on any GPU. Disclaimer, I have never used it and have no idea how it works.

CUDA zu AMD HIP conversion: https://github.com/ROCm-Developer-Tools/HIPIFY

I have been playing around in pytorch with an a770 16GB card and hit this error. The response seems to be https://github.com/intel/intel-extension-for-pytorch/issues/... that larger than 4gb allocations aren't supported even though the card is 16gb. I haven't seen a ton of stuff on intel arc for machine learning so wanted to share my experience

OK I found it. Looks like they use SYCL (which for some reason they've rebranded to DPC++): https://github.com/intel/intel-extension-for-pytorch/tree/v2...

Just to point out it does, kind of: https://github.com/intel/intel-extension-for-pytorch

I've asked before if they'll merge it back into PyTorch main and include it in the CI, not sure if they've done that yet.

Intel already has a working GPGPU stack, using oneAPI/SYCL.

They also have arguably pretty good OpenCL support, as well as downstream support for PyTorch and Tensorflow using their custom extensions https://github.com/intel/intel-extension-for-tensorflow and https://github.com/intel/intel-extension-for-pytorch which are actively developed and just recently brought up-to-date with upstream releases.

https://github.com/intel/intel-extension-for-pytorch :

> Intel® Extension for PyTorch extends PyTorch* with up-to-date features optimizations for an extra performance boost on Intel hardware. Optimizations take advantage of AVX-512 Vector Neural Network Instructions (AVX512 VNNI) and Intel® Advanced Matrix Extensions (Intel® AMX) on Intel CPUs as well as Intel Xe Matrix Extensions (XMX) AI engines on Intel discrete GPUs. Moreover, through PyTorch* xpu device, Intel® Extension for PyTorch* provides easy GPU acceleration for Intel discrete GPUs with PyTorch*

https://pytorch.org/blog/celebrate-pytorch-2.0/ :

> As part of the PyTorch 2.0 compilation stack, TorchInductor CPU backend optimization brings notable performance improvements via graph compilation over the PyTorch eager mode.

The TorchInductor CPU backend is sped up by leveraging the technologies from the Intel® Extension for PyTorch for Conv/GEMM ops with post-op fusion and weight prepacking, and PyTorch ATen CPU kernels for memory-bound ops with explicit vectorization on top of OpenMP-based thread parallelization*

DLRS Deep Learning Reference Stack: https://intel.github.io/stacks/dlrs/index.html

Install intel extensions for pytorch using docker. https://github.com/intel/intel-extension-for-pytorch

There is: https://github.com/intel/intel-extension-for-pytorch for intel cards on GPUs, but I would assume this doesn't extend to integraded graphics

Nonetheless, this issue might be relevant for your case.