kompute VS clvk

The two I know of are IREE and Kompute[1]. I'm not sure how much momentum the latter has, I don't see it referenced much. There's also a growing body of work that uses Vulkan indirectly through WebGPU. This is currently lagging in performance due to lack of subgroups and cooperative matrix mult, but I see that gap closing. There I think wonnx[2] has the most momentum, but I am aware of other efforts.

[1]: https://kompute.cc/

[2]: https://github.com/webonnx/wonnx

Great to see the positive momentum of this framework! Best wishes and upvotes from the Vulkan Kompute team :)

To a first approximation, Kompute[1] is that. It doesn't seem to be catching on, I'm seeing more buzz around WebGPU solutions, including wonnx[2] and more hand-rolled approaches, and IREE[3], the latter of which has a Vulkan back-end.

[1]: https://kompute.cc/

[2]: https://github.com/webonnx/wonnx

[3]: https://github.com/openxla/iree

I look in an example and see similar instructions, stating that the build should be quite simple. But again, it doesn't work. It generates a bunch of folders with Visual Studio stuff, but no executables, no libraries, or anything like that.

I can't figure out how, and there are no tutorials. According to https://kompute.cc/overview/build-system.html I should simply run "cmake -Bbuild". But this doesn't output what I need, and when I look in the Makefile I get the sense that this is more an example Makefile... but then that contradicts the above tutorial.

Kompute is in my opinion good example to take inspiration for abstractions.

Currently, there's a few Vulkan compute frameworks floating around (like Kompute). I would work with those. Kompute simplifies a lot of the biolerplate and seems like you could benefit from using it.

Try Kompute, a project from the Linux foundation. It is quite simple to use, and does not require deep knowledge of graphics API. It’s a bit painful to setup, but it kinda works well (and I have a project going on on it)

This so much.

This makes alot of sense considering the Linux Foundation is also in charge of Kompute which is likely to be the basis of vendor agnostic GPGPU, and thus the basis of vendor agnostic GPU-based machine learning.

I tried very similar thing. My purpose was to run llama-cpp-python with CLBlast GPU acceleration via clvk on VulkanSDK on my M1 Max computer. I downloaded VulkanSDK for macOS, cloned clvk(https://github.com/kpet/clvk) and CLBlast. Build was successful but there is a problem; when clCreateCommandQueue function was called with CL_QUEUE_OUT_OF_ORDER_EXEC_MODE_ENABLE option(in ggml-opencl.c of llama.cpp) , an error was happened and I do not know how to handle it.

OpenCL really is your best bet for a cross-platform GPU-capable framework. OpenCL 3.0 cleared out a lot of the cruft from OpenCL 2.x so it's seeing a lot more adoption. The most cross-platform solution is still OpenCL 1.2, largely for MacOS, but OpenCL 3.0 is becoming more and more common for Windows and Linux and multiple devices. Even on platforms without native OpenCL support there are compatibility layers that implement OpenCL on top of DirectX (OpenCLOn12) or Vulkan (clvk and clspv).

What we really need is CLVK, but it seems pretty limited. I'd prefer a clang based compiler which can accept opencl c++ personally, because a brand new compiler is not ideal