ArrayFire VS mdspan

Loads of people have stated why easy GPU interfaces are difficult to create, but we solve many difficult things all the time.

Ultimately I think CPUs are just satisfactory for the vast vast majority of workloads. Servers rarely come with any GPUs to speak of. The ecosystem around GPUs is unattractive. CPUs have SIMD instructions that can help. There are so many reasons not to use GPUs. By the time anyone seriously considers using GPUs they're, in my imagination, typically seriously starved for performance, and looking to control as much of the execution details as possible. GPU programmers don't want an automagic solution.

So I think the demand for easy GPU interfaces is just very weak, and therefore no effort has taken off. The amount of work needed to make it as easy to use as CPUs is massive, and the only reason anyone would even attempt to take this on is to lock you in to expensive hardware (see CUDA).

For a practical suggestion, have you taken a look at https://arrayfire.com/ ? It can run on both CUDA and OpenCL, and it has C++, Rust and Python bindings.

What about Arrayfire. https://github.com/arrayfire/arrayfire

Low-overhead — not our goal, but Flashlight is on par with or outperforming most other ML/DL frameworks with its ArrayFire reference tensor implementation, especially on nonstandard setups where framework overhead matters

Looking for frameworks with Julia + OpenCL I found array fire. It seems quite good, bonus points for rust bindings. I will keep looking for more, Julia completely fell off my radar.

Your Problem may be the lazy evaluation, see this issue: https://github.com/arrayfire/arrayfire/issues/1709

Why not use https://github.com/kokkos/mdspan though ?

It can be the base of whatever *you* write via bindings generators like pybind11. In that sense, the answer to your question is "however you like". For actual simulation code, you'll see a lot more legacy Fortran and C. That said, with things like mdspan maybe being standardized (proposal), efforts towards a standard linear algebra library, and the existence of ubiquitous HPC frameworks already having been written in C++, I would say it's only a matter of time before C++ accounts for an even bigger share of all HPC code.

I'm aware of only two relevant projects myself, I don't know much, came to reddit kind of by chance. One of the multi-dimensional array libraries proposed for potential standardisation, and a gnu machine learning library that was discontinued which could be worked off of. There's probably a lot more out there, but don't get distracted from making something awesome :)

As u/IyeOnline already made the important points about VLAs and std::vector, I would just add that you may find std::mdspan to be a helpful data structure. You can allocate 1d memory and give it a 2d shape of k with nice 2d indexing, eg auto& elem = mymdspan(row, col);.

Kokkos mdspan

The good thing here is that it heavily relies on mdpsan that is a multidimensional view that handle shape and strides. And kokkos provide a C++14 compatible implementation with a complete CUDA support.