RAJA VS kokkos

Since the ability to use C++ parallel algorithms on the GPU is a relatively new thing, some applications have used other C++ abstraction libraries instead, such as Kokkos (https://kokkos.org/) and RAJA (https://github.com/LLNL/RAJA). These both have multiple backends that support GPUs and CPUs without needing to change your application code.

I’m a novice user of Kokkos. Write code once for openmp, CUDA, and other parallel execution backends. It was designed with scientific computing applications in mind. Some numerics tools are implemented in “Kokkos kernels”, most of the BLAS operations are included iirc.

I would suggest using a C++ abstraction around thread parallelism. This will make your code easier to read and more concise, and will also make it easier to switch between different thread-parallel programming models. Kokkos is a lovely example of such an abstraction, but there are others. Modern C++ even has thread-parallel standard algorithms. Bryce Adelstein Lelbach's CppCon 2021 talk describes these.

It’s a work in progress, but Kokkos and the associated Kokkos Kernels are probably the closest thing to what you’re asking for.

Kokkos implements a programming model in C++ for writing performance portable applications targeting all major HPC platforms. It provides abstractions for both parallel execution of code and data management with a variety of backends including, but not limited to: CUDA, HIP, OpenMP, HPX, and Pthreads, with backends for OpenMPTarget and SYCL currently under development.