DistIL VS ILGPU

How is that relevant to the vast majority of the code targeted by LINQ?

The niche scenario you have outlined is partially covered by a recent System.Numerics.Tensors package update (even though I believe it would have been best if there was a community-maintained package with comparable quality that can be decoupled from .NET release cycle and trade-off support/compat guarantees in favour of more aggressive improvements in the future).

The goal of LINQ itself is to offer optimal codepaths when it can within the constraints of the current design (naturally, you could improve it significantly if not for backwards compatibility with the previous 15 or so years of .NET codebases). The argument that it's not good because it's not the tool to do BLAS is just nonsensical.

There is, however, an IL optimizer that can further vectorize certain common patterns and rewrite LINQ calls into open-coded loops: https://github.com/dubiousconst282/DistIL

Repo: https://github.com/dubiousconst282/DistIL

The lifetime problem should not be that complicated, because the class/lambda allocation is contained entirely within the method. There's an open issue on the runtime repo about object stack allocation that would on paper alleviate some of these issues, but it's quite stagnated. In my toy optimizer, I made a buggy pattern matching transform for "concretizing" lambdas invocations using very little code, but implementing something like this in the JIT can be quite challenging, based on my limited experience.

I've had a good experience with ILGPU: clean API, loads of samples, nice community. Apologies for a shameless plug, but I used it in one of my projects and happened to write a blog post about it: https://timiskhakov.github.io/posts/computing-the-convex-hull-on-gpu. Hope it helps!

https://github.com/Sergio0694/ComputesSharp is always being recommended to me. But I also just found this one https://github.com/m4rs-mt/ILGPU which looks very interesting. There are a lot of libraries which allow you to execute on the gpu

I would checkout the github for more details, or ask on the discord for more specifics, but all the kernels are compiled into IL by the C# compiler, then at runtime the ILGPU compiler converts them from IL into PTX, OpenCL, or back into IL (in a special way to maintain thread grouping and stuff). Then PTX / OpenCL /IL is compiled and run using the respective runtimes. Cuda for PTX, the OpenCL runtime for OpenCL, and .net for IL. We have talked about creating a CPU execution path that tries to match speeds with CPU code, but I do not think it is a big priority.

Github repo