cudf VS CUDA.jl

cuDF Documentation

cudf – GPU DataFrame Library

The interesting thing about Polars is that it does not try to be a drop-in replacement to pandas, like Dask, cuDF, or Modin, and instead has its own expressive API. Despite being a young project, it quickly got popular thanks to its easy installation process and its “lightning fast” performance.

Perhaps the largest for package size is the NVIDIA developed rapids toolkit https://rapids.ai/ . Even still adding things like pandas and some geospatial tools, you rapidly end up with an image well over a gigabyte, despite following cutting edge best practice with docker and python.

Yes sure, that is how OpenMP does; but on the other side: you seem to already do some basic type inference, and building an AST, no? Then you know as well the size and type of your vectors, and can execute actions in parallel if there is enough data to be worth parallelizing. Is there anyone who don't want their code to execute faster if it is possible? Those that do work in big data domain do use threads and vectorized instructions without user having to type in any directive; just import different library. Example, numpy or numpy with cuda backend, or similar GPU accelerated libraries like cudf.

Not the answer about Ray: you could use rapids.ai. I'm using it for for dataframe manipulation on GPU

To put Data Analytics on GPU Steroids, Try RAPIDS cudf https://rapids.ai/

You can scope out https://rapids.ai/. Nvidia's AI toolkits. They have some handy notebooks to poke at to get you started.

It would also mean learning Julia, but you can write GPU kernels in Julia and then compile for NVidia CUDA, AMD ROCm or IBM oneAPI.

https://juliagpu.org/

I've written CUDA kernels and I knew nothing about it going in.

See https://juliagpu.org/

If you just want to do some numerical code that requires linear algebra and GPU, your best bet would be Julia or Python+JAX.

GitHub - JuliaGPU/CUDA.jl: CUDA programming in Julia.

Should have (2017) in the title.

Indeed cool to program julia directly on the GPU and Julia on GPU and this has further evolved since then, see https://juliagpu.org/

I'm excited to eventually see something like JuliaGPU with support for multiple backends.

That's really cool! Have you looked at CUDA.jl for the Julia language? Maybe you could take some ideas from there. I am pretty sure it does the same thing you do here, and they support any arbitrary code with the limitations that you cannot allocate memory, I/O is disallowed, and badly-typed code(dynamic) will not compile.