ArrayFire VS moodycamel

Loads of people have stated why easy GPU interfaces are difficult to create, but we solve many difficult things all the time.

Ultimately I think CPUs are just satisfactory for the vast vast majority of workloads. Servers rarely come with any GPUs to speak of. The ecosystem around GPUs is unattractive. CPUs have SIMD instructions that can help. There are so many reasons not to use GPUs. By the time anyone seriously considers using GPUs they're, in my imagination, typically seriously starved for performance, and looking to control as much of the execution details as possible. GPU programmers don't want an automagic solution.

So I think the demand for easy GPU interfaces is just very weak, and therefore no effort has taken off. The amount of work needed to make it as easy to use as CPUs is massive, and the only reason anyone would even attempt to take this on is to lock you in to expensive hardware (see CUDA).

For a practical suggestion, have you taken a look at https://arrayfire.com/ ? It can run on both CUDA and OpenCL, and it has C++, Rust and Python bindings.

What about Arrayfire. https://github.com/arrayfire/arrayfire

Low-overhead — not our goal, but Flashlight is on par with or outperforming most other ML/DL frameworks with its ArrayFire reference tensor implementation, especially on nonstandard setups where framework overhead matters

Looking for frameworks with Julia + OpenCL I found array fire. It seems quite good, bonus points for rust bindings. I will keep looking for more, Julia completely fell off my radar.

Your Problem may be the lazy evaluation, see this issue: https://github.com/arrayfire/arrayfire/issues/1709

Other than that... if your stuff is good, that's a much better signal than a professional username. I've seen a lot of decently unprofessional usernames out there that get taken pretty seriously because of the good work behind them. My recent favorite is "moodycamel" who authored a great concurrent queue library in C++.

In c++ the moodycamel ConcurrentQueue is a good choice.

Here's a decent concurrent queue: moodycamel::ConcurrentQueue.

Lock free programming is hard, and probably harder than you think. I would not even try something like that myself. I would look for existing solutions, something like https://github.com/cameron314/concurrentqueue for example.

I needed a concurrent queue that would block when attempting to pop an empty queue, which allows the consuming thread to suspend while it's waiting for work. I found that using mutexes allowed me to develop a simple template adapter had several advantages with few drawbacks when compared to non-blocking queues: it can use a variety of containers, the code can be reviewed and verified as to its correctness (very hard to do with fancy concurrent programming that avoids mutexes), and it is only slightly slower than fancier solutions (when I benchmarked it originally, it was 4x slower than Moody Camel's concurrent queue, which to me is fine performance).

Not literatue but an example. This is a lock-free (not wait-free!) multi-producer multi-consumer queue, not a FIFO, but access patterns should be similar - if not the same: https://github.com/cameron314/concurrentqueue

If I do implement it, the most likely route I'd take is make a compiler in Clojure/clojurescript that uses Instaparse (I have a more-or-less-clojure grammar written that I was tinkering with) and generate C++ code that uses Immer for its data structures and Zug for transducers and what my not-quite-clojure would support would be heavily dependent on what the C++ code and libraries I use can do. I'd use Taskflow to implement a core.async style system (not sure how to implement channels, maybe this but I'm unsure if its a good fit, but I also haven't looked). I would ultimately want to be able to interact with C++ code, so having some way to call C++ classes (even templated ones) would be a must. I'm unsure if I would just copy (and extend as needed) Clojure's host interop functionality or not. I had toyed with the idea that you can define the native types (including templates) as part of the type annotations and then the user-level code basically just looks like a normal function. But I didn't take it very far yet, haven't had the time. The reason I'd take this approach is that I'm writing a good bit of C++ again and I'd love to do that in this not-quite-clojure language, if I did make it. A bunch of languages, like Haxe and Nim compile to C or C++, so I think its a perfectly reasonable approach, and if interop works well enough, then just like Clojure was able to leverage the Java ecosystem, not-quite-clojure could be bootstrapped by leveraging the C++ ecosystem. But its mostly just a vague dream right now.

I would recommend https://github.com/cameron314/concurrentqueue as it's very battle tested and fast.

This was explicitly considered for spdlog (using the moodycamel::ConcurrentQueue) but rejected for the above reason. I'm not involved in the development of spdlog but personally I agree, for me it's important that log output is not all mixed up.